ML24149A389
| ML24149A389 | |
| Person / Time | |
|---|---|
| Site: | Braidwood  |
| Issue date: | 04/19/2024 |
| From: | NRC/RGN-III/DORS/OB |
| To: | Exelon Generation Co |
| James Nance | |
| References | |
| Download: ML24149A389 | |
Text
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
1 H
3 B
S 2019 Braidwood ILE - Question #2 2
F 3
N S
NRC: No difficulty given by licensee (see General Comment #4 3
F 3
N E
S NRC: Suggest changing 4th bullet in stem to +60 degrees subcooling is indicated on Iconics Display to increase level of difficulty.
Response: The question is testing the subcooling requirement of 1BwEP ES-0.2 Operator Action Summary. The subcooling requirement is an acceptable Iconic display OR per Attachment A, using the normal containment curve (figure 1BwEP ES 0.2-1). This curve only requires approximately 20-30 degrees of subcooling (depending upon RCS pressure, but above 1880 psig) during the initial RCS cooldown. The +80 degrees subcooling curve (figure 1BwEP ES 0.2-2) is not applicable until step 15, when the RCS is being prepared for depressurization below 1880 psig. Therefore, changing the fourth stem bullet to +60 degrees subcooling would make subcooling acceptable during performance of the procedure prior to step 15 and not require any operator action.
Additionally, the +80 degrees subcooling curve is not consistently 80 degrees above the saturation curve. Rather it represents a curve that is 80 degrees above the normal containment acceptable subcooling curve. Therefore, changing the fourth bullet as suggested would make the question unable to answer correctly without giving the
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
candidate additional information about where the crew was in the procedure performance and would be beyond the scope of required RO knowledge (OAS information).
4 H
3 N
S NRC: Tier 1 because the applicant needs to know the mitigation strategy of the procedure. Knowing which valve closed due to a Phase A on 1B Train is a systems knowledge piece.
Response: None required.
5 F
2 X
B U
S Byron Exam Bank - Question 6 on Byron 2023 Exam NRC: 1) The question does not address the K/A. Nothing in the question addresses shifting to an alternate flowpath as it applies to the Loss of the Residual Heat Removal System.
- 2) This is memorization of what is in the table and does not meet the Discriminatory Validity bar.
Accepted as is.
Response: The K/A and question are bank (identical K/A and question to Q6) from the 2023 Byron NRC exam. Believe the question matches the K/A. The answer is the criteria for deciding which alternative cooling method is chosen in 1BwOA PRI-10. The reason for selecting which alternate cooling flowpath is the current plant mode and configuration. The author searched for a different question that required the candidate to choose the alternate cooling flowpath as a replacement. However, the different question is currently on the 2024 Bwd audit exam and
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
requires a provided reference. Because this question requires knowledge of the reasons for selecting the alternate cooling flowpath at the RO license level: the current plant mode and configuration, and because the question was accepted on the recent 2023 Byron NRC exam, facility requests the current question remain on this exam.
6 H
3 B
S 2009 Braidwood ILE 7
H 3
B S
Modified Byron Exam Bank - Question 2 on Byron 2023 Exam 8
F 3
B S
Byron 2019-2 Exam 9
F 3
N S
10 H
3 N
E S
NRC: 1) Why is the word MAIN included in answer C? 2) By the above, should A include the word Startup?
Response: Removed the word MAIN from c.
11 F
H 2
X N
U S
NRC: 1) This is a systems question, not a Tier 1 question. See General Comments NOTE 1, 2)
Knowing from memory that the 1C and 1D SG PORVs are the only SG PORVs that have a battery backup power supply is a fundamental knowledge level.
Response: Revised the question to make it 2 parts, with the part 1 asking the bases for the procedure step referenced in the question. The
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
basis for this step has been used as Tier 1 question on multiple Braidwood NRC exams, most recently on the 2022 NRC exam. Added CA-0.0 WOG background document to references.
Also, per NUREG 1021, Appendix A: Generally, an HCL test item will require at least two mental stepsone requires the recall of acquired knowledge, and the other requires associating two or more pieces of data. The number and types of mental steps that must be considered are those necessary to rule out plausible incorrect distractors, as well as the steps needed to identify the correct answer. Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
In this question the candidate must know the following to correctly answer the question:
- 1.
What are the normal power supplies to the SG PORV motors and controllers?
- 2.
What is the plant electrical distribution status in 1BwCA-0.0, Attachment B and how does that affect the PORVs.
- 3.
Which of the SG PORVs have a battery backup power supply?
If the question directly asked #3 (Which of the SG PORVs have a battery backup power supply?) then it would be considered LCL: one mental step -
recall. However, because the question requires an analysis of the above information requiring multiple mental steps, the question is written at the HCL.
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
12 H
3
?
N E
S Question ID 2501577 (Free Look)
NRC: 1) Does a LOOP cause a reactor scram?
Therefore, the turbine will trip and you will lose the non-ESF buses. This then challenges B as a credible distractor. 2) Does Braidwood have backup air supply bottles to keep you from immediately losing instrument air?
Response: 1) A LOOP from 100% power does not cause a reactor trip at Braidwood. All non-ESF 4kv and 6.9kv busses that were supplied by off site power (SATs) feed breakers will auto bus transfer (ABT) to the UATs that are being supplied by the main generator. The ABT occurs within 0.1 second which is fast enough to prevent the bus undervoltage relays from actuating. 4kv ESF busses will momentarily de-energize until the DGs auto start and re-energize the busses. 1BwOA ELEC-4, Attachment B contains the required actions to maintain the unit in Mode 1 following a LOOP.
- 2) Braidwood does not have a backup instrument air supply system. Instrument air is supplied from the station air compressors, with stand by compressors auto starting on low station air pressure.
13 F
H 3
B S
Bank from 2020 Bwd Cert Exam Response: In this question the candidate must know of the following to correctly answer the question:
- 1.
What is the automatic function of 1CV8114?
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
- 2.
How does a loss of instrument bus 111 affect the 1CV8114 automatic functions?
- 3.
What are the opening and closing setpoints of 1CV8114?
Because the question requires an analysis of the above information requiring multiple mental steps beyond the recall of a single piece of acquired knowledge, the question is written at the HCL.
Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL 14 H
3
?
N E
S Question ID 2501793 (Free Look)
NRC: 1) Need to look at schematic to understand breakers locations and functions. 2) Consider the Following Both Units are at 100% power.
Annunciator 1-21-E7, 125V DC BATT CHGR FD BRKR TRIP, alarms.
NO other annunciators are currently in alarm.
a)
What action should the crew take?
b)
What, if any Unit 1 annunciator will be in alarm?
Response: 1) Big note DC-1 provided as reference to show DC system diagram.
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
- 2) The proposed alternate question is very close to an SRO only question on the Bwd audit exam and would represent overlap of questions between the audit and NRC exams. The facility would prefer the question remain as is to ensure overlap is not a concern.
15 F
2 X
N U
S NRC: 1) This is a systems question, not a Tier 1 question. See General Comments NOTE 1. 2)
This is a memory question that has little discriminatory value.
Response: Reviewed the question and agree with licensee that it is satisfactory as is.
16 H
3 N
S 17 H
3 N
S 18 F
H 2
B E
S 2019 Braidwood ILE - Question #18 NRC: Part 1 of the question fits the K/A. Part 2 does not fit as well. Given the conditions, what does the procedure suggest as another means of removing decay heat (RHR?).
Response: With the given plant conditions, 1BwFR-H.1 does not offer an alternative for core cooling because 1BwFR-H.1 directs actions to recover the secondary heat sink. In the given conditions, the SGs will not act as a heat sink because RCS pressure is too low. The RCS and core would be cooled by ECCS injection flow.
Therefore, 1BwFR-H.1 directs the crew to
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
immediately return to the previous procedure and step in effect.
For this question the candidate must demonstrate the following to correctly answer the question:
- 1.
Fundamental knowledge of requirements for PWR primary to secondary heat transfer.
- 2.
Ability to apply that knowledge to the given RCS and SG pressures.
- 3.
How a loss of secondary heat sink with the given plant parameters is addressed in 1BwFR-H.1 Because the question requires recall and analysis of the above information requiring multiple mental steps, beyond the recall of a single piece of acquired knowledge, the question is written at the HCL. Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
19 H
3 X
X N
U E
S Question ID 2501856 (Free Look)
NRC: This is not a Tier 1 question. This is a great systems question. No knowledge of the procedure is required. All the applicant needs to know is which dampers auto-closed and which one does not to answer the question.
Response: Revised the question stem to place the examinee in the appropriate OA procedure step and require an evaluation of how the step will be performed.
1BwOA REFUEL-1, step 3.b checks the VQ dampers closed and the RNO column directs
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
manually closing any damper that is not closed.
This is the only step in the applicable OA procedure that addresses the KA topic of containment purge system (CPS). The step is not required from memory for an RO (not a CAS, OAS, or immediate action). The only way to test how the step would be performed is to apply systems knowledge based upon the stem conditions.
The knowledge required for correctly answering the questions includes knowledge of the Rad monitoring system and the containment purge system. Therefore, it is reasonable that the revised question meets the Tier 1 criteria of the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions.
20 F
3 X
N U
E S
Question ID 2501938 (Free Look)
NRC: 1) This is not a Tier 1 question. This is a great systems question. There is no procedure referred to. 2) The question is really not a K/A match as the K/A refers to the calculation of offsite doses due to a release from the power plant and the question does not address that. 3) Suggest deleting (1/2PM10J) from the stem.
Response: The K/A is Knowledge of the operational implications and/or cause and effect relationships of the following as they apply to an Accidental Gaseous Radwaste Release:
Calculation of offsite doses due to a release from the power plant
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
The K/A topic of an Accidental Gaseous Radwaste Release does not have an applicable OA procedure at Braidwood. The applicable procedure is RP-BR-930, POST-ACCIDENT VENT STACK SAMPLING OF THE WIDE-RANGE GAS MONITOR (WRGM). Using RP-BR-930 and its title in the question stem would be specific determiner in the question stem. Although the procedure is not referenced in the question, the abnormal operation of the SRO performing an Offsite Dose Calculation is included in the stem. Although RP-BR-930 is not an AOP, it is a procedure that would be used exclusively for abnormal plant conditions by both operations and rad protection. Section 4.2 is the specific knowledge being tested and is the procedure section that the RO would be responsible for performance of, during the posited plant condition.
It is reasonable that the K/A is met by knowledge of the operation implications of the conditions (i.e., how the RO would initiate a vent stack grab sample for the off-site dose calculation.)
If the question is still considered UNSAT, the alternate solution is rejecting and replacing the K/A.
(2) Facility would prefer to keep (1/2PM10J) in the question stem. Removing the panel number introduces confusion as to whether the question is asking about rad monitor functions performed at the RMS computer or the 1/2PM10J panel. The
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
intent is to question which rad monitor performs the sampling function at 1/2PM10J.
21 F
2 B
S 2016 Braidwood ILE 22 F
3 N
S 23 F
H 3
X M
U E
S Question ID 2501955 (Free Look)
NRC: This is not a Tier 1 question. This is a great systems question. The procedure is referred to, but the question is a system question requiring the applicant to understand the system response to an increase in letdown flow. 1) Remove the word letdown from all four answer choices.
Response: Revised question by changing part 1 to testing the overall mitigation strategy of the AOP and moving part 1 of original question to part 2 of revised question to continue meeting the K/A.
Removed the word letdown from all answers because it was repeated from the stem.
For this question the candidate must know the following to correctly answer the question:
- 1.
The effect of maximizing letdown on pressurizer level trend.
- 2.
The resulting effect on the master pressurizer level controller.
- 3.
The subsequent effect of the 1CV121 slave controller. (whether each controller is direct or reverse acting).
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
Because the question requires an analysis of the above information requiring multiple mental steps, beyond the recall of a single piece of acquired knowledge, the question is written at the HCL. Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
24 F
2 B
E S
2014 Braidwood ILE NRC: The second bullet states that an Alert EAL was declared and the question discusses notifications of the Site Area Emergency classification Please pick one or the other, but not both.
Response: Changed question stem to Alert in both sentences.
25 F
3 B
S Byron 2021 NRC Exam 26 F
3 N
E S
NRC: Last line of the stem consider are used for and/or provide an input to Response: Made suggested change 27 F
2 B
S 2006 Braidwood ILE 28 H
3 B
S 2019 Braidwood ILE 29 H
3 B
S 2019 Braidwood ILE
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
30 H
3 B
S 2009 Byron ILE 31 F
3 B
S 2016 Braidwood ILE 32 F
H 3
B S
2011 Braidwood ILE To answer this question a candidate will typically draw a one-line diagram of the ECCS system including the configuration of the cross-tie header from the RH pumps discharge lines to the suction of the CV and SI pumps, including the locations of the 1CV8804A and1SI8804B.
Because the question requires an analysis of the above information requiring multiple mental steps, beyond the recall of a single piece of acquired knowledge, the question is written at HCL.
Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
33 H
3 B
S 2022 Braidwood ILE 34 F
3 N
S 35 H
3 B
S 2016 Braidwood ILE
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
36 H
3 B
S 2009 Braidwood ILE 37 F
2 B
S 2014 Braidwood ILE 38 H
3 B
S 2018 Braidwood ILE 39 F
H 3
N S
Response: To answer this question a candidate will typically draw a one-line diagram of the chill water system containment isolation penetrations and the VP chiller pump trip circuit (see training bignote WO-2). Because the question requires an analysis of the above information requiring multiple mental steps, beyond the recall of a single piece of acquired knowledge, the question is written at HCL. Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
40 F
3 M
S 2019 Braidwood ILE 41 H
3 M
S 2011 Braidwood Cert Exam 42 F
3 B
S 2022 Braidwood Cert Exam 43 H
3 B
S NRC: Braidwood COPS Bank ???????
Response: The COPS exams in the ILT program are given at the end of the simulator training phase. They are comprehensive exams
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
constructed in the format of an NRC exam. The data base used to construct the exams consist of a bank of questions and 2 comprehensive exams constructed from questions within the bank. This question is from that bank, but is not currently used on either of the comprehensive exams.
44 F
3 N
S 45 H
3 B
S 2018 Braidwood Exam 46 H
2 B
S Embedded Reference 2016 Braidwood Exam 47 F
3 N
S Question ID 2502975 (Free Look) 48 F
2 B
E S
2022 Braidwood ILE NRC: Remove the word Bimba from the answer D Response: Removed the word Bimba from the answer as suggested.
49 F
2 B
S 2009 Braidwood ILE 50 F
2 X
N U
E S
Question ID 2502985 (Free Look)
NRC: This is a very basic question. Two distractors can immediately be eliminated, C & D.
- 1) Remove the (2) from the blank at the end of the stem of the question.
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
Response: Revised question by removing part 1 and added distractors to make a 1 part question.
Removed the (2) in question blank as suggested.
51 H
3 B
S Byron Exam Bank - Question 53 on Byron 2023 Exam 52 H
3 B
E S
Embedded Reference 2016 Braidwood ILE NRC: With instrument air header pressure at 75 PSIG the instrument air dryer bypass valve will have already opened as pressure dropped below 80 PSIG. the automatic plant response was Please modify stem and answers to reflect the above.
Response: Revised question as suggested.
53 F
2 X
X B
U S
Braidwood 2020 Cert Exam NRC: The K/A requires Conditions requiring containment evacuation (including recognition of the alarm. All answers assume the conditions for the alarm are already there and does not test the applicants knowledge of what the conditions are for the alarm. Also, just knowing where the manual actuation switch is answers the question.
Response: Agree that the question does not ask alarm recognition. Modified the question to make it 2 parts, with the part 2 asking the alarm recognition.
54 F
2 B
S 2019 Braidwood ILE - Not an embedded reference
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
55 H
2 B
S 2014 Braidwood ILE 56 F
2 N
S Because the fuel pool level and alarm setpoint are given in height above the fuel racks, and the question asks how far the fuel pool level drops to the low level alarm setpoint, the candidate must perform a calculation to determine the correct answer. Therefore, the question requires an analysis beyond the recall of a single piece of acquired knowledge, and the question is written at HCL. Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
57 H
3 N
S 58 F
3 N
S 59 F
H 2
B S
Byron Exam Bank - Question 61 on Byron 2023 Exam To answer this question, the candidate must understand the function of each rad monitor listed in the answer choices and make an evaluation of which will alarm first. Additionally, this question was accepted as a HCL question on a recent 2023 Byron NRC exam. Therefore, it is reasonable to consider the question is HCL.
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
60 F
2 B
S Braidwood Cert Exam Bank 61 H
3 N
E NRC: Distractor B is a potential second correct answer if the online GDT reaches 95 psig. I.E., no operator action Response: The standby GDT pressure will not rise during this event for the same reasons that the on-line GDT pressure will not rise. 1. Both tanks input from the waste gas compressor will not receive flow with the waste gas compressor control switch in stop. 2. Both tanks are separated from the cover gas header by pressure regulator 0GW9293 which will prevent back flow from the cover gas header to any GDT outlet.
62 F
3 B
E S
2014 Braidwood ILE NRC: Stem, 4th bullet, Then,.. fails HIGH..
Response: Revised question as suggested.
63 F
3 N
E S
NRC: Delete Distractor D. Distractor B - Two detectors, BOTH smoke in the affected zone.
Distractor C - Two detectors, one smoke AND one heat in the affected zone. Distractor D - Two detectors, BOTH Heat in the affected zone.
Response: Revised question as suggested.
64 F
2 B
S 2014 Braidwood ILE 65 F
2 B
S Byron Exam Bank - Question 69 on Byron 2023 Exam
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
66 H
3
?
B E
S Byron Exam Bank - Question 67 on Byron 2023 Exam NRC: The K/A states, Knowledge of limiting conditions for operation and safety limits. Safety limits are not addressed by this question.
Response: The facility agrees the question does not meet part 2 of the coupled K/A.
An alternative 2 part question has been prepared that tests both parts of the K/A.
67 H
F 3
N S
NRC: Missing Training Objective Response: Added training objective to question pedigree table. The question can be answered by applying a note from common surveillance procedure and tech spec basis. Therefore, it is reasonable to consider the question is LCL.
68 F
3 B
S 2011 Braidwood ILE 69 F
2 B
S 2018 Braidwood ILE 70 H
3 B
S Byron Exam Bank - Question 70 on Byron 2022 Exam 71 F
3 B
S Byron Exam Bank - Question 71 on Byron 2023 Exam To answer this question a candidate will typically draw a diagram of the boron letdown curve (BwCB Figure 11) and analyze the Y axis over core life.
Additionally, this question was accepted as a HCL question on a recent 2023 Byron NRC exam.
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
Therefore, it is reasonable to consider the question is HCL. Furthermore, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
72 H
3 B
S 2016 Braidwood ILE 73 H
3 B
S Byron Exam Bank - Question 73 on Byron 2023 Exam 74 F
H 3
B S
Constellation GFE exam bank For this question the candidate must determine the following to correctly answer the question:
- 1.
The effect of starting an additional CW pump on condensate depression.
- 2.
The resulting effect on the condensate pump NPSH.
- 3.
The subsequent effect of raising condensate depression with no change in condenser vacuum on overall plant efficiency.
Because the question requires an analysis of the above information, requiring multiple mental steps, beyond the recall of a single piece of acquired knowledge, the question is written at HCL.
Additionally, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
75 F
3 M
S Constellation GFE exam bank NRC:
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
76 H
3 X
B S
U S
Replacement Question from Bank from Byron 2023 NRC Exam.
Question ID 2504589 NRC 2009 Exam (Free Look) Add SRO justification NRC: SRO-only knowledge generally cannot be claimed for questions that can be answered solely based on knowledge of 1-hour action statements and the safety limits since ROs are typically required to know these items.
Response: Added SRO only justification to question pedigree information.
Agree that the question is not SRO only. Based upon the above and the amount of time already expended to develop this question, the proposal is to reject the K/A because (per NUREG 1021 section ES 4.2 (3) It is not possible (without expending an unreasonable amount of resources) to prepare a psychometrically sound question related to the subject K/A.
Replaced with a bank question with replacement K/A.
77 H
3 M
S 78 H
3 B
S 79 H
3 N
S
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
80 F
2 N
E S
Question ID 2504647 (Free Look)
NRC: This is a basic question about operability and Tech Specs. that requires an SRO applicant to recall a note from a procedure. Please explain how this is SRO Only knowledge.
Response: 0BwOA ELEC-1 is a AOP that would typically be performed by the SRO to determine operability of the qualified off-site circuits under abnormal grid conditions. Although an RO candidate is typically required to know Tech Spec entry conditions based upon above the line LCO information and unit applicability modes, the only requirement in Tech Spec 3.8.1 surveillance requirements for operable qualified off-site circuits is the AC distribution system breaker alignments (which the RO checks during performance of 1BwOSR 3.8.1.1, OFFSITE AC POWER AVAILABIITY WEEKLY SURVEILLANCE).
Degraded grid conditions requiring 0BwOA ELEC-1 entry is not addressed in TS 3.8.1. Although the RO may assist in data gathering for 0BwOA ELEC-1, the operability determination and Tech Spec entry conditions will be evaluated and performed by the SRO. The procedure note which is being tested by this question is specific to the operability determination and Tech Spec entry conditions and therefore is applicable to the SRO function.
81 F
3 N
S
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
82 H
3 B
S Byron Exam Bank - Question 82 on Byron 2023 Exam 83 H
3 N
S 84 F
3 X
N U
S NRC: This does not appear to be a K/A match.
The question is asking for the bases of the Containment Floor Water Level and what makes up that bases. It does not show Ability to determine and/or interpret the following as they apply to Containment Flooding: (CFR: 41.10 / 43.5
/ 45.13)
Tank levels and/or system flow indications for systems that supply components inside containment Response: Revised the question to meet the K/A.
85 F
3 B
E S
2014 Braidwood ILE NRC: Why has the answer and the three distractors changed from the 2014 exam?
Response: Since the 2014 exam, the containment charcoal filter system has been abandoned in place at Braidwood station. Therefore, 1BwFR-Z.3 has been revised, which accounts for the changes to the question.
86 H
3 N
S 87 H
3 N
S REFERENCE PROVIDED NRC: Giving the SRO applicant the whole Constellation Emergency Action Levels for
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
Braidwood Station is excessive. Would this be better addressed in an Administrative JPM?
Response: The intent is to only give the applicant the EAL matrix. Pages 2-1 through 2-25 of EP-AA-1001 Addendum 3. Revised the question designated provided reference to reflect only the EAL matrix. This is consistent with at least the last 7 Braidwood NRC written exams, which included questions with the EAL matrix as a provided reference (including the 2020 exam that was authored by the NRC).
88 H
3 N
S Agree that the question can reasonably designated as HCL.
89 F
2 M
S First part from 2020 Braidwood ILE - Question #89 NRC: Need original question and how is was modified in the exam file submission Response: Added original question that part 1 was based upon to question pedigree information.
Agree that the question can reasonably designated as LCL.
90 F
3 N
S 91 F
3 N
B
?
S REFERENCE PROVIDED NRC: 1) Providing applicants with the entire reportability manual? Would this be better addressed in an Administrative JPM?
- 2) Question on procedure allowing power to exceed 100% LPL Response: Since 2009 at least 3 Braidwood NRC written exams have included questions with the
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
Reportability Manual as a provided reference (including the 2020 exam that was authored by the NRC). The contents of the manual include a table of contents, a decision tree and the individual reportable events. The reportable events sections are a standard template that include the applicable 10CFR reportability requirements, the time limits for reporting events and a discussion of how to evaluate each event for reportability. The manual has not been identified as a potential source for cueing another exam question in past usage or this exam.
However, because the question premise is not actually a reportable event (rather is only advised to be screened for reportability), suggesting the question be replaced by a bank question without a provided reference.
92 H
3 N
S 93 H
3 N
S 94 H
3 B
S 2016 Braidwood ILE 95 F
3 B
S 2014 Braidwood ILE 96 F
3 B
S 2020 Braidwood ILE - Question # 97
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
NOTE 4: Level of Difficulty is not given for several questions.
NOTE 5: For questions from the Byron Exam Bank. When were those questions last used on Byron ILEs? Please document for each question.
97 H
3 B
E S
REFERENCE PROVIDED Question ID 2505199 NRC Braidwood 2016 ILE Exam (Free Look)
NRC: Suggest inverting the answers, so that the longest time is A and the shortest time is D.
Response: : The facility preference is to keep the answer choices in chronological order (shortest to longest). This is the typical practice of facility generated exams and what the examinees are used to. By inverting the answer choices, the facility believes it may induce a misreading or transposition error by the examinee even though they may have evaluated for the correct response.
98 H
3 B
S 2022 Braidwood ILE - Question # 83 REFERENCE PROVIDED 99 F
3 B
S Byron COPS bank 100 F
2 B
S 2019 Braidwood ILE
Form 2.3-5 Written Examination Review Worksheet Facility: Any Plant Exam Date: 12th of Never General comment concerning cog level concerns:
Facility response: For the original exam submittal, the author counted 41 high cog RO questions for the exam quality checklist (based upon his exam development status sheet). The author performed a review of all the RO questions to verify cog level in response to this concern. Typically, the author uses a conservative approach to assigning cog level (i.e. will default to low cog on questions that could be classified either way). Per NUREG 1021, Appendix A, two key elements are evaluated when assessing level of knowledge (1) the number and type of mental steps necessary to process the given data and arrive at the correct answer, and (2) the training and experience level of the target test group.
Also, per NUREG 1021, Appendix A: Generally, an HCL test item will require at least two mental stepsone requires the recall of acquired knowledge, and the other requires associating two or more pieces of data. The number and types of mental steps that must be considered are those necessary to rule out plausible incorrect distractors, as well as the steps needed to identify the correct answer. Furthermore, per NUREG 1021, if there is doubt concerning the number of associations, an item should be classified as HCL.
In addition to assessing the number of cognitive steps required to eliminate distractors and select the correct answer, the author uses the NUREG 1021 guidance that When assessing level of knowledge, the examination writers/reviewers must use the perspective of the test taker in the target group (i.e., novice versus expert) to avoid A-8 perceptual bias. The reviewer has seen the item, knows the answer, and may not appreciate the mental processes that an examinee may use to answer the question.
During the review the author found 1 question (Q 8) that was designated low cog in the pedigree information but was misidentified on the status sheet as high cog. That brought the high cog RO question count down to 40.
The facility response, to each question identified by the NRC that may have cognitive level concerns, are noted in the #8 Explanation column associated with the individual question.
Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.
Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:
- 1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.
Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.
Form 2.3-5 Written Examination Review Worksheet Facility: Braidwood Exam Date: March 2024 T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.
Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.
- 4. Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.
Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.
- 5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.
- 6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.
- 7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.
- 8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).
- 9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.