Text

Form 2.3-5 Written Examination Review Worksheet (Byron 2023 Ile)
	ML23325A053
Person / Time
Site:	Byron
Issue date:	11/13/2023
From:	April Nguyen; NRC/RGN-III/DORS/OB
To:	; Constellation Energy Generation
	Travis Iskierka-Boggs
Shared Package
22122A031	List: ML23325A049; ML23325A050; ML23325A051; ML23325A052; ML23325A053; ML23339A018; ML23339A023;
References
	50-454/23-301, 50-455/23-301 50-454/OL-23, 50-455/OL-23
	Download: ML23325A053 (1)
	v • d • e

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

1 H

2 X

B U

S See NOTE 1 - no tie to Tier 1 Bank: 2019 BWD NRC (What Question Number?)

NRC:

1)

When providing a list of conditions in the stem, it may be best to provide a statement similar to, Given the following conditions:

2)

-Remove the extra space prior to BEP in stem.

3)

No tie to Note 1 based on the question places the candidate in a reactor trip scenario but does not ask any information from BEP-0 and the root of the question is simply a GFE knowledge question and reference provided is INPO GFE material.

4)

Need to add 1 to BEP-0 Facility

1)

Discussed with chief examiner and agreed to not add stem statement

2)

N/A due to question replacement

3)

Replaced with new question

4)

N/A due to question replacement NRC - Replacement question is SAT 2

H 3

N E

S See NOTE 1 -tie to (1)

Reference Provided - Mollier Steam Table?

NRC:

1)

Is it necessary to have (may be stuck open)? Changing this to stating that PORV 456 is stuck open will help clean up the question by removing if it does not close.

2)

Add valve nomenclature to 1RY8000B similar to the PORVs

3)

Distractor Analysis for B appears to be written for C as it answers superheated plausibility and not the 2345 psig plausibility.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

4)

Distractor Analysis for D needs to include a statement that points to superheated plausibility as well

5)

Technical Reference states Steam Table, Mollier Diagram, but References Provided states NONE. Are applicants expected to answer this question without reference provided? (Only reference that does not count towards references provided is the GFE Equation Sheet)

Facility:

1)

Removed may be and if it does not close. Stated valve is stuck open.

2)

Added nomenclature to 1RY8000B

3)

Fixed distractor plausibility statement.

4)

Fixed distractor plausibility statement.

5)

NUREG 1021 section ES-4.3 A.1.f states Make an English dictionary, steam tables, and the Generic Fundamentals Equations and Conversions Sheet, Form 4.3-1, available in the examination room. This implies that these items do not count as references provided NRC - Changes made resulted in question being SAT 3

F 2

B E

S See NOTE 1 - tie to (1)

Bank: Byron LOR bank NRC:

1)

Question ties to information contained in the EOP however formatting needs improvement.

2)

Rewrite the question to provide initial conditions similar to Q1 and 2 with pressure less than 1425 psig, and then ask, In accordance with BEP 1, Loss of Reactor or Secondary Coolant, Step 1, why

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

the RCPs are tripped. This provides the question in a better format for initial license examinations.

3)

Basis document provided does not state the reason similar to the answer analysis provided. Can you please point out where the basis states this reasoning?

Facility:

1)

Made suggested format changes.

2)

Rewrote question as suggested.

3)

Added reference WOG GENERIC ISSUE RCP TRIP/RESTART which is referred to in EP-1 BD. Need to send to NRC.

NRC - Changes made result in question being SAT 4

F 3

B S

See NOTE 1 - tie to (1)

Bank: Byron 2014 NRC NRC:

5 H

3 N

S See NOTE 1 - tie to (1) 6 F

2 X

X M

S See NOTE 1 - tie to (1)

Modified from Byron ILT Bank 7

F 2

M E

S See NOTE 1 - tie to (1)

Modified from BWD ILT Bank NRC:

1)

Add that U2 RCS pressure is currently 2360 psig and stable to eliminate any ambiguity with the question. This should also help candidates determine that PORVs are not open.

2)

Distractor analysis A needs to have void the coefficient in last sentence swapped to the void coefficient.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

3)

Validate that the basis for this step is RO knowledge.

Facility:

1)

Made suggested change.

2)

Made suggested change.

3)

Question is RO knowledge. Training objective T.FR1-04, Given a step, note or caution from an S-series procedure step, EXPLAIN the basis of that step, note, or caution is applicable to ROs.

NRC - Changes result in question being SAT 8

H 3

N S

See NOTE 1 - tie to (1) 9 H

3 X

X B

U S

See NOTE 1 - no tie to Tier 1 Bank: BWD LOR Bank NRC:

1)

How would a faulted SG not result in lowered SG Pressure or a change in NR level? Are the conditions provided operationally valid for a faulted SG?

2)

Suggestion is to add additional indications to 1C SG for a Rupture requiring the applicant to determine Fault vs Rupture as well.

3)

No Tie to Tier 1. Recommend tying stem to 1BEP-0 to meet Tier 1 Facility:

1)

Depending on the size of the steam break (assuming a small break), if the SG is being fed by AF and decay heat exceeds the heat sink provided, pressure and level would not necessarily change.

2)

Raised 1C SG level in second set of stem SG parameters to indicate possible SG

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

tube leak/rupture. (1C SG already had elevated rad indications which also would indicate a tubeleak/rupture.

3)

Added tie to 1BEP-0 NRC - Changes resulted in Tier 1 tie and question is now SAT 10 H

3 B

S See NOTE 1 - tie to (3) & (4)

Bank: Byron 2022 Exam Q10 (LAST TWO) 11 H

4 N

S See NOTE 1 - tie to (1) & (4)

NRC:

1)

Bulleting format for the first part is different than other questions seen for previous questions. They are smaller than other questions.

2)

Items following the loss of offsite power should be bulleted similar to other times multiple conditions are provided.

Facility:

1)

2) 12 H

3 N

S See NOTE 1 - tie to (1) & (4)

NRC:

1)

First bullet does not match the others in terms of formatting.

13 F

2 N

E S

See NOTE 1 - tie to (1)

NRC:

1)

Is the assumption that CC outlet temperature referring to the outlet temperature of the U1 CC HX, correct? If so, specify in the stem Facility:

1)

Added clarification to 2nd bullet in stem.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

NRC - Changes made resulted in question being SAT.

14 F

2 N

S See NOTE 1 - tie to (1) 15 F

2 X

N U

S See NOTE 1 - tie to (1)

NRC:

1)

Distractor analysis is not accurate.

Distractor A states Plausible and then Correct.

2)

Distractor B appears to be incomplete.

Please correct the distractor analyses.

3)

What does the grid disturbance have to do with the crux of the question stem (which is dealing with the rod block and how to reset the rod block)? It seems arbitrary to meet the k/a. What would cause rods to step out from a LOOP? Question is unsat due to inability to see how stem conditions lead to the conditions provided for the crux of the question.

4)

Is it required knowledge for ROs to know alarm reset criteria?

Facility:

1)

Removed Plausible from choice A explanation.

2)

Added Plausible from to distractor B explanation.

3)

When the DGs start and re-energize the ESF busses, the 1A AF Pump will start on the ESF bus sequencer. This introduction of cold water into the SGs will cause RCS Tave to drop and subsequently control rods will step out automatically due to a Tref/Tave mismatch.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

4)

Changed objective to S.RD1-14 (this is a better objective tie) which states LIST all the Rod Stops, including any setpoints, coincidence, interlocks, and automatic actions associated with each.

NRC - changes made to the question and explanation of event result in the question being SAT 16 F

3 N

S See NOTE 1 - tie to (1) & (2)

NRC:

- Editorial change for question pedigree. See Provided early look document to update K/A reference Tier from 3 to Tier 1 Group 1.

(This needs to be updated still)

- Good Tier 1 question with tie for RO knowledge. No issues with question as written.

- Since the procedures are used on Unit 1, make an editorial change to the first bullet that states the RCS LOCA has been confirmed on Unit 1 Changes made are SAT 17 F

2 B

S See NOTE 1 - tie to (1)

Bank: BWD 2016 NRC Exam NRC:

1)

Answers are arranged in order of length, which is generally the ideal choice, but numerical values are not, which is also ideal. Is there any concern from the facility that having the 15% value being after 4, 12, and 31 being missed in anyway by applicants? Rearranging the items so that they are numerically arranged, makes the longest answer the correct one and thus stands out due to length. I am okay with the

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

way the answers are arranged but wanted to bring this up with the facility.

Facility:

1)

Kept order as is based on format.

NRC - Question kept as-is and is still considered SAT 18 F

3 X

M U

E S

See NOTE 1 - tie to (3) & (4)

Modified from Byron 2022 Exam NRC:

1)

Bullets do not match formatting from other questions (smaller)

2)

Distractors C and D are a subset of B in that they are all greater than 557F and rising, therefore are all correct, thus making the question UNSAT. (Do not see purpose of the only in parenthesis, adding this does not remove the subset issue.)

Facility:

1)

Bullets sizes are a result of changing document style for RTF to DOCX. Need to check and correct if necessary when printing new document.

2)

Removed the ONLY from question choices. Added a stem statement that answer must exactly match the note in 1BFR-H.1 to eliminate subset issue.

NRC - Okay with removal of the ONLY on the distractors. Delete the newly added note in parenthesis that states (Which of the following exactly matches the note?) - wording is unnecessary and not needed to get to the point of what the question is asking when Per the note is used.

NRC - Changes incorporated. Question is SAT.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

19 F

2 N

E S

See NOTE 1 - tie to (1)

NRC:

1)

Question only asks candidate to assess the one provided annunciator. In order to better align with the K/A and prioritizing the significance of each alarm, it would be best to provide additional alarms in the stem of the question.

2)

I would prefer that the question be established in which various alarms are provided in SFP in which the candidate is to assess which would be the best response. Simply determining fill methods is not the best way to meet this K/A and question borders on being Unsat.

3)

Does procedural guidance exist to fill the SFP using demin water? BOP FC-11 allows to use PW via a hose drop (common practice for level maintenance). Distractor plausible?

4)

Consider referencing BOP FC-11 and BOP FC-14, which contain the methods to fill the SFP and contain procedural guidance for the answer/distractors.

Facility:

1)

The question does prioritize the mitigation strategies to the event. There are no other MCR alarms that would reasonably come in during this scenario.

2)

See response to comment 1.

3) 1BOA REFUEL-2 step 11 RNO gives direction for filling SFP with demin water

4)

Referencing BOP FC-11 and BOP FC-14 will add unnecessary text to the question

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

stem. The same note that is referenced in 1BOA REFUEL-2 is in BOP FC-11.

NRC - based on information provided by the facility, Question is considered SAT.

20 H

3 B

E S

See NOTE 1 - tie to (1)

Reference Provided: Figure 1BOA SEC-8-1 Graph Bank: Braidwood 2022 Exam NRC:

1)

In order to make the distractors a bit more plausible, I would like for Choice C to be 517F and 1162 psig; and Choice D to be 507F and 812 psig. Changing the choice distractors to these values places all points on the figure closer together, and has the values above maximum and minimum in closer proximity without an outlier vastly different than the others. See Image Below:

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Facility:

1)

Suggested change to distractor C is not a plausible distractor because it is outside the limits of the curves and above ruptured SG pressure. Instead, changed distractor C to 500F and 1050psig which meets the suggested goal of keeping the distractors values on the figure closer together, and has the values above maximum and minimum in closer proximity without an outlier vastly different than the others.

Made changes to distractor D as suggested.

NRC - Changes incorporated make question SAT 21 H

4 X

B S

See NOTE 1 - tie to (1)

Reference Provided: Figure 1BOA SEC-3-1 Graph

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Bank: Braidwood 2016 22 F

2 N

S See NOTE 1 - tie to (1) 23 H

3 X

X X

N U

S See NOTE 1 - tie to (1)

NRC:

1)

Question is UNSAT for multiple reasons.

See below:

2)

Question K/A states Knowledge of reasons for the following responses and/or actions as they apply to High Containment Pressure: Ensuring that MSLI has occurred. The question does not provide any indication of High Containment Pressure in the stem or answer choices.

3)

Stem Focus: Question pedigree distractor analysis and correct answer explanation all discuss location of the break and protection that MSLI affords dependent upon the break. There is no indication of where the break is located in the stem of the question that helps support any of the information provided in the pedigree.

4)

Credible Distractors: Distractor D is not credible given the fact that the location of the break is not provided. Furthermore, it is vastly different than the correct answer and other three distractors that it is potentially cuing as to being an incorrect answer for this question, making it easier to eliminate as a possible answer.

Facility:

1)

See below responses. As written, it was assumed the students would know that

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

containment pressure rose because the stem had the crew enter 1BFR-Z.1, RESPONSE TO HIGH CONTAINMENT PRESSURE.

2)

Added stem statement that Containment pressure rose above the criteria to enter 1BFR-Z.1, RESPONSE TO HIGH CONTAINMENT PRESSURE.

3)

Adding that containment pressure rose in the stem gives indication that the fault was inside containment.

4)

Moved STOP the steam flow to the question stem as it was in 3 answers and slightly changed distractor D such that the grammar correctly finishes the stem statement. Distractor D is still different from the other choices. However, it is a plausible distractor because it is a valid reason for checking MSIVs closed during another DB accident. Also NUREG 1021, section 4.2 D.15 addresses specific determiners in this manner as being applicable to the correct answer (vs. distractors).

NRC - Changes made bring the question up to SAT status.

24 F

2 M

E S

See NOTE 1 - tie to (1)

Modified from Byron 2021 NRC Exam NRC:

- What procedural direction is used to isolate letdown for RCS piping corrosion? Provide this information in the distractor analysis as to why distractors C and D are incorrect.

NRC:

- Procedure direction included in distractor analysis.

Question is now SAT.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

25 H

3 X

B E

S See NOTE 1 - tie to (1)

Bank: Braidwood 2009 NRC Exam NRC:

1)

Distractor B is not plausible. Distractor analysis B further emphases why it is not plausible to consider shutting down the only running SX pump as the first leak detection activity that is performed.

2)

Stem Focus does not provide enough indications that place the applicants in 1BOA PRI-7 Step 6.d RNO based on simply providing that the leak is in containment. Additionally, the distractors have some plausibility concerns that need to be addressed. Please see the following suggestion for changes to improve the question:

Leave all of the information in the stem as given but change the wording of the question to as follows:

With the above conditions, per 1BOA PRI-7, Step 6.d RNO, the FIRST MCR action the crew will take to identify the leak location is A. shutdown 1A & 1C RCFCs B. Close 1SX016A, RCFC SX Inlet Valve C. Shutdown 1B & 1D RCFCs D. Close 1SX016B, RCFC SX Inlet Valve Update the distractor analysis accordingly. Changing the question to this manner will eliminate any distractors that are not credible, and provides three

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

other distractors that are credible according to the same RNO step, and the FIRST action that must be taken iaw PU&A is to Shutdown the NON-running SX Pumps train RCFC, which in this case is the 1B & 1D RCFCs.

Facility:

1)

Made suggested changes to question.

2)

Made suggested changes to question.

NRC - Changes incorporated to make question SAT 26 H

3 B

E S

See NOTE 1 - tie to (1)

Steam Tables Necessary to answer question but no references listed as provided during exam NUREG 1021 section ES-4.3 A.1.f states Make an English dictionary, steam tables, and the Generic Fundamentals Equations and Conversions Sheet, Form 4.3-1, available in the examination room. This implies that these items do not count as references provided Bank: Braidwood 2013 NRC Exam NRC:

1)

Slight cueing is present in the question statement, SATURATED conditions are indicated in the This statement gives the applicant the information that there is saturated conditions and which directly leads to answering the second part of the question for response to saturated conditions.

2)

Change the question statement to, SATURATED conditions, if present, would be indicated in the____

Facility:

1)

Made suggested changes to question.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

2)

Made suggested changes to question.

NRC - Changes make the question SAT 27 T2 H

2 X

X B

E S

Bank: Braidwood 2022 Cert Exam NRC:

1)

Is there any chance that Seal Outlet Temps would NOT rise? Based on explanation, it is apparent that they will rise. Remove may prior to rise in Distractors A and B

2)

Distractor C is not plausible and the temperature is not used in any of the other distractors as to provide cueing.

3)

Based upon the review of question 78, there is overlap testing RCP bearing temperature limits of the procedures.

Question 78s K/A is more directly tied to RCP bearing temperatures than this question. Therefore, this question needs to be rewritten to be more closely tied to RCP Cooling Water Supplies without asking about bearing temperature requirements from the procedure.

4)

In order to accomplish this, I suggest changing the second part of the question to ask a question pertaining to operating RCP cooling water supplies rather than bearing temperatures.

5)

Do not reference horsenotes in the exam technical reference, unless these are now being updated and approved training materials, as lesson plans are. Make sure these are removed for any question that refers to them.

Facility:

1)

Removed may and replaced with will

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

2)

Removed temperature criteria from question. Rewrote answer and distractors to only ask for the pump components that will be affected.

3)

Removed temperature criteria from question. Rewrote answer and distractors to only ask for the pump components that will be affected.

4)

Removed training notes from references on question 27, 30, 31, 32, 37, 47 NRC - Question changes match K/A and result in question being SAT.

28 H

3 B

S Bank: Braidwood 2019 NRC Exam NRC:

29 H

2 B

S 30 H

3 B

S Bank: Braidwood 2020 NRC Exam NRC:

31 H

2 B

S Bank: Braidwood 2018 NRC Exam NRC:

32 H

2 X

N E

S NRC:

1)

Question could be answered by looking at distractors and choosing the one answer that was different than the other three without knowing system lineup specific to Byron.

2)

To avoid the above problem consider changing one distractor to ONLY RCDT pressure will rise and use 100 psig since

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

the RCDT has the same relief mechanism according to distractor B.

3)

Do not use # on the exam. Change all locations to psig.

Facility:

1)

Change distractor D to suggested distractor and moved it up to distractor B position to put choices in order by length.

2)

See response 1.

3)

Removed # in question and replaced with psig.

NRC - changes made result in question being SAT 33 F

3 B

E S

Bank: Byron 2021 NRC Exam Q35 (LAST TWO)

NRC:

1)

Recommend making the question a 2 part question, as the what action is required portion of the question (the (b) part of the K/A) blends into the (a) part in B & D.

Making this a 2 part question would clarify and enhance readability. Additionally, the (b) part of the K/A is a weak match in this question.

Facility:

1)

Changed question to a 2 part question as suggested and added explanation to part 2 and another OA reference to strengthen K/A tie.

NRC - changes incorporated make question SAT 34 H

3 B

S Bank: Byron 2022 NRC Exam Q36 (LAST TWO) 35 H

2 B

S Bank: Braidwood 2018 NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

36 F

3 M

S Modified from 2009 Braidwood Exam 37 H

3 B

S 2020 Braidwood Exam NRC:

1)

Bullets not all formatted properly

2)

Indent conditions following Unit 1 Mode 3 statement to indicate that they are present with that statement.

38 F

2 N

S 39 H

2 N

S 40 F

2 N

E S

NRC:

1) 1GBP 100-1 states that 1 Reactivity Cavity fan needs to be started when RCS Temperature exceeds 135F. The question is asking what the minimum temperature given exceeds the extended exposure limits. The keyed answer is 135F which meets but does not exceed. It could be argued that 150F is the correct answer as the lowest temperature given that EXCEEDS 135F.

2)

In order to keep 135F as the keyed correct answer, change the stem of the question to state something similar to, the MINIMUM temperature, above which, that exceeds Facility:

1)

Made suggested change to stem.

2)

See comment 1.

NRC - changes incorporated make question SAT

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

41 F

2 N

S 42 F

2

?

N S

E S

- Good Tier 2 Generic question. Information is asked in manner that is generic enough to the system to meet the intent of the Generic K/A tie to the system K/A

- Update Tier category on K/A References section of question pedigree. This question is Tier 2 Group 1, not Tier 3 NRC:

1)

How is the question tying the main & reheat steam system to the K/A? Question appears to be the secondary chemistry in FW. K/A mismatch?

2)

Upon review of the Question and K/A in comparison to overall exam overlap, recommend swapping the System used for this K/A to 059 (SF4S MFW) MAIN FEEDWATER SYSTEM and keep Generic G2.1.34 associated so that this question can be used on the exam.

3)

Update Form 4.1 Written Outline and provide updated version to Chief.

Licensee:

1)

Replaced K/A system as suggested. Added item to form ES-401-4. Need to update outline with ALL K/A rejections and return to NRC.

2)

Tier 3 label in question K/A references section is auto populated in VISION database based upon a generic K/A selected. This will be manually corrected

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

when the exam is printed and transformed to a WORD.DOC NRC - Changes incorporated make question SAT 43 F

2 N

S NRC:

1)

Align answers/distractors from shortest to longest or longest to shortest, and update keyed answer.

Licensee:

1)

Aligned answers shortest to longest as suggested.

44 F

2 B

S 2019 Braidwood Exam NRC:

1)

Since the question stem references unit 1 FRV bypass valves, specify Unit 1 is ramping rather than the unit Licensee:

1)

Made suggested change to stem 45 F

3 M

E S

Modified from 2009 Braidwood Exam NRC:

1)

The original bank question was an ideal K/A match. Part 2 of the modified question seems arbitrary in terms of meeting the K/A. Recommend using the bank question, unmodified.

Licensee:

1)

Changed to original bank question as suggested.

NRC - Original Bank Question is SAT.

46 H

3 N

S

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

47 F

H 3

B M

S 2018 Braidwood Exam

- This question is identified as a Braidwood Bank exam item. This comment is to just clarify that this question was used verbatim and not modified from the original similar to Q49.

Modified from 2022 Byron Exam NRC:

1)

Recommend not referencing horsenotes in the exam technical reference, unless these are now being updated and approved training materials, as lesson plans are.

2)

This is not the question that was submitted for an early look item. Provide an explanation as to why the question as changed without notifying CE.

Licensee:

1)

Horse note reference was removed.

2)

During the exam validation process, the original question was identified as having reverse logic. The original question gave a faulted bus diagnosis in the stem and then asked which MCR indication would indicate the diagnosis. (vs, giving the indication and asking the examinee to make the diagnosis). Therefore, the Byron exam team and facility rep. asked that the question be replaced. Not notifying the CE was an oversight (not recognizing the question was on the free look submittal) of the Byron exam team.

48 F

3 B

E S

2018 Braidwood Exam NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

1) 3 of 4 bullets are smaller size, possible word conversion issue. Double check upon print.

2)

Distractor C is confusing as written. Is it saying for unit 1 immediately and unit 2 after 2 additional hours OR both units after an additional 2 hours2.314815e-5 days 5.555556e-4 hours 3.306878e-6 weeks 7.61e-7 months ? Based on the answer explanation, the latter seems correct. Recommend rewording the distractor to clarify.

3)

See comment 2 for distractor D. If Unit 1 is immediately (as in distractor A), state as such.

Licensee:

1)

Re-bulleted the stem in VISION data base.

Will check formatting during word conversion.

2)

Reworded distractor as suggested.

3)

Reworded distractor as suggested NRC - changes incorporated make question SAT 49 H

3 M

N E

S 2016 Braidwood Exam

- Second portion of the question has component label written differently than the procedure does. See Prelim Exam Document.

NRC:

1)

This is modified such that this can be considered a new question.

Licensee:

1)

Changed question pedigree to new question.

50 F

2 B

E S

2016 Braidwood Exam

- Q-K/A tie is weak but acceptable. K/A is asking for diverse indications that will be used to support other

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

indications. This question is essentially a power supply question that is asking where the diverse indication of breaker status can be found. Granted, I like the question as it is asking a common misconception and therefore, we will leave it on the exam.

- The question asked to the applicants is a bit wordy and confusing. Consider re-wording as follows:

Which of the following DC buses will the EO check the associated breaker closed for each indicating light on the 1B DG local control panel?

NRC:

1)

Consider referencing the breakers listed in Att D of 1BOA ELEC-3. #1 light - DC Bus 112, BF1-1. #2 light - DC Bus 112, BR1-1.

Licensee:

1)

Reworded question as suggested. Added 1BOA ELEC-3 to question re3ferences.

NRC - Changes incorporated make question SAT.

51 F

2 M

S Modified from 2016 Braidwood Exam 52 F

2 N

S 53 H

2 N

E S

NRC:

1)

Reword Distractor C to be the opposite in wording of keyed answer D such that 1A066 closes first then 1A065 closes. Keep distractors A and B the same.

Facility:

1)

Changed distractor D as suggested and updated plausibility statement.

NRC - changes incorporated make question SAT

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

54 H

3 B

S 2011 Braidwood Exam 55 H

3 B

E S

2018 Braidwood Exam NRC:

1)

Bullet formatting not consistent

2)

Is it possible for control rod D-4 being 16 steps out from CBD prior to any automatic actions stopping the rod movement prior to taking rod control to manual?

3)

Answer analysis states D-4 is over 20 steps out of alignment. 192 - 176 = 16 steps. Correct distractor analysis to be consistent with conditions present.

Facility:

1)

Re-bulleted the stem in VISION data base.

Will check formatting during word.

2)

There are no automatic rod stops that would prevent rod insertion with a stuck or misaligned rod.

3)

Answer explanation (as much as 20 steps misalignment) represents the maximum possible misalignment considering DRPI inaccuracy (+ 4 steps). Added to explanation of correct answer.

NRC - explanation provided resolves identified concerns. Question is SAT 56 H

2 N

S 57 H

2 N

E S

NRC:

1)

Distractor C should say and will remain running rather than can

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

2)

Stem condition 1VG05C, Mini purge status should be bulleted to mimic other conditions Licensee:

1)

Changed distractor as suggested.

2)

Added bullet to stem statement.

NRC - Changes incorporated make question SAT 58 F

3 N

E S

NRC:

1)

Distractors A and D have two different usage of its and its. Remove the word altogether and use the room temperature Facility:

1)

Made suggested changes.

NRC - changes incorporated make question SAT 59 H

2 X

N U

S Provided reference (embedded)

NRC:

1)

Question is tier 3 and requires no system level knowledge to answer.

2)

Upon review of question and K/A associated with the system, it is recommended to replace the generic K/A associated with this system, unless question can be written with stronger tie to system knowledge.

3)

Replace this K/A with the following:

(045)(SF4S MTG) MAIN TURBINE GENERATOR SYSTEM (G.2.2.44 - Ability to interpret control room indications to verify the status and operation of a system and understand how operator actions and directives affect plant and system conditions. (CFR: 41.5 / 43.5 / 45.15).

4) Update outline and provide newest revision to the chief

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Licensee:

1)

As discussed with CE, removed information from question stem about system pressure in special instructions bullet. This now requires system knowledge to answer the question correctly.

2)

See response 1.

3)

See response 1.

4)

See response 1.

NRC - changes incorporated make question SAT 60 H

2 M

S NRC:

1)

Q52 and 81 specify that the bus is de-energized upon the fault. While not necessary, adding it would make it consistent with the rest of similar exam questions.

Licensee:

1)

Added de-energizes for consistency.

61 H

2 B

S 2018 Braidwood Exam NRC:

1)

Bullets are not formatted consistently.

2)

Add a 0 to the leakrate value such that it reads 0.015 GPM.

Facility:

1)

Re-bulleted the stem in VISION data base.

Will check formatting during word.

2)

Added 0 as suggested.

62 F

3 N

S NRC:

1) 3rd bullet appears to have a sizing issue (bullet not standard size). Might be a word conversion issue? Double check on print.

2) on line is split into two words, use same formatting as throughout rest of the exam

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Licensee:

1)

Re-bulleted the stem in VISION data base.

Will check formatting during word.

2)

Changed to online.

63 F

2 X

B U

S 2011 Braidwood Exam NRC:

1)

Part (a) of the K/A is matched, however, part 2 does not hit part (b) of the K/A, which is the most important part of the K/A to match (ES-4.2 C.2).

Licensee:

1)

Revised question into a 2 part question with part 1 asking to predict impact on the FPS and part 2 using procedure BAP 1100-10 to mitigate consequences.

NRC - changes incorporated make question SAT.

64 H

2 N

E S

Provided reference NRC:

1)

Why is it necessary to provide the reference for this question? Question is more of an admin JPM and the written exam should not be an extension of the operating test. Question can easily be rewritten to meet the K/A without the use of a reference.

Licensee:

1)

Byron exam team agreed the question can be answered without a reference.

Removed the reference provided from the question.

NRC - question is SAT 65 F

2 B

S 2020 Braidwood Exam

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

66 F

2 N

E S

NRC:

1)

LOD for this question is borderline LOD 1.

This question could easily be rewritten to make it more operationally valid.

Licensee:

1)

Discuss with CE. Need better understanding of comment.

NRC - further discussion with facility concluded question meets requirement to test operators regarding unit differences and is not taught during PAT and therefore meets requirements. Question is SAT.

67 F

2 N

E S

NRC:

1)

Overlap concern. Question stem of Q99 eliminates distractor A as a plausible answer.

Licensee:

1)

Changed 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months distractor to 12 hours1.388889e-4 days 0.00333 hours 1.984127e-5 weeks 4.566e-6 months . This is a more plausible distractor as 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months for a controlled ramp from full power is an extremely short amount of time. This eliminates the overlap with question 99 because the 5 hour5.787037e-5 days 0.00139 hours 8.267196e-6 weeks 1.9025e-6 months ramp in Q99 is less than all choices in this question.

NRC - agree with changed distractor. Changes incorporated make question SAT.

68 F

2 B

S 2019 Braidwood Exam 69 F

2 N

S 70 H

2 N

S

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

71 H

2 N

S 72 F

3 N

S Tier 4

- Good tie of Tier 4 to plant specific question for K/A asked.

- Please provide 1BGP 100-2 as reference so I can validate distractors.

- Typo on distractor C. See Prelim Exam Document Facility:

Added 1BGP 100-2 as reference.

Corrected typo and removed unnecessary capitalized word 73 H

3 M

S Modified from 2022 Byron Exam NRC:

1) 1226 psig would be a more plausible distractor than 1241 psig. Is there a reason 1226 psig was not used?

2)

Add Unit 1 to the stem initial conditions to make the question plant specific Licensee:

1)

Used the same logic (errors) for determining distractors as the original question.

2)

Added Unit 1 to stem.

74 H

3 N

S 75 H

2 N

S

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

76 H

3 B

S 2018 Braidwood Exam 77 H

3 N

S NRC:

- Bullet formatting is not consistent.

Licensee:

Re-bulleted the stem in VISION data base. Will check formatting during word 78 H

3 X

M U

S Modified from 2021 Byron Exam NRC:

1)

BOA RCP-2 appears to not address motor bearing temps - only providing a max allowable lower radial bearing temp (225 deg). If this is true, there is no correct answer, as part 1 asks about the motor bearing temp IAW BOA RCP-2 step 2.

BOA PRI-6 addresses the max motor bearing temp in the OAS page, but nowhere else. Recommend changing part 1 of the stem to ask IAW BOA PRI-6.

2)

Overlap concern with question 27 which also tests applicants on RCP bearing temperatures. Question 27 needs to be rewritten. Question 78s K/A is tied to bearing temperatures, and should keep that item tested according to issue 1. OR use original question from 2021 Byron exam as it meets the K/A requirements.

Licensee:

1)

Changed procedure reference in question stem and explanations to 1BOA PRI-6.

1BOA RCP-2 was a carry-over from the original question but was not the correct procedure for the modified question.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

2)

Revised question 27 to eliminate the overlap.

NRC - changes incorporated make question SAT 79 H

3 N

S NRC:

- Bullets are not formatted consistently Licensee:

Re-bulleted the stem in VISION data base. Will check formatting during word 80 F

2 N

S 81 F

3 X

B U

S 2022 Byron Exam Q13 (LAST TWO)

NRC:

1)

Not SRO-only. Question 13 (RO question) on 2022 exam. Question tests overall mitigative strategy of procedure.

Licensee:

1)

While the question was used on the Byron 2022 RO exam section, it may be more appropriate for the SRO exam section. The question originated on the 2016 Braidwood exam as an SRO question (Q #90) and was used as a bank question on the 2020 Braidwood exam (Q #80) which was authored by the NRC. The question SRO only justification is logical, and it meets the K/A. Request that the CE evaluate the question again for SRO only use. This questions SRO only format is very similar to question #87 on this exam. However, if replacement of the question is warranted, may request that the K/A be rejected as the author has spent a reasonable amount of resources trying to replace the question.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

NRC - after an assessment of the question per NUREG SRO question flowchart, a review of previously mentioned exams where the question was used as an SRO question, it has been determined that the previous CE oversaw the use of an SRO question on the RO exam for the 2022 Byron Exam.

Question is SAT to use as SRO question on this exam.

82 H

3 N

S 83 H

3 B

S Provided reference: 0BOL 11.a Rev. 13 2022 Braidwood Exam NRC:

1)

Correct spacing in distractor B, liquidrelease

2)

Correct spacing in distractor D, andrelease by adding space after and Licensee:

1)

Corrected spacing

2)

Corrected spacing 84 F

H 2

3 X

B U

S 2020 Braidwood NRC Exam See NOTE 1 - No Tie to Tier 1

- Question is not Tier 1. This is a systems level response and does not address any of the items in the above referenced NOTE. (Not meeting Tier 1 requirements does not make the question Unsat.)

- Question is not SRO-level of knowledge. This question is asking about a system interlock from an area radmonitor. Reference provided points to System Lesson plan. Is this lesson only provided to SRO candidates?

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

- Refer to NUREG-1021, ES-4.2, and Figure 4.2-3 when assessing SRO questions not testing Tech Specs to aid in determining if it meets SRO level of knowledge. ROs should be aware of system interlocks with area radiation monitors therefore this question does not meet SRO level of knowledge.

- Question IDd as High Cog. I would say it is Fundamental knowledge to know system interlocks.

Suggestion to improve the question:

- To meet Tier 1 tie, provide a situation in which fuel moves are being conducted, receipt of area radiation alarms in control room, and have candidate assess multiple alarms and then ask them how to respond to the event by directing an AOP such as Fuel Accident (or similar to facility procedures). You can also ask the interlock question as a second part. This will meet the Tier requirements and make the question SRO level for directing response to an event after assessing plant conditions.

NRC:

- New Question used in place of originally submitted question.

- New Question is SAT.

Provided reference 2019-2 Byron Exam 85 H

3 N

S 86 F

H T2 2

3 X

X N

B U

E S

- Question pedigree states Tier 3. This is a Tier 2 Group 1 question. See Prelim Exam document for specifics

- Question is not SRO level of knowledge. An RO would be able to answer this question by elimination of items known to not contain the designated location for recording the RC filter DP as it is not on any of their turnover items.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

- Distractors do not meet credible SRO level distractors and therefore are not entirely plausible.

Suggestion to make SRO level of knowledge:

- Write a question based on daily leak rate calculations and tie to either leak rate TS actions, or reporting actions based on leak rate limits being exceeded. This will make strong tie to SRO level of knowledge.

- Licensee replaced early look question with question from below referenced exam:

2018 Braidwood Exam NRC;

1)

Distractor D is not plausible. If boron concentration were to rise, entering BOA PRI-12 (uncontrolled dilution) is not plausible.

Licensee:

1)

Replaced distractor D part 2 with 1BOA ROD-1 as a rise in boron concentration would cause unanticipated rod motion.

2)

Tier 3 label in question K/A references section is auto populated in VISION database based upon a generic K/A selected. This will be manually corrected when the exam is printed and transformed to a WORD.DOC NRC - Changes incorporated make question SAT.

87 H

3 N

S NRC:

1)

In the answer analysis, specify Att E, step 7 of 1BOA PRI-5, to reset the phase A signal

2)

Chill Water pump should read Chilled water pump (see BOP VP-1)

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Licensee:

1)

Made suggested changes to answer explanation and stem.

88 H

2 M

E S

Modified from 2022 Braidwood Exam NRC:

1)

Question has overlap with Simulator JPM

d. In order to remove the overlap, suggestion is to write a question pertaining to loss of FW heating and procedural direction from that to eliminate overlap concern.

2)

Additionally, the outcome of system response between the JPM and the written question have two very different results.

The written question states that a runback will be directed, and the JPM has the applicant restart the pump. In order to ensure that the runback has to occur, provide additional information that the tripped pump cannot be restarted, or indications that a restart is not feasible.

Facility:

1)

Because the question and JPM have different outcomes due to the availability/or lack of a standby pump, suggest that the overlap is acceptable.

2)

Added to stem statement that standby pump will not manually start.

NRC - based on assessment of JPM and written question, overlap is not a concern. Changes incorporated make question SAT.

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

89 H

2 X

N U

S NRC:

1)

How is the applicability of the U-0 CC HX LCO to both units using procedures to correct, control, or mitigate the consequences of those abnormal conditions? Part (a) of the K/A is matched, however, part 2 does not hit part (b) of the K/A, which is the most important part of the K/A to match (ES-4.2 C.2)

Licensee:

1)

The use of LCOs as procedures to mitigate consequences of events is based upon the 10CFR50.36 which states, When a limiting condition for operation of a nuclear reactor is not met, the licensee shall shut down the reactor or follow any remedial action permitted by the technical specifications until the condition can be met. The K/A selected for this question is difficult to use a station procedure (OP/OA/EP) to write a discriminating valid question. The K/A system (Service Water System/Heat Exchanger Failure) has limited components that are candidates for a discriminating question (RCFCs and CC heat exchangers). The other heat exchangers in the SX system are small HXs specific to cooling a single component, that do not offer good topic for discriminating questions upon failure. An RCFC coil leak has already been used on this exam (Q #25),

so the author tried to write a question based upon a CC HX leak. Possible answers and distractors for this situation are BOA PRI-6 and PRI-7. Just selecting one of these would be knowledge of entry

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

conditions (RO knowledge). The mitigation action in BOA PRI-6 is simply locate and isolate the leak without specific information how to perform it. Therefore, we found it difficult (unable) to create a valid SRO question using the OA procedure. Suggest keeping the question as written and substitute generic K/A P2.2.5, Knowledge of the bases in TS for LCOs. (Keeping the same system this would maintain the K/A category totals on the SRO tier 2/group 1 section within 1 point of each other).

NRC - Suggested change to K/A is acceptable.

Update the Form-4.1-1 Rejection of K/As and the question becomes SAT for the K/A presented based on the workload put in to generating this question.

90 F

2 N

S Provided reference (embedded)

NRC:

1)

Provide explanation why the embedded reference is required to answer the question. There does not appear to be any relevant information gained from including this graphic into this question. Question can be answered without the reference.

2)

Remove the embedded image as reference.

Facility:

1)

There are 8 containment pressure indications in the MCR including 1CS-934.935,936,937 (TS 3.3.2), 1PDI-VP236 (in 2 locations, TS 3.6.4) and 1PI-PC004 &

1PI-PC005 (TS 3.3.13). The embedded reference was added to give the examinee a fair chance to identify the instrument in the question topic without having to

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

memorize the equipment ID number (which we thought was an unreasonable expectation). The black dots on the instrument labels are indication of instrumentation that is required by Reg.

Guide 1.97. BAP 330-4 section C.3.h describes the use of black dots on the MCR instrument IDs. While this includes TS 3.3.3 PAMs instrumentation, it is not exclusive to PAMs instrumentation. For example, the containment pressure instruments 1CS-934.935,936,937, also have black dots on their labels (required by Reg. Guide 1.97) but they are not TS 3.3.3 instruments.

2)

Based upon the above added explanation, request CE to evaluate whether or not the embedded references should be removed.

NRC - based upon the discussion by the facility, it was determined that the imbedded reference can remain on the exam, and the question remains as SAT.

91 F

3 N

S E

S Question is SAT with following questions for distractors:

- Distractor analysis for choice C states emergency boration valve is limited to 1BOA PRI-2. What are the entry criteria for 1BOA PRI-2. Is there any conditions provided above that a candidate may deem entry into this procedure is necessary, thus making the distractor a plausible correct answer?

- Distractor analysis for correct choice D states ramping turbine is significant reactivity manipulation that should be evaluated. Is there procedural guidance that states this?

- Question IDd as Tier 3. It is Tier 2 Group 2 see exam document

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

- Good question for a generic tied to a systems type question.

NRC:

1)

Distractor C states that the rate of the initial boration will be slowed. The stem states that the initial boration is complete. The crew has completed the initial boration and the charging FCV failed to minimum. Why would a subsequent boration flowrate be slowed? Will the boration valve not open to allow whatever the pot is set at and the PW open up enough to fill the balance of the 52 gpm (i.e. 30 gal BA and 22 gal PW)?

2)

In order to remove the confusion identified through NRC comment 1 above, clarification can be added to the stem of the question such that it reads as follows:

Per OP-AA-300, REACTIVITY MANAGEMENT, the SRO will direct the turbine ramp to Licensee:

1)

The concept of a slower boration is not accurate. Revised the explanation to distractor C. It was based upon a dilution (vs. boration) that is injected into the top of the VCT and then slowly mixed into RCS with charging flow. Boration additions are injected into the charging pump suction line and would not be slowed in this scenario.

2)

Made suggested change to stem question.

NRC - changes incorporated make question SAT.

92 H

3 B

S 2020 Braidwood Exam

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

93 F

3 B

E S

2016 Braidwood Exam NRC:

1)

Distractor analysis for choice A does not have information provided for Part 2 of the question.

Facility:

1)

Added plausibility statement for distractor A, part 2.

NRC - Changes incorporated make question SAT 94 H

3 N

S 95 H

3 X

M U

S Modified from 2020 Braidwood Exam Reference Provided TS 3.1.7 and Basis NRC:

1)

Part 2 of the question is leading - provides clue to correctly answer part 1.

Licensee:

1)

Modifying the question led to the cueing issue. Replaced with the original question from the 2020 Bwd NRC exam.

96 F

2 B

S 2011 Braidwood Exam 97 F

2 B

S 2020 Braidwood Exam 98 F

3 M

S Modified from 2022 Byron Exam 99 H

3 N

S Provided reference: Reportability Procedures (2) 100 F

2 B

E S

2020 Braidwood Exam NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: Byron Station Exam Date: October 16-21, 2023

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

1)

Applicant may ask who is in C&C; MCR or TSC. While this should not matter to correctly answer the question, the applicant may ask.

2)

Add clarification to answer & distractor analysis - NRC notifications are not delegated to EOF, only state & local (S/L) are. Answer is Notification to Offsite Agencies, which the applicant may imply means NRC notification as well.

Licensee:

1)

Question stem states the EOF is staffed and ready to assume C & C, which implies C & C is at the station (MCR or TSC).

2)

Added (state and local) to Notification of Offsite Agencies to eliminate any confusion about NRC notifications being included.

NRC - Changes incorporated make question SAT.

Form 2.3-5 Written Examination Review Worksheet Facility: Any Plant Exam Date: 12th of Never Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.

ML23325A053

Text

Navigation menu

Search