ML23172A180

Form 2.3-5 Written Examination Review Worksheet (LCS 2023)

ML23172A180
Person / Time
Site:	LaSalle
Issue date:	06/12/2023
From:	NRC/RGN-III/DORS/OB
To:	Constellation Energy Generation
Travis Iskierka-Boggs
Shared Package
21188A269	List: ML23171A952 ML23171A954 ML23171A956 ML23172A178 ML23172A180 ML23172A191 ML23172A194 ML23193A860
References
Download: ML23172A180 (1)

Text

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

1 H

3 B

S Bank. Not on previous ILE Reference Provided: Procedure Attachment PTFM See NOTE 1 - tie to (1)

NRC:

- What is the limit for LPRM oscillations that would drive for a Manual scram while in the OPRM Trip Enabled region?

- Provide explanation into analysis how this reference has been assessed to not assist any other questions on both exams

Response

-From LOA-RR-201 Manual Scram is required when Core Instabilities (2x normal on APRM/LPRM) with OPRM trip capability not maintained.

This question does not assist any other questions on both exams.

2 H

3 B

E/S Bank. Not on previous ILE See NOTE 1 - tie to (4)

NRC:

- Answer choices have three part answers built into them. Is it necessary to have all three parts?

- Distractor C can be easily eliminated based on only selection to remain closed and does not match other formatting

- Answer explanation discusses manual actions taken by the NSO but the question is asking about automatic actions. I would like to discuss this question further for my understanding.

Response

-Answer and distractor explanations have been revised.

Working on this question NRC: New formatting for Q2 is SAT.

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

3 H

3 B

S Bank. Not on Previous ILE See NOTE 1 - tie to (4)

NRC:

4 H

3 N

S See NOTE 1 - tie to (4)

NRC:

5 H

2 N

S See NOTE 1 - tie to (1)

NRC:

- Would the action of distractor 2 ever be enacted under any conditions?

Response: Answer explanation revised to address this.

This action is used in the same section if IRM was at a higher range NRC: Question is SAT 6

M 2

B E/S Bank. Not on previous ILE See NOTE 1 - No to Tier 1 NRC:

-Question simply asks about what components can be operated at the Remote Shutdown Panel. This is a component location question for Tier 2.

- To make Tier 1 you could state that NSOs took action to place emergency transfer switch to emergency position while responding to an event (give conditions) that necessitated control room abandonment. Then ask which SRVs can be operated from the RSP.

Response

Revised stem as suggested.

NRC: Second bullet wording is awkward.

Response: Not changing per REE NRC: Question is SAT.

7 H

3 B

S Bank: 13-1 NRC Exam See NOTE 1 - tie to (1/4)

NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

8 H

3 N

E/S New Question.

Reference Provided: P&ID NRC:

See NOTE 1 -tie to (1/4)

- Answer can be determined using system knowledge not explicitly found in LOA-IA-201 and provided reference. Thus, making a weak-tie to abnormal procedures.

Changes provided on early look document for wording of the stem to remove IA leak thus adding for one more element of assessment to the question.

LAS:

- Distractors B, and C changed to incorporate HDP valve isolations thus making the tie to tier 1 having more integrated plant operations.

NRC:

- Question incorporates more integrated plant ops.

Question is now considered SAT.

- Move Reference P&ID down a line to provide more spacing.

Response: space added in stem as requested NRC: Fix is SAT 9

H 3

N S

See NOTE 1 - tie to (2)

NRC:

- Is there a procedure that provides further discussion on distractor analysis for distractor 2 and 3? Looking for some sort of guidance/verbiage that can backup operational concerns as they may be argued for correct answers by candidates.

Response

-Added reference to TS 3.4.10 in answer explanation.

Explains why answer is 1st priority.

NRC: Change is SAT

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

10 H

4 B

S Bank: LaSalle13-1 ILE Reference provided: Picture of Radmonitors See NOTE 1 - tie to (3)

NRC:

11 H

3 X

N U/S New Question See NOTE 1 - tie to (2)

NRC:

The question provided tests the applicants ability to assess given plant conditions and determine the next applicable steps, making for a good Tier 1 question.

However, the question does not address the generic portion of the K/A by questioning the candidates ability to demonstrate knowledge of terms and definitions from the procedure.

This question could use the given stem and ask something similar to:

1) What is the next required action to control drywell pressure? (using given distractors already used)

2) Ask candidates the definition, if provided in the basis document for LGA-003 (or LGA usage procedure), of the term vent, or some other term that is defined in the document that can correlate to this question.

Incorporating part two of this two-part example provides a direct tie to the Generic K/A portion.

LAS:

- Added second part to the question incorporating definition for the type of release.

NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

- Change to second part addresses generic portion of K/A.

Question is now considered SAT.

12 H

3 N

E/S New Question See NOTE 1 - tie to (4)

NRC:

- How likely is it that a loss of both Divisions of DC power would occur? Operational Validity concern.

LAS - operationally valid per UFSAR

- Recommend incorporating to the stem a statement that addresses that there is no operator response to this question. While the ES-1.2 (formerly App. E) briefing to the candidates states to not assume anything happens that is not specifically addressed in the question, I feel that this question would benefit from adding this statement due to an eventual closing of MSIVs leading to an overpressure event that is corrected by the SRVs.

- Stem formatting required to make question more psychometrically sound. See written exam early look document for specifics.

LAS - changes incorporated as suggested.

NRC: Question is now SAT.

13 F

2 N

S See NOTE 1 - tie to (1)(3)

NRC:

- Question pedigree description of correct answer does not address why the second part of the answer is correct. Provide the document that states purpose

Response

Added TS Bases to explanation for second part of correct answer.

NRC: Changes made are SAT

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

14 F

2 N

S See NOTE 1 - tie to (1)

NRC:

-LGA-003 is being supplied as a reference on a different question. Appears that all values for actions are removed and will not aid in this question.

- Swap distractors A and C making selection A correct.

OR, you could swap both C/D into the A/B positions and move A/B into C/D positions keeping like answers together There is a lot of D and C occurring in this section of the exam.

Response: Moved C/D to A/B corrected numbering on distractors to align with move.

NRC: Changes are SAT 15 H

2 N

S References Provided: LGA-003 Supp Leg; LGA-001 Press Leg See NOTE 1 - tie to (1)

NRC:

- Two LGAs provided that are heavily redacted for all values. (LGA-003 Supp provided elsewhere on exam)

- Description needed that explains how any reference provided has been analyzed and will not provide a direct look up for any other question on the exam.

Response: Removed references NRC: Changes SAT 16 H

4 N

S Reference Provided: Annunciator panel imbedded See NOTE 1 - tie to (4)

NRC:

- What is provided that indicates that only half of the switches picked up the low level?

- Ensure all imbedded images are sized appropriately for candidates use (Assumed issues from program used). Also, when enlarging image, make sure question and distractors are on same page to the best of your ability

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Response: size is appropriate

-added the actual picture of the annunciator panel to the review exam and revised answer explanation.

Save as PDF so that Travis can see what candidates will see NRC: Image is blurry making it harder to read ann.s.

Response

Replaced picture with a new picture Revised answer explanation to address how the information in the picture addresses the question.

NRC: Question is SAT 17 H

3 B

E/S Bank: 09-1 ILE.

See NOTE 1 - tie to (3)(4)

NRC:

- Uncapitalize C in Cooldown for choice A.

Unnecessary capitalization can be viewed as cuing.

Response: corrected as suggested NRC: Question is SAT 18 F/H 2

B E/S Bank Question: 99-01 LaSalle ILE 2020 LAS EXAM See NOTE 1 - Although this question does have a K/A match, there is no tie to Tier 1.

(For your information: there is no defined grading criteria in the NUREG in which a no-tie/weak-tie to tier 1 is graded => Enhancement.)

NRC:

- Provided suggestion to make tie to Tier 1 on the early look document. Requires modifying the stem with additional information that the candidates will be required to assess for the first part of a two-part question.

LAS: New K/A Requested. Question replaced with 2020 NRC Exam Bank Question.

NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

- 2020 ILE exam had this question as an RO question as on this exam. Remove SRO justification for the question if valid as an RO question and intent is to use as RO question.

- Replacement question is SAT.

19 H

3 N

S See NOTE 1 - tie to 4 NRC:

- This question is closely tied to Q#6. Both are asking about SRV operation from RSP.

- Remove the extra bullet following reactor pressure

Response

Removed extra bullet as suggested This question focuses on the Multiple Spurious Operations issue testing the understanding that power to the SRVs NOT operated from the RSDP must have power removed to prevent spurious SRV actuations due to the fire.

Revised answer explanation to clarify the focus of the question.

NRC: Question is SAT.

20 H

3 M

E/S Modified from Monticello 2010 ILE See NOTE 1 - weak tie to (1)

NRC:

- Question can be answered by knowing that drive power is AC and distractors A, B, and D can be eliminated immediately.

- NUREG-1021, ES-4.2, Section D.3 states to not use All of the above as the answer or incorrect distractor.

Answer choice D is a representation of all of the above.

- Suggestion to correct these issues is to include another AC powered distractor and arrange the choices such that recognizing one of them does not automatically eliminate all three incorrect distractors.

Response: No Change NRC: Following discussion, Question is SAT.

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

21 F

3 N

S See NOTE 1 - tie to (3)

NRC:

22 H

3 B

S Bank: 2013-1 LAS ILE See NOTE 1 - tie (2)

NRC:

23 H

3 B

E/S Reference Provided: LGA-003 press control leg, LGA-003 drywell temp leg, LGA-001 level leg See NOTE 1 - tie to (2)

NRC:

- Provide justification to how the provided LGAs will not aid in answering other questions via direct look up.

- Add a trend to reactor pressure in first bullet.

- Question pedigree has the correct KA information following SLIDE to. The KA information under the PRA is incorrect and needs to be removed.

Response: trend added to 1st bullet. Generic KA and SLIDE to deleted Provided References have been removed NRC: Question is SAT 24 H

3 B

S Bank: 2002 LAS ILE See NOTE 1 - tie to (2)

NRC:

- Question is SAT, however is it possible to format the distractors to be more legible?

Response: revised distractors by moving procedure and title in stem and distractors have procedure designator only Placed both procedures on one line NRC: Changes made are SAT.

25 H

3 B

S Bank: Not used on previous exam See NOTE 1 - tie to (4) 26 H

3 N

S See NOTE 1 - tie to (3)

NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

- Understand reasoning for using time in choice D.

However, based on all the other choices being 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> off, the 1-hour delta between C and D does not align with the others (possible cuing).

Response: Changed D to 20:00, and revised distractor information NRC: Changes made are SAT.

27 H

3 N

S 28 H

3 B

E/S Bank : Fermi 2010 RO ILE NRC:

- Consider adding trends following the parameters (rising, lowering, stable).

- K/A tie is to the cause and effect of RHR Injection on RPV not physical connection (K/A match)

Response: Added trends to parameters in stem, revised answer explanation to further justify tie to K/A NRC: Question is SAT 29 H

3 B

E/S Bank: 2006 Columbia ILE NRC:

- Scram is in all CAPS for this question and all lower case in other questions. Ensure consistency for terms used throughout the exam.

- I understand why twelve is spelled out right after the 3 in Mode 3, however it might be helpful to place (12) following the spelled out word.

- 1A RHR HX OUTLET VLV should be located following the valve number following convention used on other questions.

- Distractor analysis does not provide adequate explanation for each distractor but a general overview as to how they are incorrect. Describe specifics for each distractor on how it is incorrect iaw ES-4.2, Section B.6.b Response: Change scram to all lower-case letters, added (12) to stem, moved valve noun name to after

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

valve number, Added fuller distractor explanations for each distractor.

NRC: Question is SAT 30 F

2 B

S Bank: ILE 11-01 LAS ILE 31 F

2 B

S Bank: Not on previous ILE 32 F

3 B

E/S Bank: 13-1 LAS ILE NRC:

- Valve number and nomenclature not formatted similar to the rest of the exam. Move number to the front and place name in all CAPS Response: reformatted nomenclature and placed all caps NRC: Question is SAT 33 F

3 N

S NRC:

- Not required, but would it be beneficial to ask the question, which of the following is the preferred method.in accordance with LGA-RT-203?

Response: added procedure title NRC: Changes are SAT 34 F

2 X

B E/S Bank: Not used on previous ILE NRC:

- SCRAM in all CAPS when not this way on other Qs

- Distractor 1 is not very plausible with pressure being so low compared to reset pressure of 70 psig. Provide a higher value unless 45 psig is typically seen when scram is not reset.

- Distractor D nomenclature is not aligned with numbering and naming as other annunciators. Cuing to D being incorrect.

Response: Change scram to lower case, changed air pressure to 55 psig (note this value used was based on validation comments from incumbents), added full LOR title to D distractor

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

NRC: Capitalize s for scram in distractor B to start the distractor with similar punctuation as others

Response

Changed as requested NRC: Question is SAT 35 H

3 B

S Bank: Not used on previous ILE 36 H

3 B

S Bank: 13-1 LAS ILE 37 H

3 N

S NRC:

- Distractor analysis does not provide adequate explanation for each distractor but a general overview as to how they are incorrect. Describe specifics for each distractor on how it is incorrect iaw ES-4.2, Section B.6.b. Fix typos as well.

Response: Typos fixed, Revised distractor analysis NRC: Changes are SAT 38 F

2 N

E/S NRC:

- Swap distractors B and C to arrange the distractors in order of length. Distractor analysis will need to match changes made as well.

Response: Revised order of distractors and analysis NRC: The ONLY was removed from distractor C after the reorder. If it is desired to remove the ONLY on C, then do the same for distractor B as well.

Response

Both B and C have ONLY NRC: Question is SAT 39 H

4 N

S NRC:

- Distractor 1 analysis does not provide description for second part of the question.

- We discussed previously, but wanted to verify that it is expected that the candidates know location of isolation valve and this is not minutia.

Response: Students train on LOA-NB-101 in the simulator and actions from the LOA direct isolation

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

NRC: Add (1) and (2) to Questions and answers similar to other question formats

Response

Changed as requested NRC: Question is SAT.

40 F

3 X

B S

Bank: 15-1 LAS ILE NRC:

- K/A does not provide system information in question pedigree (Ability to monitor operation of the LOW PRESSURE CORE SPRAY system including: valve operation)

- Question is not written to the K/A. K/A mismatch as there is no tie to LPCS system in the question or distractors.

Response: Per Travis this K/A is incorrect and the correct K/A is a match to the question NRC: Question is SAT. K/A is correct 41 H

3 B

S Bank: Not on previous ILE 42 H

3 B

S Bank Question: LaSalle 13-1 ILE Good K/A match 43 H

3 N

S NRC:

- Suggest adding the prior to the second blank in the stem Response: Added the prior to the second blank NRC: Add question numbers (1) and (2) to answers to match other question formatting

Response

Changed as requested NRC: Question is SAT 44 H

3 N

S NRC:

-Third bullet under the following states lo range sample pump is there a typo on lo or is this correct?

Response: Fixed typo NRC: Changes are SAT

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

45 F

3 N

E/S NRC:

- Distractor analysis for all distractors is referring to 211X, and 212X. These are not the distractors provided for answer choices A, B, or C.

- Does the answer choices need to be changed (assuming not based on unit conventions) or does the distractor analysis?

Response: Fixed distractor analysis Remove numbers in choices NRC: Question is SAT 46 F

3 N

S 47 F/H 3

M S

Modified from 2002-01 LAS ILE NRC:

- Either add the original question to the Modified pedigree so we can evaluate how it has been significantly modified, OR provide a description of the modification that has been made to the question that meets NUREG requirements.

- Question is identified as Memory recall, however candidates are required to remember the power supply and then determine if the supplied action is applicable. Re-classified as Higher cognition.

Response: Added previous version of question and changed Cog level NRC: Changes are SAT

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

48 H

3 B

E/S Bank: 1999 LAS ILE

- The second part to answer choice B (Remain the same) is not plausible, and potentially cuing as it is the only selection.

- Determine if it is the intention to test candidates knowledge on voltage control versus governor control, OR Lead versus Lag.

- Depending on the determination above, the answer choices should all reflect the same questions. As it stands now, the question is testing three concepts through the answer choices.

Suggest keeping it voltage control versus regulator.

This results in only needing to change second part of D to Remain the same without need to call the question UNSAT.

Response: Changed second part of D to remain the same NRC: Add () around 1 and 2s for Questions and Answers in the distractors to format similar to rest of exam.

Response

Changed as requested NRC: Question is SAT 49 F

3 N

E/S NRC:

- Distractor analysis does not provide adequate explanation for each distractor but a general overview as to how they are incorrect. Describe specifics for each distractor on how it is incorrect iaw ES-4.2, Section B.6.b.

- Reference provided for F300A/B shows IA being ported to the valve actuator differently for 300A compared to 300B. Neither valve has a FO/FC indication listed on P&ID. Please provide further references/explanation that shows fail position for these valves being the same fail position.

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Response: Added reference material for the F300A/b valves to show that they both close on a loss of IA and revised the distractor analysis NRC: New formatting is worse psychometrically than before. Add (1) to F300 and (2) to F057 and then use (1) and (2) in distractors accordingly as done in other questions.

Response

Changed as requested NRC: Question is SAT 50 F

3 N

S 51 F

3 N

E/S New Question Question has good K/A Match NRC:

Second part of the question needs rewording to remove redundancy from the answer choices.

Suggested provided on early look document.

LAS - changes incorporated as suggested NRC: Question is SAT.

52 F

2 X

B E/S Bank: Not used on previous ILE NRC:

- Choice C for 8 psid stating that the 8 comes from the 8 minutes as a value of time makes the distractor an implausible distractor. Change this to 4 psid which is associated with the annunciator for better plausibility.

After making the change, arrange the distractors in numerical order from lowest to highest as already indicated => B becomes 4.0 psid, and C becomes 5.0 psid.

- Distractor analysis does not provide adequate explanation for each distractor but a general overview

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

as to how they are incorrect. Describe specifics for each distractor on how it is iaw ES-4.2, Section B.6.b.

Response

Revised distractor replacing 8.0 psid with 4.0 psid to correlate with pressures associated with strainer described in LOP-WS-05.

Explanations for plausibility of each distractor were provided.

Answers provided with arranged in descending numerical order.

NRC: Question is SAT 53 H

3 B

E/S Bank: Not used on previous ILE NRC:

- Answer choices are separated by choice (1) and (2) but the question stem does not provide the question in this format.

Suggest wording as follows:

Based on the given conditions, which of the following describes the effects on the following over the next five minutes:

(1) Reactor power (2) Flow into the SDV

- Remove but the plant will continue to operate at power this cues the applicant to the correct answer

- If using the above formatting suggestion, remove the redundant verbiage to make the question less of an English test on the candidates:

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

(1) Remains at 100% power versus Reduces to lower power level (2) NO flow into the SDV versus Flow to SDV within capacity of the SDV drain

Response

Question stem revise to simplify and indicate the two parts of the question.

Answers simplified/shorted concisely fit the choices listed in the revised stem.

Answers and distractors arranged in an ascending order based on length.

NRC: There is an extra space between choice C and D. Remove this space. Other changes are SAT.

Response

Changed as requested NRC: Question is SAT 54 H

3 B

E/S Bank: 2010 Fitzpatrick ILE NRC:

- The second part of distractor A does not align with the rest of the choices. Change this to Reactor trips on Turbine Control Valve fast closure Wording on question can be changed to reduce amount of reading in the answer choices by incorporating changes seen during the early look and similar to question 53. See suggestion below:

Suggestion:

Based on the above conditions, the Main Turbine Control Valves close at a __(1)__ rate and the reactor trips due to __(2)__.

A. (1) normal

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

(2) Turbine Control Valve fast closure B. (1) rapid (2) high RPV pressure C. (1) normal (2) high RPV pressure D. (1) rapid (2) Turbine Control Valve fast closure

Response

Question stem revise to simplify and indicate the two parts of the question.

Answers simplified/shorted concisely fit the choices listed in the revised stem.

Answers and distractors arranged in an ascending order based on length.

Clarification added to distractor explanation.

NRC: Question is SAT 55 H

N E/S NRC:

- Question is overlap with Scenario 2, Event 5: Rod 46-31 Drifts Out (some overlap is allowed and can be determined by the CE - okay with the question remaining on the exam)

- Choices B, C, and D have multiple actions built into them. Question stem is asking for next action (singular). Suggest finding a way to make choice A multiple actions similar to others and changing stem to What are the next actions..

OR: Breaking pieces out of the current choices and having only one action per selection.

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Response: Additional actions added to the Choice A and Stem revised to address next actions NRC: Question is SAT 56 H

3

?

B E/S Bank: 2008 Fitzpatrick NRC Exam (Still this bank question after K/A Swap???)

NRC:

Depending on how the failure is set up in the stem of the question (ie what is driving the failure) this may or may not be a K/A mismatch for the question. If it is not a match, the question will only need a bit of adjusting to get it closer aligned. If the question cannot be adjusted to meet the K/A then it would be considered as an UNSAT question and would be graded as such.

This information is being provided for your awareness in the grading process of questions that do not meet K/A requirements. The severity of how badly the question matches the K/A has an effect on how the question is ultimately graded.

LAS: K/A Change requested on Form 4.1-1. Bank question used.

NRC:

- K/A swapped to 202001 K6.02

- New question identified as Fitz Bank. Correct?

- New question is SAT 57 H

3 N

E/S NRC:

- Second part of distractors has three variables.

Suggest changing choice B to Withdrawal Rod Blocks or choice D to Both Insert and Withdrawal Rod Blocks - seems more operationally valid to ask only about withdrawal?

- If sticking with Withdrawal, then ensure the W is capitalized throughout the question

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Response

Insert block removed from choice B Only Withdrawal Rod Block is capitalized as it is the title of a function.

NRC: Question is SAT 58 F

3 B

E/S Bank: Not previously used on ILE NRC:

- Break out the question into two separate questions using format previously used on other questions.

(1) What RBM channel (2) What is the RBM system Format the choices with similar formatting.

- This statement in distractor analysis seems out of place:

The other answers are incorrect because APRM 1D is associated with RBM Channel 1B not 1A.

Response

Stem reformatted as a two-part question.

The statement in the distractor analysis was deleted as the information was redundant to what was stated earlier, and the distractor analysis does not need additional justification.

NRC: Question is SAT 59 H

3 N

S 60 F

2 N

S NRC:

- Weak tie to Generic K/A for Conduct of Operations requirements as question is answered using system procedure

Response

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Added reference to OP-AA-20 Conduct of Operations Process Description OP-AA-20 Conduct of Operations Process Description that supports use of procedures to control plant evolutions. Performance is to approved procedures, not this document, that provide detailed instruction on actions required to implement the intent of this process. In the case of controlling plant evolutions that involve placing objects in the fuel pool it is critical for operations to understand the risks of placing objects in the pool and what controls must be put in place to manage this evolution.

This question seeks to determine the operators understanding of inherent risks and why the specific compensatory actions are in place.

NRC: Changes and explanation are SAT 61 F

2 B

S Bank: 2008 Browns Ferry ILE NRC:

- Procedure name should be in all CAPS to coincide with rest of the exam

- Is Distractor A (SED Role) considered RO knowledge? If not, then this needs to be swapped with a different distractor.

Response

Procedure name capitalized Use of the assembly siren subsequent personnel response is understood by the RO, The distractor 1 explanation was revised with additional justification for why it would not be used.

NRC: Changes are SAT 62 H

3 N

S

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

63 H

3 B

S Bank: 2009 Columbia ILE 64 F

2 B

S Bank: 1999-01 LAS ILE 65 F

2 B

E/S Bank: Not used on previous ILE NRC:

Question does not adhere to Tier 3 by asking what the purpose of a specific system is and providing specific actions that the system performs. A way to meet the tier requirements is to provide a definition of a system and ask what system it pertains to and giving multiple systems as distractors. This brings many systems into the question and makes it more plant generic.

Response

Question rewritten with the system purpose stated in in the stem with the choices being 4 different systems.

NRC: Question is SAT 66 F

2 B

S Bank: 2013-1 LAS ILE NRC: on Unit 1 was removed from stem and Mode Switch is in STARTUP removed as well. Was this based on feedback from validation? No reason provided here, or on calls that I remember discussing.

Response

Added information back into question NRC: Question is SAT 67 F

2 N

S NRC:

- Distractor D needs to have panel added following 0PM15J and prior to only.

Response

Added word panel to Distractor D NRC: Changes are SAT 68 F

3 B

S

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

69 F

3 N

E NRC:

- Change 1/4 to 0.25 inch to match procedural reference.

Response

Changed 1/4 to 0.25 inch NRC: Changes are SAT 70 H

3 B

S Reference Provided: Generic Fundamental Equations 71 F

2 B

E/S Bank Question: Is this a LaSalle Exam Bank question that just was not used on previous NRC Exams?

Question partially meets K/A. Definition of burnable poisons is not included in the question. Suggestion provided on the early submittal exam document.

LAS - incorporated feedback provided and made two part question.

NRC:

- Question is SAT.

72 H

3 B

S 73 H

3 B

S Reference Provided: Generic Fundamental Equations NRC:

- Distractor B needs an E added to MW

Response

Distractor B, changed MW to MWE NRC: Changes are SAT 74 F

3 B

S NRC:

- K/A add the word depression following condensate (missed on Written outline and carried forward, question meets the K/A though)

Response

Added depression to K/A description listed in the answer explanation.

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

NRC: Changes are SAT 75 F

2 B

S 76 1

H 4

B S

Reference Provided: LOA-RX-101, Att. A See NOTE 1 - tie to (1)(3)

NRC:

- Provide justification to how the provided references will not aid in answering other questions via direct look up nor provide direct elimination of distractors.

Response: Providing attachment A does NOT provide any guidance to other questions on the exam, this is the only question that requires the SRO to calculate the cooldown rate and to respond as required.

NRC: Question is SAT 77 2

H 3

B S

Bank: ILE LAS 09-01 See NOTE 1 - tie to (1)

NRC:

78 3

H 3

B E/S Bank: Not used on previous ILE See NOTE 1 - tie to (1)

NRC:

- Distractor B and D are written slightly different in that B states one hundred days and D states one hundred day. Make them match so that there is not cuing towards an incorrect answer.

Response: Made both B and D match as requested NRC: Question is SAT 79 4

H 4

X N

U/S References provided (4): HCTL, RCIC NPSH, LGA-003 SP Temp, LGA-001 Press See NOTE 1 - tie to (1)(2)

NRC:

- Provide justification to how the provided references will not aid in answering other questions via direct look up nor provide direct elimination of distractors.

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

- Answer explanation has wrong verbiage for the correct answer. Update the verbiage to match the answer choice that is correct.

- Based on SP Temp and RPV level, point near HCTL is very close to limit and could be argued that Distractor A may also be correct. Moving the parameters further from the limit will help prevent this issue.

- Distractor C and D are essentially the same answer and are both incorrect for the same reason. A candidate that is able to eliminate one for the reason of continued operation will add heat will eliminate both answers.

Response: Fixed answer explanation verbiage to match correct answer.

Change references supplied will only be graphs NC RCIC NPSH Limit and HCTL.

Providing these two references will NOT provide any information to answer any other question on both the RO and SRO exam. These references are required for the candidate to help analyze conditions in the question stem and to come up with the correct answer.

I changed distractor C to OPEN SRVs to reduce reactor pressure until shutdown cooling interlocks clear, and then secure both RCIC and SRV operation.

In stem revised suppression pool temperature to remove operation farther from violating the HCTL violation (I had to adjust drywell pressure to 8 psig to place RCIC as violating NPSH curve)

NRC: Question is SAT 80 5

H 3

N S

See NOTE 1 - tie to (2)

NRC:

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

81 6

H 3

N S

See NOTE 1 - tie to (1)

NRC:

82 7

H 3

B U/S References provided: LGA-004/006. NONE Bank: Not used on previous ILE See NOTE 1 - tie to (2)

NRC:

- Question pedigree states this is Tier 2 Group 1, when the outline is requiring the question to be Tier 1 Group 1. Question is a Tier 1 Question

- Question is not apparent as to how it addresses the Generic portion of the K/A to demonstrate an ability to coordinate personnel activities outside of the control room in the given condition.

- Provide justification to how the provided references will not aid in answering other questions via direct look up nor provide direct elimination of distractors.

- References needs to have the procedure numbers redacted for each reference provided. If this is done, then the references can stay on as provided.

Response: Fixed K/A tier identification Provided in answer explanation why the question meets the generic portion of the K/A statement, because actions are directed outside of the main control room LGA-RI-102 attachment A.

NRC: Question is SAT.

83 8

H 2/3 X

X N

U/S New Question See NOTE 1 - tie to (1)

NRC:

Question has more than 1 implausible distractors (STA used in more than one distractor).

See early look document for further discussion.

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Initial conditions in stem of the question do not appear to be operationally valid.

Question does not meet Generic K/A testing requirements.

LAS: Requested K/A Change. New Question being submitted for approval.

NRC:

- New K/A provided. Bank Question from 19-1 LAS ILE (previous 2 exams)

- Bank question is SAT.

NRC: Add (1) and (2) to the two questions in stem to match the answer choices

Response

Revise as requested.

NRC: Question is SAT 84 9

H 3

B E/S Bank: Not used on previous ILE See NOTE 1 - tie to (2)

NRC:

- Distractor C needs to have the two procedures swapped so that LOA-EH-101 is on the top and LOA-FW-101 is on the bottom similar to the other distractors.

- Question pedigree describes multiple alarms received when only one is present in the IC for the question. The explanation states that an annunciator is received for high reactor pressure is received.

Should this be provided in the initial set up for this question, or are the candidates supposed to be aware that this alarm would come in and it is not something that needs to be assumed?

Response

Revised distractor C as requested Revised answer explanation

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

NRC: Question is SAT 85 10 H

3 N

S See NOTE 1 - tie to (2)

NRC:

86 11 F

2 X

B/M U/E/S Bank Question: Not used on previous NRC Exam NRC:

Question does not meet SRO Level knowledge. See early look document for further discussion.

To aid in SRO level assessment justification, provide answers to, or show flowpath used, in response to the questions provided on Figures 4.2.2 and 4.2.3 in ES-4.2, Section E, Instructions for Developing and Reviewing SRO-Level Questions.

LAS: New question provided from 2012 QC ILE NRC:

Reference Provided: Imbedded Image of Core Mimic

- Question Source in pedigree states Bank, and Question History states, Modified from Quad Cities 2012 ILT NRC Exam. Provide description of the modification that was made to alter this exam question to be denoted as Modified.

- Imbedded image provided of core mimic with SRM locations clearly denoted. Is this graphic necessary to be included with this question? It is system knowledge to know where the core internals are loaded and having this graphic makes the question significantly easier. If it is desired to use a core mimic, then use a PPC image from a screen that would be used during refueling activities and that does not specifically denote the SRM locations (UNLESS this is the screen

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

that is used during refuel). This image appears to not be in a condition ready for core offload with multiple control rods at 12 steps and not operationally valid for the conditions presented.

Response

-Removed core mimic

-revised stem and answer explanation NRC: Question is SAT 87 12 H

2 N

S Reference Provided: LGA-004 NONE NRC:

- Provide justification to how the provided references will not aid in answering other questions via direct look up nor provide direct elimination of distractors.

Response

Removed Reference Corrected spelling errors in answer explanation NRC: Changes are SAT 88 13 H

3 B

S Bank: 2001-01 LAS ILE 89 14 H

4 N

S NRC:

- Asking FLEX operations questions could be considered minutia. However, OLPF FAQ 4.2.16 (401.57) states the following:

Test items on FLEX that meet one or more of the following criteria will provide evaluative benefit for making a licensing decision:

The task is expected to be performed by licensed operators during an extended loss of AC power scenario; or The test item covers a FLEX strategy that is credited in the emergency operating procedures or abnormal operating procedures; or

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

Therefore, the question is documented as SAT for an SRO candidate.

90 15 H

3 N

S Reference Provided: LGA-003 Temp Leg NONE NRC:

- Provide justification to how the provided references will not aid in answering other questions via direct look up nor provide direct elimination of distractors.

- Provided reference cues candidates into distractors C and D not being correct based on not being mentioned in provided reference. See reference document.

Question is SAT without references.

Response: Removed reference NRC: Changes are SAT 91 16 H

3 N

S Reference Provided: ODCM 12.2.1 NRC:

92 17 H

3 N

S 93 18 H

3 N

E/S NRC:

- Formatting and verbiage of the question is odd and needs to be re-written. Suggest writing the question as follows:

The Unit Supervisor will direct the performance of

__(1)__ to recover from Reactor Building abnormal ventilation lineup.

The Unit Supervisor will direct to secure __(2)__

SBGT per LOP-VG-02 SHUTDOWN OF THE STANDBY GAS TREATEMENT SYSTEM (SGBT).

Response

Verbiage revised as suggested.

NRC: Question is SAT

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

94 19 F

2 B

S Bank: 2019 River Bend NRC ILE 95 20 F

2 B

E/S Bank: 2013-01 LAS ILE NRC:

- Distractor C needs to be changed so the wording is not cuing into being an incorrect choice.

Restate as, directly supervising core alterations from refuel floor and is NOT required to be on Refuel Bridge.

Response

Removed NOT required to be on Refuel Bridge. from Distractor C. Also, rearranged choices based on length of choice.

NRC: Question is SAT 96 21 H

3 B

E/S Reference Provided: TS LCO 3.6.1.3 Bank: Not on previous ILE NRC:

- Breakout the question into two separate statements to something similar to:

Which of the following describes the:

(1) Technical Specifications that must be entered, if any; and (2) Why or why not.

Response

Verbiage revised as suggested.

NRC: Question is SAT 97 22 H

3 N

S

Form 2.3-5 Written Examination Review Worksheet Facility: LaSalle County Station Exam Date: May 1, 2023 Early Look Unsat Enhancement Satisfactory

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

6.

Source (B/M/N)

7.

Status (U/E/S)

8.

Explanation General Comments:

NOTE 1: For Tier 1 questions, does it test:

(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?

NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?

NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.

98 23 F

3 N

E/S NRC:

- Under Answer Explanation there are two numbered items that describe a procedure violation and that the Unit Supervisor is the correct answer. These two items make it seem that this question is being used from a previous question and that the question is not actually a new question as identified on the pedigree. Is this question modified from a different question?

- Question is SAT as written. Justification and pedigree needs to be updated IF this is from a modified question. If it is a modified question, explanation needs to be provided as to what was changed to make the question modified.

- Question is identified as High Cognitive when in fact the candidate is required only to recall two memory items from procedure requirements for Rad Worker practices. Re-classified as Fundamental.

Response

Answer explanation revised.

This question is a new question. The explanation discrepancy was an oversight that was not revised following reviews during question development.

NRC: Question is SAT 99 24 F

3 N

S 100 25 H

3 N

S NRC:

- Provided there is no Safeguards, this question is SAT.

Form 2.3-5 Written Examination Review Worksheet Facility: Any Plant Exam Date: 12th of Never Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.