ML22192A205

2021 QC Ile ES-401-9 Written Examination Review Worksheet-Final

ML22192A205
Person / Time
Site:	Quad Cities
Issue date:	07/09/2021
From:	Randy Baker NRC/RGN-III/DRS/OLB
To:
Baker R
Shared Package
ML20139A015	List: ML22192A193 ML22192A195 ML22192A196 ML22192A203 ML22192A204 ML22192A205 ML22192A206
References
Download: ML22192A205 (22)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).

T/F: The answer choices are a collection of unrelated true/false statements.

Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

Backward: The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only. (K/A and license-level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).

Page 1 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only Generic Comments Randomize the location of the correct answers. The first 4 questions all have correct answers of A and then the next 3 questions all have the correct answers of B. In the first 20 questions answers of C and D are only used once. C has been used only once in the first 25 questions.

Additionally, The distractors are not numbered sequentially. The distractor numbering should be consistent between the Exam and the Question Justifications.

NOTE 1: This question does not meet the requirements of FAQ 401.55 in that it doesnt test information 1) from the AOP/EOPs 2) diagnosis of procedures to be used to address the situation 3) progression of an event, OR 4) assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions.

Updated to meet tier 1 requirements NOTE 2: Bank questions need to be updated to current quality requirements. This applies to all Bank questions.

NOTE 3: Bank questions need to be updated with previous use history if used on a previous NRC exam, not simply annotated as Bank used on this exam.

Modified use of bank to include past NRC exam usage.

ES-401 2

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 2 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 1

F 3

X N

E S

NRC: Question is not specific and should be changed to What Immediate Operator Action is required per a specific procedural reference. Is there actually any Immediate Action required due to the conditions stated in the stem? (Not from QCAN 901(2)-5 H-1, or QCOA 0202-04.) The QCAN for 901(2)-0 H-1 does not reference QCOA 0700-03 so why would someone prioritize this QCOA over 0202-04 for the Recirc pump trip? Need a stronger tie to the Loss of Neutron Flux Indication QCOA.

Perhaps another Annunciator alarms.

Reference includes QCOA 0202-04, Revision 08 which should be revision 48. In addition, this reference does not have an immediate operator action to hold power constant. This reference should be removed. (Based on the first statement in the stem, this QCOA is appropriate.)

Distractor 3 justification needs to be reworded/corrected.

Response

Updated to reference actions required by QCOA 0202-04.

Question is now SAT.

2 H

2 N

E S

NRC: See NOTE 1. Believe this meets Tier 1 Objective. (3) Need to see the QCANs referenced in the stem (A1, D3, F3)

Both parts of the 2 x 2 question should be written to address the K/A and the first part of this question is related to diagnosis and not related to the K/A. The first part does tie to the K/A for determining the cause for loss of AC, and the second part is a loose tie to procedure QCOA 6500-04. Stem question needs to be more targeted to a specific abnormal procedure.

The Correct Answer rationale and the Distractor justifications needs to be clarified (Dash Buses).

Response

Added 901-8 A1 4KV MAIN FEED BREAKER TRIP to stem to tie to loss of specific bus procedure.

Removed dash bus and put bus nomenclature for distractor analysis.

Question is now SAT.

3 H

3 N

E S

NRC: See NOTE 1. Believe this meets Tier 1 Objective. (4)

Since the answer and the distractors relate to the starting and control of Unit 1 EDG have the question focus on the impact to the starting and control circuit. Instead of stating the drywell pressure for getting the applicant to focus on the starting of the Unit 1 EDG state how is the Unit 1 EDG start affected. Discuss whether to reword the stem question to better focus the stem on immediate actions.

Response

Focused stem by using starting of the EDG and reworded distractors for clarity.

Question is now SAT.

ES-401 3

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 3 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 4

H 2

B E

S NRC: See NOTE 1 and NOTE 2. Believe this meets Tier 1 Objective. (4) Add AOP Reference (QCOA 5650-01) to the Question justification and verify justification complete.

QC NRC 2011 ILE Cert

Response

Added reference to pedigree Question is now SAT.

5 F

2 N

E S

NRC: See NOTE 1. This question has a very weak tie to the Tier 1 Objective. (3)

Perhaps add a different IC to the stem stating, The ATC has not attempted to reset the Scram and drain the SDV instead of No operator action has been taken.

Response

Modified stem to follow scram actions per QCGP 2-3 Question is now SAT.

6 F

2 B

S NRC: See NOTE 1. This question has a very weak tie to the Tier 1 Objective. (3) This question is labeled as New when it is a copy of Question 6 from the 2020 NRC ILT exam. Labeled it as a Bank question and that it was last used in the 2020 NRC ILT QC NRC 2020 ILE - Q6 (PREVIOUS 2 EXAMS)

Response

Question as written is SAT.

7 H

3 X

N E

S NRC: See NOTE 1. Believe this meets Tier 1 Objective. (3)

The distractor explanations and the possible answers do not line up. For example, Answer A is CRD and Turbine Bypass Valves yet Distractor 1 explanation addresses RWCU and HPCI.

The stem states that Both units are at 100% when a total loss of TBCCW has occurred and that QCOA 3800-03 has been entered and executed. Confusing as to the status of the reactor since it cant be both at 100% and QCOA 3800-03 calls for a SCRAM. Change the tense. The confusion results from providing subsequent data for only 1 Unit. Clarify for both Units.

Distractor 2 states that CRD is cooled by TBCCW and is secured yet QCOA 3800-03 states that a CRD can be left in operation if the bearing temperatures are monitored.

Since QCGP 2-3 Attachment F places bypass valves in service this means that A is also a correct choice. Again, need to specify using the ICs in the stem what entered and executed actually means for step completion within the QCOA. Closing the MSIVs and securing the EHC pumps removes BPVs as a viable option, even if CRD is still in service. (How is RWCU used for Pressure control, simply lowering RPV temperature? Maybe swap around?)

Response

Focused stem to a specific unit and to say steps of procedure are complete

ES-401 4

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 4 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only Question is now SAT.

8 H

2 N

E S

NRC: See NOTE 1. This question has a very weak tie to the Tier 1 Objective. (3)

There is a reference to the Loss of Instrument Air QOA in the explanation but there is no need to use knowledge from the QOA in order to answer the question. Remove the reference. Since the answer is supported by the QOA info, the reference is OK.

Response

Added Procedure and bullets to stem.

Question is now SAT.

9 H

3 B

E S

NRC: See NOTE 1 and NOTE 2. Believe this meets Tier 1 Objective. (3) Add AOP Reference (QCOA 1000-04) to the Question justification and verify justification complete.

The stem includes a vague phrase of In accordance with plant procedures. Add the reference to specific procedures in the stem. (Consider rewording.)

QC NRC 2011 ILE Cert

Response

Added procedural tie to stem QCOA 1000-04 Question is now SAT.

10 H

3 B

E S

NRC: See NOTE 1. This question does not appear to have a tie to the Tier 1 Objectives. (4?)

Stem is not asking a procedure based question. Please provide procedure that addresses this situation and include as a reference in the question justification.

(QCOA 0010-08?)

Response

Added drywell evacuation procedure to stem QCOS 0010-08 Bank, not on previous NRC exam Question is now SAT.

11 H

2 N

E S

NRC: See NOTE 1. This question has a very weak tie to the Tier 1 Objective. (3 or 4?)

Stem is not asking a procedure based question. Which procedures directs this action? Please provide procedure that addresses this situation and include as a reference in the question justification.

Response

Added QCOP1000-30 which directs actions in question Question is now SAT.

12 F

2 N

E S

NRC: See NOTE 1. This question does not appear to have a tie to the Tier 1 Objectives. (1 or 4?)

ES-401 5

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 5 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only Stem is not asking a procedure based question it is asking a general systems knowledge question taken from the UFSAR or a lesson plan.

QCOA 0202-04 reference does not address 1200 psig trip from the ATWS system.

Add the correct reference. Please provide procedure that addresses this situation and include as a reference in the question justification.

Response

Added EOP tie ATWS trips. Added reference.

Question is now SAT.

13 H

3 N

E S

NRC: See NOTE 1. Believe this meets Tier 1 Objective. (3)

Add ICs to provide initial Unit power levels and allow determination if EOP entry was required and which QCOP is applicable. Clarify grammar in the ICs. Change MAXIMUM to MAXIMIZE in the question stem.

To discriminate distractor B from correct answer A the candidate must know from memory bus loading limits and that flow must be maintained under 7600 GPM. To discriminate distractor D from the correct answer the candidate is expected to have memorized the precautions and limitations of a normal operating procedure.

Provide Lesson Plan objectives to validate that this is required RO knowledge.

Response

Added bullets and procedure name to stem Added objective SR-1000-K21 to answer justification.

Question is now SAT.

14 H

3 N

S NRC: See NOTE 1. This question has a very weak tie to the Tier 1 Objective. (1 or 4?)

Question is acceptable but is better suited to either Tier 2, Plant Systems or Tier 3, Generic Knowledge.

Embedded

Reference:

Response

This question is SAT as written.

15 F

3 X

N B

U S

NRC: This question must be replaced because it does not match the Generic K/A and tie that to the required EPE. The replacement question must also meet the Tier 1 Objective.

If this question is put in the bank for a Tier 3 question, it must be reworded to provide only 1 correct answer based on ICs given, and relate to something other than Immediate Actions when referring to EOP action steps. (Correct answer given as A because DW air temperature is above the QGA 200 entry conditions. No other QGA entries are met. Yet QGA 200 limit Drywell temperature limit is 180 degrees and the conditions given in the question is 140 degrees. Drywell pressure is given as 2.5 psig and the entry condition for DW pressure is above 2.5 psig. Since a trend is not provided, but DW pressure is rising, this could mean meeting the entry condition for

ES-401 6

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 6 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only DW Pressure as the QGA criteria is to hold DW pressure below 2.5 psig using coolers and torus sprays. Answer C would then be considered partially correct and possibly D.

Response

Modified question to focus on low torus level and actions which now matches the K/A.

Bank not used on previous NRC exam Question is now SAT.

16 H

3 N

E S

NRC: See NOTE 1. Believe this meets Tier 1 Objective. (1 or 3?)

DW Pressure and Temperature are not required in the stem as they have no impact upon the correct answers or the distractors. Generic feedback - frequently there is additional information that is not required to reach the correct answer or to add to the distractors effectiveness. Reduce or eliminate the extra information.

Reword the stem for clarity. Please provide a more specific reference for where this is found in the EOP bases, and the learning objective that validates this is required RO knowledge

Response

Added basis and page for reference and QDC specific objective. Removed irrelevant conditions from stem.

Question is now SAT.

17 F

2 N

N U

S NRC: See NOTE 1. This question does not appear to have a tie to a Tier 1 Objective.

K/A mismatch as the stem does not address SCRAM Condition Present and Reactor Power above APRM Downscale or unknown The stem only addresses ARI as a system question.

Perhaps provide ICs which establish the context of when to use the ARI, as well as how it works.

Distractor justification needs to be clarified since the same plausibility rationale is given for the two different actions. Both are not correct. Also, please clarify the Reference used.

Response

Question replaced.

Replacement question is SAT.

18 F

3 N

E S

NRC: : See NOTE 1. This question has a weak tie to the Tier 1 Objective. (3 or 4?)

Please annotate which procedure is address for this situation and include as a reference in the question justification. (Is the QCAN credited as the Emergency/

Abnormal procedure referenced?) Will a Group 2/3 Isolation also occur?

Also provide a more specific section reference from the UFSAR

Response

ES-401 7

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 7 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only Added QCAN to procedure. Added UFSAR page reference.

Question is now SAT.

19 H

3 N

N U

S NRC: This question must be replaced because it does not match the K/A and tie to the required APE.

The replacement question must also meet the Tier 1 Objective.

K/A mismatch as there is no tie to a plant fire on site.

Response

Replaced with a procedure to fit K/A of fire on site and Tier 1 objective.

Replacement question is SAT.

20 H

3 N

E S

NRC: See NOTE 1. Believe this meets Tier 1 Objective. (3)

May need to clarify the stem question, and reword the Distractors in a 2 x 2 format to clearly address the stem question and matches the stem question format.

Is Distractor A more plausible if degraded voltage is > or = to 75% of nominal?

Distractor justifications should be clarified.

Response

Reworded stem for clarity and modified nominal voltage value as requested.

Question is now SAT.

21 H

3 N

S NRC: See NOTE 1. Believe this meets Tier 1 Objective. (3)

Response

This question is SAT as written.

22 H

3 N

N U

S NRC: See NOTE 1. Believe this meets Tier 1 Objective. (1)

This question may not match the K/A as written. The question states that the 70%

core flow automatic runback is to prevent a Rx trip on low RPV water level. The K/A concerns the reason for a Recirc pump runback if a low RPV water level occurs.

Please explain the 32% speed runback - if this is also an automatic runback, the stem must be reworded as it asks for the reason the automatic runback occurs.

Response

Modified stem to follow QCGP 2-3 reactor scram and clarify reasons for the recirc runback.

Question is now SAT 23 H

3 N

S NRC: See NOTE 1. Believe this meets Tier 1 Objective. (1 or 3)

Response

This question is SAT as written.

24 H

3 N

E NRC: See NOTE 1. Believe this meets Tier 1 Objective. (1)

ES-401 8

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 8 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only S

Please provide procedural guidance for not initially attempting to individually scram the withdrawn control rod. Clarify the Distractor justifications.

Response

Added procedure guidance for initially manually inserting vs individually scramming..

Question is now SAT.

25 H

3 N

N U

S NRC: This question must be replaced because it does not match the K/A and tie to the required APE.

K/A mismatch - the K/A is Ability to interpret and execute procedure steps yet no specific procedures or steps are given in the stem. Smoke in the area of the CRD pump motor vents is vague, need specific info if this is to be used for a decision.

Response

Replaced question.

Replacement question is SAT.

26 F

2 X

N E

S NRC: See NOTE 1. This question has a weak tie to the Tier 1 Objective. (3 or 4?)

Answer B and D both use operable and are true. Max safe values are well above the operability values used for motors, wires and controllers. Operability is not typically used in conjunction with the UFSAR. Change to meets environmental qualification limits or mission time as a possible answer. This would more closely tie to the K/A for personnel protection. Maintaining any form of operability could be associated with personnel protection by assuring the ability to safely shut down the plant.

Reword Distractors B & D to preclude more than 1 correct answer.

The distractor analysis contains errors. Clarify/correct the distractor justifications.

If possible choose the PSTG EOP bases document as the reference in place of the BWROG EPGs/SAGs.

Response

Added QDC specific reference for definitions. Also modified distractors to ensure safe is in more than one.

Question is now SAT.

27 F

2 N

E S

NRC: See NOTE 1. Believe this meets Tier 1 Objective. (1 or 3)

If possible choose the PSTG EOP bases document as the reference in place of the BWROG EPGs/SAGs.

Reword Distractor C for a more complete correct answer.

Response

Added QDC EOP reference. Clarified distractors by adding processed Question is now SAT.

28 H

2 N

U NRC: Question had same answer as #9

ES-401 9

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 9 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only S

Response

. Replaced this question with new question.

Replacement question is SAT.

29 H

3 B

S NRC:

QC NRC 2012 ILE - Q?

Response

This question is SAT as written.

30 F

5 X

N U

S NRC: Is there a learning objective that requires the applicants to know all of the limitations and actions of the QCOPs?

Please provide a lesson plan objective requiring operators to know when Primary System Boundary Thermal Limits apply, and add to references in question justification.

Response

No QDC objective tie to knowledge requirements. Replaced question to required knowledge to tie to K/A Replacement question is SAT.

31 F

3 N

S NRC: None.

Response

This question is SAT as written.

32 H

3 N

S NRC: None.

Response

This question is SAT as written.

33 H

3 N

S NRC: None.

Embedded

Reference:

Response

This question is SAT as written.

34 F

3 N

E S

NRC: The question stem begins with the unit at 100% and need for a manual SCRAM but does not reference a need for a control room evacuation or the entry into procedure QOA 0010-05. Add this information. Consider rewording the stem to simply state that with the Unit at 100% power, a control room evacuation is required; then ask the stem question as is.

Response

Modified stem to add clarity and brevity.

Question is now SAT.

ES-401 10 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 10 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 35 H

3 N

S NRC: Consider removing the RPS channel identifier from the correct answer.

Response

Removed identifier.

This question is SAT as written.

36 F

3 N

E S

NRC: Need to change the connectors (OR) to preclude C also being a correct answer.

Response

Modified distractor per comment.

Question is now SAT.

37 H

3 X

N E

S NRC: Distractor A references a loss of 24 / 48 VDC power while answer D references a total loss of 24 / 48 VDC power. If Distractor A should be a total loss of DC power then it should so state.

The stem should have bullets and include the annunciator number The stem should be clarified to focus on the SRMs and not the vague plant response Mismatch between the Distractor explanations and the order of the possible answers While a spike in both trains of the power supply is unlikely it has happened at other facilities due to a lightning strike and for other causes. Distractor C neither states that the spike is in only one Bus or that it isnt. This could lead to confusion and a failure to eliminate it as a possible answer. It could be seen as true.

Need to have the annunciators correctly listed (as on the rest of the exam) and shorten the Distractor B rationale to be more consistent with the other distractors/answer.

Response

Modified annunciators. Modified distractors to answer stem.

Question is now SAT.

38 H

3 N

E S

NRC: Distractor D is the only one with a specific TS reference. Add at least one other TS reference Answer A is the only one to include APRM 1 and that is an obvious clue.

In the answer explanation please include the reference for the applicant being required to know the +1 and -2% TS limits.

Response

Modified distractors per comment.

Question is now SAT.

39 H

3 N

E S

NRC: Unclear if knowledge of the ARPs being tested or knowledge of 1300-04. This question is below the line. Is there a system knowledge requirement for the RCIC with a decision using a subsequent operator action step? Please add this reference.

ES-401 11 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 11 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

Response

Reference added to distractor analysis.

Question is now SAT.

40 H

3 B

S NRC: None.

QC NRC 2016 ILE - Q?

Response

This question is SAT as written.

41 H

3 B

S NRC: None.

QC NRC 2009 ILE - Q?

Response

This question is SAT as written.

42 H

3 N

S NRC: None.

Response

This question is SAT as written.

43 H

3 N

E S

NRC: All Distractors are true. The stem asks for conditions at two minutes and then says that the Distractor is false because reactor water level will be restored to normal at five minutes. (Is this a Typo for the 2 minutes in the stem? Should it be 5?)

If 3 of the Distractor all have MWe lowering, how is this addressing the K/A predict/

monitor Turbine load?

Response

Added 5th parameter to distractors Changed the distractor one answer explanation to read 2 min.

Question is now SAT.

44 H

3 N

N U

S NRC: K/A mismatch as the K/A calls for knowledge of alarm setpoints and control operations identified in an ARP but the stem is focused on a QCOA.

Incomplete distractor analysis, Distractor 3 was not completed.

Distractor B Start Torus Cooling is also true given the stem conditions of a SCRAM with the MSIVs shut.

Response

Replaced question.

Replacement question is SAT.

45 H

3 N

E S

NRC: Answer explanation has numerous missing words and initialisms that are not spelled out. Clarify/correct question and Distractor justifications.

Response

ES-401 12 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 12 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only Corrections made for distractor analysis.

Question is now SAT 46 H

3 N

E S

NRC: The K/A calls for a loss or malfunction of standby gas. If Bus 19 is the malfunction then there needs to be a tie between the possible answers and the loss of Bus 19.

Response

Added bullets to indications, removed last (repeated) question from stem, and added the power supply to justification to clarify plausibility.

Question is now SAT.

47 F

3 N

E S

NRC: The single phase fault can be removed and replaced with just a trip of line 0404. Consider rewording the stem question.

Response

Removed single phase.

Question is now SAT.

48 F

3 N

S NRC: None.

Response

This question is SAT as written.

49 F

3 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam This question is SAT as written.

50 F

2 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam This question is SAT as written.

51 H

2 N

E S

NRC: Answer explanations and Distractors dont line up

Response

Answer explanations lined up.

Question is now SAT.

52 F

1 B

S NRC: See NOTE 3. The question focus is unclear as it does not reference a procedure and thus almost anything the applicant chooses to do would be true. The answer explanation references QOA 4700-06 this should be placed in the stem.

Immediate action required by the annunciator response; not needed for stem clarity.

Response

ES-401 13 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 13 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only Bank, not on previous NRC exam Question is now SAT.

53 H

3 N

S NRC: None.

Response

This question is SAT as written.

54 F

2 N

S NRC: None.

Response

This question is SAT as written.

55 F

3 B

S NRC: See NOTE 3

Response

Bank not used on previous NRC exam.

This question is SAT as written.

56 H

3 B

S NRC: See NOTE 3 Reference provided: M-47 Sheet 1, Diagram of RWCU Piping.

Response

Bank not used on previous NRC exam.

This question is SAT as written.

57 F

3 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam.

This question is SAT as written.

58 H

3 B

E S

NRC: See NOTE 3. Correct answer B is the only answer that has two RHR pumps by themselves as a possible selection. All other selections involve both RHR and RHRSW pumps. Add RHRSW pumps to answer B.

Please clarify question and distractors justifications.

Response

Added clarification to question and distractors.

Bank, not on previous NRC exam.

Question is now SAT.

59 H

4 N

S NRC: None.

Embedded

Reference:

Response

This question is SAT as written.

ES-401 14 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 14 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 60 F

2 B

S NRC: See NOTE #

Response

Bank, not on previous NRC exam.

This question is SAT as written.

61 H

3 B

E S

NRC: The stem asks for the LEAST amount of heat addition to the torus and then offers two answers that will not add heat to the torus under any condition. Either the stem should be rephrased or two of the answers should be modified.

Reword the stem question to make it specific to this event.

Response

Stem modified Bank not used on previous NRC exam.

Question is now SAT.

62 H

3 B

E S

NRC: See NOTE 3 Reference provided: Power-to-Flow Map for Normal Feedwater Temp.

Please explain why this reference is necessary. If needed, consider providing both Normal and Reduced.

Response

Reference Handout removed; added more explanation for distractor analysis.

Bank, not on previous NRC exam.

Question is now SAT.

63 F

3 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam.

This question is SAT as written.

64 H

3 N

S NRC: None.

Response

This question is SAT as written.

65 H

3 N

E S

NRC: Not stating the reactor water level that is reached will leave the applicant room to select the level that they desire. Clearly state the water level that is reached.

Consider replacing normal RPV water level transient with level response, or provide a lesson plan objective that validated the response is required knowledge.

Response

Modified question. Added group 2 isolation has occurred which clearly defines the RWL reached.

Question is now SAT.

ES-401 15 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 15 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 66 F

3 B

E S

NRC: See NOTE 3 K/A Mismatch - The K/A focus is on vital area and controlled area access while the stem focuses on control room access that is primarily due to operational and noise constraints.

Please provide justification for the Control Room being designated as a Vital Area.

Response

The control room is a vital/controlled area.

Bank, not on previous NRC exam.

Question is now SAT.

67 F

2 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam.

This question is SAT as written.

68 H

3 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam.

This question is SAT as written.

69 H

3 B

S NRC: None.

QC NRC 2011 ILE Cert - Q?

Response

Bank, not on previous NRC exam.

This question is SAT as written.

70 F

2 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam.

This question is SAT as written.

71 H

2 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam.

This question is SAT as written.

72 F

3 B

S NRC: None.

LaSalle NRC 2007 ILE - Q?

Response

ES-401 16 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 16 of 18 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only This question is SAT as written.

73 F

2 B

S NRC: None.

QC NRC 2012 ILE - Q? Vermont Yankee NRC 2010 ILE - Q?

Response

This question is SAT as written.

74 F

2 B

S NRC: None.

QC NRC 2011 ILE Cert - Q?

Response

Bank, not on previous NRC exam.

This question is SAT as written.

75 F

3 B

E S

NRC: See NOTE 3. The question stem asks which of the following responses is within the guidance of plant procedures? since inserting a manual reactor scram is allowed at any time if the crew deems it necessary this would make Answer A true as well as Answer B It would be better to ask what actions is called for in QCAN 901-5.

Similarly, for C and D also.

Reword the stem to compare responses to guidance provided in the applicable QCAN.

Response

Stem reworded and procedure referenced actions added.

Bank, not on previous NRC exam.

Question is now SAT.

ES-401 2

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 2 of 7 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 76 F

3 B

E S

NRC: See NOTE 3 Distractor 1 (A) is not plausible as QGA 500-4 Detail P3 requirements state that with 2 ADS valves open RPV pressure is 829 psig and the distractor states that with 2 ADS valves open RPV pressure is 800 psig.

Should this 3 ADS valves open? Yes-Typo.

Reference Provided: QGA 500-4 (Modified?)

Response

Modified distractors. All QGA references have entry conditions removed.

Bank, not on previous NRC exam.

Question is now SAT.

77 F

2 B

E S

NRC: See NOTE 3 K/A 295004.AA2.02 states Partial or Complete Loss of D.C. Power: Ability to determine and/or interpret the following as they apply to PARTIAL OR COMPLETE LOSS OF DC POWER: Extent of partial or complete loss of DC power. The stem of the question tells the SRO that the charger is inoperable. It does not allow the SRO to determine that it is inoperable. Need to reword the 3rd sentence to remove the term an operable charger from the sentence; this will allow applicant to determine charger status.

Reference Provided: TS LCO 3.8.4 (LCO and applicability statements deleted) &

SRs

Response

Removed operable charger wording.

Bank, not on previous NRC exam.

Question is now SAT.

78 H

2 B

E S

NRC: See NOTE 3 Knowledge of the procedure numbering system and title will give the applicant the correct answer (i.e., LOD 1). Specifically, QCARP 0060-01, CT-1 is for Unit 1, Unit 2 is QCARP 0060-02 and CT-1 is for Cable Tunnel Unit 1 whereas CT-2 is for Cable Tunnel Unit 2.

Are the QCARPs considered Abnormal procedures? How does this match the K/A?

Reference Provided: QCOA 001012 Attachment D

Response

QCOA 0010-12 is an abnormal procedure which references you to the QCARP which is a procedure to combat attachment r fires which is abnormal.

Bank, not on previous NRC exam.

Question is now SAT.

ES-401 3

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 3 of 7 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 79 H

3 N

E S

NRC: The correct answer (A) is the only answer that states that shutdown cooling has isolated. B and D are not plausible because shutdown cooling automatically isolates at 100 psig in the RPV.

Remove the statement Shutdown Cooling has isolated. from A and reword B to be plausible.

Response

Removed statement per above.

Added RHRSW pressure to stem to make distractor plausible by forcing examinee to analyze given pressure from normal.

Question is now SAT.

80 H

3 B

S NRC: See NOTE 3 Reference Provided: EP-AA-1006, Hot & Cold Matrix (Cueing for other questions?)

Response

Bank, not on previous NRC exam.

Question is SAT as written.

81 H

3 N

S NRC: None.

Reference Provided: QGA 101 & 200 (Modified?)

Response

Initial conditions removed from EOPs.

Question is SAT as written.

82 H

3 N

S NRC: None.

Response: Question is SAT as written.

83 H

2 N

S NRC: None.

Response: Question is SAT as written.

84 H

3 B

E S

NRC: See NOTE 3. The K/A reference is missing. Answer choice C is not clear and would potentially confuse examinees. Suggest rewording Answer choice C.

Reword C to ensure not potentially correct.

Reference Provided: QGA 200 (Modified?)

Response

Modified distractor; added K/A reference to question justification.

Bank, not on previous NRC exam.

Question is now SAT.

85 H

3 N

E S

NRC: QGA 300 entry condition for reactor building area water level is above 1 in.

The level given in the stem is that water level in the basement area of the Torus is +1 in. and slowly rising, therefore the entry condition is met.

ES-401 4

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 4 of 7 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only The question and answer is not a K/A match. K/A 295036 EA2.03 Ability to determine and/or interpret the following as they apply to SECONDARY CONTAINMENT HIGH SUMP AREA WATER LEVEL: Cause of the high water level.

The question asks what actions must be directed, not what the cause of the high water level is which would satisfy the Ability to determine and/or interpret.

Reword ICs to allow applicant to determine where the leak originates and which EOPs are entered.

Reference Provided: QGA 200 & 300 (Modified?)

Response

Changed basement area water level to +2 inches to better match K/A, per comment during onsite validation.

Question is now SAT.

86 H

3 B

E S

NRC: See NOTE 3. Are the Distractors in the correct order? Distractor 1 (B) does not explain why No action Required is plausible. Analysis does not address this issue.

Reference Provided: TS LCO 3.1.7

Response

Modified distractors for length and updated no action required.

Bank, not on previous NRC exam.

Question is now SAT.

87 H

3 N

E S

NRC: Too much information. This would be a good 2x2 question-what TSs and why.

Reference Provided: TS LCO 3.3.6.2, 3.3.8.2, & 3.6.1.3

Response

Clarified stem.

Question is now SAT.

88 H

3 N

E S

NRC: The distractors contain cues. Only one answer contains a procedure that addresses the ESS Bus as a whole. The other three address individual equipment on the bus or plant effects caused by individual components on the bus.

Change distractor C to < 100% and potentially re-structure to a 2 x 2 with Bus power supplies and distractor C 2nd part.

Response

Modified to a 2x2. Modified distractors to initiate an emergency power reduction to limit power increase.

Question is now SAT.

89 H

4 B

S NRC: See NOTE 3 Reference Provided: TS LCO 3.8.1

ES-401 5

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 5 of 7 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

Response

Bank, not on previous NRC exam.

Question is SAT as written.

90 H

2 N

E S

NRC: Distractor # 2 - Is implausible in that the equipment must be acquired from off-site and will take 8 to 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> to be delivered on-site and one to two shifts to be setup and placed in-service. Header pressure is 78 psig and lowering. At 75 psig QOA 4700-06 directs the closing of 1-3207A, FW TO LO FLOW OUTLET ISOL VLV.

Response

Added second full loss of instrument air to stem.

Question is now SAT.

91 H

2 B

S NRC: See NOTE 3 Is Supercritical the correct terminology? (Critical?)

Response

Modified to critical for clarity.

Bank, not on previous NRC exam.

Question is SAT as written.

92 H

2 B

E S

NRC: See NOTE 3 Need to correct the referenced K/A in the justification. (Section 3 of the Generic K/As not permitted for Tier 1 & 2 => 2.3.11 not allowed.)

Reference Provided: CY-QC-170-301 ODCM, pages 43-46

Response

Modified K/A to 268000 2.2.38.

Bank, not on previous NRC exam.

Question is now SAT.

93 H

3 B

S NRC: See NOTE 3 Reference Provided: TS LCO 3.7.6, TRM section 3.7

Response

Bank, not on previous NRC exam.

Question is SAT as written.

94 F

2 X

B U

S NRC: See NOTE 3 This is not a K/A match. The K/A: 2.1.4 Knowledge of individual licensed operator responsibilities related to shift staffing, such as medical requirements, no-solo operation, maintenance of active license status, 10CFR55, etc. This question does not address any of the seven items listed in 10CFR55.43(b)(1-7) and therefore is not

ES-401 6

Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 6 of 7 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only a valid SRO question. (Im not sure how you Turn over Authority of the SM without an actual relief. Are you referring to Emergency Director Duties?)

Response

Modified distractor to clearly reference emergency director responsibilities.

Bank, not on previous NRC exam.

Question is now SAT.

95 H

3 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam.

Question is SAT as written.

96 F

2 X

X B

U S

NRC: See NOTE 3 This K/A 2.2.21: Knowledge of pre-and post-maintenance operability requirements for a SRO has an importance level 4.1. The proposed question does not demonstrate competence of pre-and post-maintenance requirements for an SRO, only knowledge of which procedure provides standard process requirements and expectations.

Response

Modified stem for clarity, better match the K/A, and preclude multiple correct distractors.

Bank, not on previous NRC exam.

Question is now SAT.

97 H

3 B

S NRC: None.

NRC 0216 ILE - Q?

Response: Question is SAT as written.

98 H

3 N

S NRC: None.

Response: Question is SAT as written.

99 H

3 N

E S

NRC: Answer C: If technically ok, change to Lower RPV pressure to 700 psig (same as D)

Response

Changed Distractor wording.

Question is now SAT.

100 H

3 B

S NRC: See NOTE 3

Response

Bank, not on previous NRC exam Question is SAT as written.

ES-401 18 Form ES-401-9 Facility: QUAD CITIES, UNITS 1 & 2 Exam Date: JUNE 1 - 14, 2021 Page 18 of 18 RO TOTALS:

B= 29 F= 30 E= 34 [S=33]

Additional Notes: *Limit on H LOK questions is 38-45.

M= 0 H= 45*

U= 8 N= 46 SRO TOTALS:

B= 15 F= 4 E= 12 Additional Notes:

M= 0 H= 21 U= 2 N= 10

GENERAL COMMENT

S:

1. Question numbers highlighted as xx are sample test items the facility submitted for pre-review. (No questions were submitted for pre-review.)

2.

Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N

3.

Questions from the previous 2 NRC Exams: __1__ (RO)/ __0__ (SRO)

4. Average difficulty is ____ on the RO exam and ____ on the SRO exam.

5.

The 10CFR55.41/43 distribution is: RO / SRO 41.1 =

41.8 =

43.1 =

41.2 =

41.9 =

43.2 =

41.3 =

41.10 =

43.3 =

41.4 =

41.11 =

43.4 =

41.5 =

41.12 =

43.5 =

41.6 =

41.13 =

43.6 =

41.7 =

41.14 =

43.7 =

6.

The answer distribution is:

RO

/

SRO A =

/

B =

/

C =

/

D =

/

7.

There are __3__ (RO)/ __11__ (SRO) questions with references/attachments provided.