ML23247A008

NRC Comments on Draft Licensee-Developed Written Exam

ML23247A008
Person / Time
Site:	Ginna
Issue date:	09/28/2022
From:	Joseph Demarshall NRC/RGN-I/DORS/OB
To:	Constellation Energy Generation
References
EPID L-2022-OLL-0008
Download: ML23247A008 (1)

Text

Page 1 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 1

F 2

B M

E S

2010 GINNA NRC SRO RETAKE EXAM NRC:

Revise Part 1 to test applicant knowledge of which valves the CO will use to confirm the Turbine is Tripped (i.e., Stop Valves or Control Valves), rather than what color (i.e., Red or Green) is used to verify the Stop Valves are closed, in order to raise the overall Discrimination Validity of the question. Revise the associated Question Worksheet explanations accordingly.

LICENSEE:

Added in accordance with E-0 to end of overhead Stem Question.

Revised Part 1 Question Statement to read The CO will verify Turbine __ are CLOSED to determine the turbine is tripped.

Changed Part 1 of answer choices to be either Stop Valves or Control Valves.

Changed correct answer to A.

Revised answer explanation and plausibility statements.

Changed Question Source to Modified Bank.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 2 of 56 2

H 3

X X

N E

S NRC Early Review Question NRC:

Potential K/A Mismatch flaw. Questioning whether the weld failure leak location meets the intent of a Pressurizer Vapor Space event. Would the initial inventory loss from a weld failure leak in the Pressurizer Spray Line be entirely in the form of steam, as stated in the correct answer explanation? Does Ginna operate with Spray Flow continuously circulating through the Pressurizer? If so, then how would the Spray Line weld failure be the equivalent of a Pressurizer Vapor Space Leak?

Explanation required. One suggestion for a probable Pressurizer steam space leakage path other than a PORV or code safety, would be a leak in the steam section of reference leg of the Pressurizer Level Transmitter.

Part 1 of Distractors C and D states piping failure on the PZR liquid space sample line. Suggest eliminating the words liquid space to avoid potentially cueing the applicant.

Can Part 2 of the Question be answered solely using RO integrated plant and system knowledge, i.e., knowledge not unique to the SRO position? Question requires the applicant to know from memory information contained in the both the FOLDOUT Page and at Step 19 of E-1, LOSS OF REACTOR OR SECONDARY COOLANT. As such, would Part 2 of the Question be considered RO License Level knowledge at Ginna?

LICENSEE:

Revised Part 1 of Question Statement to be a piping failure of PRZR LT-426 with the correct answer being the reference leg and the distractor being the variable leg.

Revised the given PRZR pressure in the stem to change the correct procedure transition to be to ES-1.1 vice ES-1.2. This will make the correct answer A.

Added RO justification statement The first part of the question is RO knowledge since it can be answered with system knowledge and knowledge of the effect of a Pressurizer vapor space break and the subsequent plant response. The second part is also RO knowledge as it tests the Applicants ability to use knowledge of the Foldout Page criteria in E-1 which is RO knowledge at Ginna.

CHIEF EXAMINER Supplemental Comment for Early Review Question (7/28/2022):

Enhance the question as follows to more closely match the K/A:

1)

Revise the stem to state that a piping failure has

Page 3 of 56 occurred on the Reference Leg condensing pot of Pressurizer Level Transmitter LT-426.

2)

Remove the stem condition that states Pressurizer level is off scale high.

3)

Remove the stem condition that states An automatic reactor trip and Safety Injection (SI) occurred as a result of lowering RCS pressure. A minimally competent applicant should know that these actions have occurred based on the RCS pressure information provided in the last bullet of the stem.

4)

Revise the last bullet of the stem to state RCS pressure is 1655 psig and stable.

5)

Revise the Part 1 Question Statement and associated answer options to test applicant knowledge of whether PZR Level will Rise or Lower based on the failure of the Reference Leg piping.

Appears that there may actually be two correct answers to Part 2. Plausibility explanations for ES-1.2, Post LOCA Cooldown and Depressurization, in Distractors B and D, state that (a) Step 19 of E-1 would direct the Operator to transition to ES-1.2 IF RCS pressure was between 300 psig (350 psig adverse CNMT) and 1650 psig, and (b) the answer is incorrect since RCS pressure is greater than or equal to 1650 psig and slowly rising and therefore a transition to ES-1.1 is required. Step 19 of E-1 only states RCS pressure greater than 300 psig (350 adverse CNMT); i.e., it does not provide a range of 300 -

1650 psig. An applicant could reasonably make the argument that transitioning the ES-1.2 would also be correct given the actual wording of Step 19 and the stem condition that RCS pressure is 1650 psig and slowly rising.

Clarify and explain why the transition to ES-1.1 is the only correct answer to Part 2.

LICENSEE Supplemental Response:

Added Stem Bullet A piping failure occurred on the Reference Leg Condensing Pot for Pressurizer Level Transmitter LT-426.

Deleted Stem Bullet Pressurizer level is off-scale high.

Deleted Stem Bullet An automatic reactor trip and Safety Injection (SI) occurred as a result of lowering RCS pressure.

Revised last Stem Bullet to RCS pressure is 1655 psig and stable.

Revised Part 1 Question Statement to Actual Pressurizer level will be ____.

Changed Part 1 of answer choices to either rising or lowering.

Revised answer explanation and plausibility statements to clarify why transition to ES-1.1 is the only correct answer for Part 2.

Page 4 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level Question determined to be Satisfactory based on Chief Examiner evaluation.

3 H

2 N

E S

NRC:

Enhance the explanations by clarifying what it means when the White light is Lit.

LICENSEE:

Revised plausibility statements to clarify what the Green and White light being LIT means.

Question determined to be Satisfactory based on Chief Examiner evaluation.

4 F

2 N

E S

NRC:

Specify that RCS Pressure in Distractor C is Wide Range Pressure, as indicated in TS Table 3.3.3-1, Post Accident Monitoring Instrumentation.

LICENSEE:

Revised answer choices to read as indicated on Technical Specification Table 3.3.3-1.

Swapped answer choices A and C for psychometrics and associated plausibility statements.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 5 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 5

F 2

X B

E S

2011 TURKEY POINT NRC EXAM NRC:

Reword the Part 2 Question Statement to read:

(2) the reason for the S/G level control band is to preclude Auxiliary Feedwater (AFW) automatic re-initiation AND

________?

The words establish a heat sink which will, can be leading, with the potential to cue the applicant given that a heat sink is required for S/G coupling and establishment of natural circulation.

LICENSEE:

Deleted establish a heat sink which will from Part 2 Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

6 H

3 M

S

Page 6 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 7

H 3

X N

U S

NRC Early Review Question NRC:

Question appears to be written at the SRO License Level.

Questions requires the applicant to know from memory CONDITION, REQUIRED ACTION, and COMPLETION TIME information associated with TS LCO 3.9.4, Residual Heat Removal and Coolant Circulation - Water Level 23 Ft. Per NUREG-1021, ROs are typically expected to know the LCO statements and associated applicability information (i.e., the information above the double line separating the ACTIONS from the LCO and associated applicability statements in the standardized TS). Is this Question considered RO knowledge at GINNA? Question acceptance is contingent upon Licensee confirmation of RO knowledge.

LICENSEE:

Added RO Justification Statement This question tests RO level knowledge of Technical Specifications by process of elimination of one hour Technical Specification actions.

In accordance with ES-4.2, Section E.3.b SRO-only knowledge generally cannot be claimed for questions that can be answered solely based on knowledge of 1-hour action statements and the safety limits since ROs are typically required to know these items. Technical Specification actions one hour are required RO knowledge at Ginna.

Changed Question Cognitive Level to Comprehension or Analysis.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 7 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 8

F H

3 N

E S

NRC:

Suggest changing 80 inches to 75 inches for RCS Loop Level in the third bullet of the stem, to raise the overall Discrimination Validity of the question.

Level of Knowledge (LOK) is classified as Low Cognitive Level (LCL) (i.e., Fundamental) on the Form 4.2-1 Worksheet. Appears that the question should be classified as High Cognitive Level (HCL) based on the guidance in NUREG-1021, Appendix A, regarding the number and types of mental steps necessary to identify the correct answer. Explain the Low Cog assignment.

LICENSEE:

Changed loop level in third stem bullet to 75 inches.

Deleted fourth and sixth stem bullets and added a stem bullet that reads A loss of CCW occurs. Since it is already stated in the Part 1 Question Statement that a loss of CCW has occurred, the original bullets unnecessarily complicated the stem.

Changed Question Cognitive Level to Comprehension or Analysis.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 8 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 9

H 3

X B

E S

NRC:

Revise Distractor D to state:

Reduce load to maintain RCS DNB parameters within limits.

Use of the words while correcting the malfunction, in Distractor D, can potentially imply that there may be some time involved to perform this option, which would not necessarily lend itself to the next required action in accordance with AP-PRZR.1 aspect of the Question Statement, bringing the plausibility of Distractor D into question.

NOTE: The suggested changes enhance the plausibility of Distractor D.

Revise Distractor D plausibility explanation to specifically state why a power reduction to maintain RCS DNB parameters would be incorrect for the question as written.

The last sentence in the plausibility explanation for Distractor A states Incorrect since a reactor trip setpoint has been exceeded. The plausibility of Distractor A is not associated with a reactor trip setpoint being exceeded; but rather RCS pressure being < 200 psig.

LICENSEE:

Revised Distractor D to read Reduce load to maintain RCS Departure from Nucleate Boiling (DNB) parameters within limits.

Revised Distractor D plausibility statement.

Revised Distractor A plausibility statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 9 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 10 F

2 N

E S

NRC:

Revise the Question Statement to read:

The MINIMUM reactor power level above which the ATWS NOTE: The suggested changes avoid the Specific Determiner cited in NUREG-1021, ES-4.2, D.15.d, which states an option that includes another option, also called a subset (e.g., (A) less than 5; (B) less 3).

LICENSEE:

Changed The MINIMUM reactor power level that ATWS to The MINIMUM reactor power level above which the ATWS in the Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

11 H

2 B

E S

2011 HARRIS NRC EXAM NRC:

Change the 2nd bullet of the stem conditions from 15% to 10% to raise the overall Discrimination Validity of the question.

LICENSEE:

Changed 15% to 10% in second stem bullet.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 10 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 12 F

H 3

X M

E S

MODIFIED BANK NRC:

Questioning the plausibility of Distractor B. Why is the required action for a failed PR Channel, to allow for reinstating the P-10 Permissive as described in Distractor B, different from that of Distractor A in the Parent Question from the 2018 GINNA CERT Exam? Why not just use Distractor A exactly as written in the Parent Question? In addition, the action described in Distractor B to place both PR Channel N-42 and N-44 Permissive Bypass switches to BYPASS, appears to be just one of many steps to accomplish the actual DEFEAT of a PR Channel from Normal Configuration, per the applicable Attachment in ER-NIS.3, PR Malfunction, which further brings the plausibility of Distractor B into question.

Separately, why does Distractor B only provide direction for placing the P-10 Permissive Bypass switches for PR Channels N-42 and N-44 in BYPASS?

Level of Knowledge (LOK) is classified as Low Cognitive Level (LCL) (i.e., Fundamental) on the Form 4.2-1 Worksheet. Appears that the question should be classified as High Cognitive Level (HCL) based on the guidance in NUREG-1021, Appendix A, regarding the number and types of mental steps necessary to identify the correct answer. Explain the Low Cog assignment.

LICENSEE:

Revised Distractor B plausibility statement to explain why only N-42 and N-44 would need to be bypassed. A plant modification was completed installing Permissive Bypass switches on the PRNI drawers. This removed the necessity to have I&C technicians install jumpers in order to reinstate the P-10 permissive upon a loss of power to the PRNIs.

Changed Question Cognitive Level to Comprehension or Analysis.

Question determined to be Satisfactory based on Chief Examiner evaluation.

13 H

3 B

S 2017 GINNA NRC EXAM

Page 11 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 14 H

4 X

N E

S NRC:

Unable to confirm, based upon review of the reference material provided, i.e., ER-INST.3, Instrument Bus Power Restoration, that placing the ROD CONTROL BANK SELECTOR SWITCH to Manual is the correct answer for the Loss of A Instrument Bus.

Question appears to require in-depth procedural knowledge of ER-INST.3 and associated Attachments, as well as ER-INST.1, Reactor Protection Bistble Defeat After Instrumentation Loop Failure. Would the RO applicants at GINNA be expected to answer this question using integrated plant and system knowledge (i.e.,

knowledge that is not unique to the SRO position)?

Question acceptance, in part, is contingent upon Licensee confirmation of RO knowledge.

LICENSEE:

Deleted L/428A and P/429A from the respective stem bullets.

ER-INST.3, Section 6.1 is entered whenever an Instrument Bus is de-energized. Step 6.1.3 is an evaluation step and has the Operator evaluate plant conditions and take the appropriate action if necessary.

Revised answer explanation statement.

Added an RO Justification Statement This question is RO knowledge since the Applicant can use integrated system knowledge and knowledge of P-10, INSTRUMENT FAILURE REFERENCE MANUAL, to answer the question. Additionally, Ginnas Facility Representative has determined that this question is RO knowledge.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 12 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 15 H

3 M

E S

MODIFIED BANK (2013 GINNA NRC RO RETAKE EXAM)

NRC:

Reference to Bus 11B in the plausibility explanations for Distractors A and C appears to be incorrect. Suspect it should be 12B instead of 11B.

LICENSEE:

Removed reference to Bus 11B in plausibility statements for Distractors A and C as it is not necessary.

Question determined to be Satisfactory based on Chief Examiner evaluation.

16 H

3 M

S MODIFIED BANK (2017 GINNA NRC RO RETAKE EXAM)

Page 13 of 56 17 F

H 2

X B

U S

NRC:

Part 1 of Distractors A and B do not appear to be plausible. Why would an applicant reasonably believe that an effective strategy for leak identification / isolation during a LOCA would be to close system isolation valves that a) are normally closed, b) would not automatically reposition in response to the event, and c) are not part of the SI Injection flowpath.

Part 2 of Distractors B and D do not appear to be plausible, especially in lieu of the fact that the correct answer is RCS pressure. Why would monitoring for a change in SI flow after isolating the pump discharge be an effective means of identifying the leak or determining isolation? The associated plausibility explanations merely state The second part is plausible since SI flow would change once the leak is isolated. SI flow is going to change by virtue of the fact that the pump discharge has been isolated. How would it be expected to change with respect to leak isolation?

Level of Knowledge (LOK) is classified as Low Cognitive Level (LCL) (i.e., Fundamental) on the Form 4.2-1 Worksheet. Appears that the question should be classified as High Cognitive Level (HCL) based on the guidance in NUREG-1021, Appendix A, regarding the number and types of mental steps necessary to identify the correct answer. Explain the Low Cog assignment.

LICENSEE:

Added a stem bullet for HCO reporting Annunciators that are LIT.

Revised Question Statement to For the given plant conditions, which of the following correctly completes the statements below? (1) The source of the LOCA outside CNMT is ____. (2) In accordance with ECA-1.2, the parameter monitored to determine successful leak isolation is ____.

Changed Part 1 of answer choices to either SI Pump discharge to RCS Cold Legs or RHR Pump discharge to Reactor Vessel Deluge.

Revised Distractor plausibility statements for revised Part 1 answer choices.

Revised Part 2 of plausibility statements for Distractors B and D to better explain plausibility of Safety Injection flow.

Changed Questions Cognitive Level to Comprehension or Analysis.

Question determined to be Satisfactory based on Chief Examiner evaluation.

18 F

1 2

X N

U S

NRC:

LOD=1, regardless of the fact that the question meets the K/A

Page 14 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level Distractors B and C do not appear to be plausible. Why would an applicant reasonably believe, with Bleed and Feed established, that Charging Flow needs to be maximized to a) maintain RCS Pressure / Level (Distractor B), or b) maximize Auxiliary Spray Flow (Distractor C)?

If unable to develop a discriminating RO level question for the assigned K/A without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

LICENSEE:

Changed original Distractor B to To mitigate potential core uncovery.

Changed original Distractor C to To establish conditions for stopping the first SI Pump.

Re-ordered answer choices and associated plausibility statements for psychometrics.

Correct answer changed to C.

Question determined to be Satisfactory based on Chief Examiner evaluation.

19 H

2 N

E S

NRC:

Revise the Part 2 Question Statement to read:

If PRZR level continues to rise, the PRZR will be considered INOPERABLE when level rises above ____.

NOTE: The suggested changes avoid the Specific Determiner cited in NUREG-1021, ES-4.2, D.15.d, which states an option that includes another option, also called a subset (e.g., (A) less than 5; (B) less 3).

LICENSEE:

Deleted L/428A from second stem bullet.

Changed at a level greater than to when level rises above in Part 2 Question Statement.

Changed Part 2 of Distractors A and C from 70% to 85%.

Revised Part 2 of Distractor plausibility statements A and C.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 15 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 20 F

2 X

N E

S NRC Early Review Question NRC:

Recommend revising the Part 2 Question Statement by eliminating the words At least and changing the word Door to Door(s) to raise the overall Discrimination Validity of the Question and avoid potentially cueing the applicant.

LICENSEE:

Changed At least __ Equipment Hatch Access Door to A minimum of __ Equipment Hatch Access Door(s) in Part 2 Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

21 H

3 N

S 22 F

2 B

S 2013 GINNA NRC RO RETAKE EXAM 23 H

2 B

E S

2007 GINNA NRC EXAM NRC:

Revise the RCS Pressure information in the stem to include rate and trend information, similar to Q62 on the referenced 2007 GINNA NRC Exam. Suggest the following:

RCS pressure is 1000 psig and slowly lowering Revise the Question Statement to specify in accordance with ES-1.1 LICENSEE:

Changed stable to slowly lowering in sixth stem bullet for RCS pressure.

Added in accordance with ES-1.1 to end of Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 16 of 56 24 F

2 X

B U

S NRC Early Review Question NRC:

Question, as written, does not sufficiently relate the Residual Heat Removal System (RHRS) to the Containment Flooding event insofar as meeting the intent of the K/A. Part 2 of the Question loosely infers that the RHRS is amongst the population of critical plant components necessary for plant recovery that may be damaged and rendered Inoperable in a Containment Flooding event. While this may be true, there is no aspect of the question that specifically tests applicant knowledge RHRS relatability to Containment Flooding. The applicant can reasonably be expected to answer Part 2 of the question correctly without any knowledge of the relationship of RHRS components to B Containment Sump Levels during a Containment Flooding event. Note that reference document EOP-DIRECTED-TSC-ACTIONS, BACKGROUND INFORMATION FOR GINNA STATION TSC EOP DECISION POINTS, states:

It is reasonable to assume that MOV-851A and MOV-851B valves will remain capable of being closed post-accident from the standpoint of emergency procedures.

In the event they could not be closed, MOV-850A and MOV-850B should be capable of closing. MOV-852A and MOV-852B would not be expected to be capable of closing post-accident, due to flooding, and should therefore be left in their post-accident open position, for emergency procedures that direct transition to RHR cooldown operation.

A possible option would be to incorporate the use of this RHRS specific information, for example, to address the relatability concern. If unable to develop a discriminating RO level question that clearly relates the RHRS to the Containment Flooding event without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

LICENSEE:

K/A W/E15 EK2.07 replaced with randomly reselected K/A W/E15 EK2.03. Question replaced.

CHIEF EXAMINER Supplemental Comment for Early Review Question (8/2/2022):

K/A referenced on the Question Worksheet is incorrect; should be EK2.03, not EK2.07.

Requirement Item 2 of the Question Statement in the proposed replacement Q reads as follows:

SW cooling to BOTH Reactor Compartment Cooling

Page 17 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level Fans can be isolated simultaneously.

Item 2, as written, is not a requirement. How does use of the words can be constitute a requirement? Question makes little sense as written.

Requirement Item 1 of the Question Statement in the proposed replacement Q reads as follows:

Call the Control Room prior to isolating Service Water to any fans.

Calling the Control Room prior to performing an action (in this case, isolating Service Water to any fans) does not constitute a relationship between Containment Flooding and the Service Water System, as required by the K/A.

Note that the K/A associated with the referenced Bank Question used on the 2014 GINNA NRC Exam is different than the K/A for this question. The K/A on the 2014 Exam states:

Ability to determine and interpret the following as they apply to the (Containment Flooding): Adherence to appropriate procedures and operation within the limitations in the facility's license and amendments.

LICENSEE Supplemental Response:

CE randomly reselected K/A EPE W/E15 EK3.05.

Replaced Question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 18 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 25 H

3 B

E S

2014 GINNA NRC EXAM Reference provided:

Detached Reference (Separate Handout); EOP FIG-1.0, Figure Min Subcooling NRC:

Plausibility explanation for Distractors A and C regarding why inventory makeup is adequate and core uncovery is not an issue, is deficient. Explanations are silent regarding the relevance of the RVLIS information provided in the stem.

Distractor D does not appear to be plausible. If the RCS is subcooled, would this not satisfy the pre-requisite requirements in Steps 12.b and 13.b of ES-1.2, Post LOCA Cooldown and Depressurization, for SI reduction and RCP restart when power is restored? Explanation is silent regarding the plausibility of the operational implication, other than to state that the operational implication is correct.

LICENSEE:

Changed second part of Distractors A and C to Inadequate inventory makeup to preclude boron precipitation and long-term reduced heat transfer.

Deleted and RCP restart when power is restored from answer choices B and D.

Revised answer explanation.

Revised Distractor plausibility statements A and C to explain the relevance of RVLIS level provided.

Revised Distractor plausibility statement D to explain the plausibility of the operational implication.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 19 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 26 H

3 X

X B

E S

2008 GINNA NRC EXAM NRC:

Question appears to be written at the SRO license level.

The guidance for maintaining RCS Pressure and Temperature stable for the 1-Hour SOAK period is provided at Step 29.b of FR.P-1.

Would RO applicants realistically be expected to know this specific detail from memory as it applies to ES-1.2 Foldout Page guidance for SI Reinitiation Criteria (to maintain PZR Level above 10%) during concurrent implementation of ES-1.2?

Would RO applicants realistically be expected to know this specific detail from memory as it applies to ES-1.2 Continuous Action Step 6.b direction to Control feed flow to maintain narrow range S/G level between 17 % and 50%, during concurrent implementation of ES-1.2?

Question acceptance contingent upon Licensee confirmation of RO knowledge.

Remove the word Fully from Distractor B to prevent inadvertently cueing the applicant.

Revise Distractor D to read Rapidly feed the S/Gs to return S/G narrow range levels to normal, to prevent inadvertently cueing the applicant.

LICENSEE:

Added RO Justification Statement The question is RO knowledge as it is testing a Major Action Category (Soak if Necessary Prior to Further Restricted Cooldown) and overall mitigative strategy in FR-P.1, not the specific step requirements in ES-1.2. The answer choices are simply whether the given operation will not violate the soak requirements of FR-P.1. Additionally, ROs are required to know Continuous Action Steps.

Deleted Fully from Distractor B.

Revised Distractor D to Start one RCP to establish forced circulation flow.

Revised Distractor D plausibility statement.

Swapped Distractors C and D and associated plausibility statements for psychometrics.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 20 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 27 F

3 X

B E

S NRC:

Distractor B does not appear to be credible. The associated explanation does not address the plausibility of a rapid depressurization of the RCS. The explanation instead describes a condition in which the S/G Temperature is higher than RCS Temperature, resulting in RCS expansion and pressurization.

Revise correct Answer D to state: a low temperature event due to a thermal transient. Indicated change avoids use of the Specific Determiner cited in NUREG-1021, ES-4.2, D.15.f, which states a correct answer that is longer or shorter than the distractors. Answer D is notably longer than the three distractors and contains the information when the RCP is started, which can potentially cue the applicant. Note that the information about starting the RCP is a) provided in both the third bullet of the stem and correct Answer D, b) not mentioned in any of the distractors, and c) not required to correctly answer the question.

LICENSEE:

Revised Distractor B plausibility statement.

Deleted when the RCP is started from Distractor D.

Question determined to be Satisfactory based on Chief Examiner evaluation.

28 H

3 N

S 29 H

2 B

S 2014 BRAIDWOOD NRC EXAM 30 F

2 M

S MODIFIED BANK 31 F

3 N

S

Page 21 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 32 F

2 X

B E

S 2019 GINNA NRC RETAKE EXAM (Previous 2 NRC Exams)

NRC:

Delete the words and counts are stable from the bulleted item in the stem associated with Radiation Monitor R-17, so that it reads the same as RO Question 35 on the 2019 NRC Retake Exam. The inclusion of these words can potentially cue the applicant and are not needed to correctly answer the question.

LICENSEE:

Deleted and counts are stable from the R-17 stem bullet.

Question determined to be Satisfactory based on Chief Examiner evaluation.

33 H

3 N

S 34 H

3 N

S LICENSEE:

Deleted P/429A from third stem bullet for consistency.

35 F

H 3

N B

U S

NRC Early Review Question NRC:

In Part 1 of Distractors A and B, an argument can be made that selection of 400 psig is not incorrect on the basis that 400 psig is 410 psig. The PTLR reference document, Section 2, Low Temperature Overpressure Protection System Setpoints, states The lift setting for the Pressurizer Power Operated Relief Valve (PORVS) is 410 psig (includes instrument uncertainty). Per NUREG-1021, ES-4.2, D.15.d, this is a Specific Determiner that is to be avoided. The Specific Determiner associated with D.15.d states an option that includes another option, also called a subset (e.g., (A) less than 5; (B) less 3).

LICENSEE:

K/A replaced with 010 K4.03. Question replaced.

Deleted P/429A from second stem bullet for consistency.

CHIEF EXAMINER Supplemental Comment for Early Review Question (8/3/2022):

Replacement question evaluated as Satisfactory.

Page 22 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 36 H

4 B

E S

NRC:

Remove the words if any from the Question Statement.

LICENSEE:

Deleted if any from the Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

37 H

3 N

S QUESTION DELETED based on review of post-examination comments.

38 F

2 B

E S

2018 GINNA NRC EXAM NRC:

Revise the answer choices such that each contains a list of three conditions (instead of two), the same as RO Question 58 on the 2018 GINNA NRC Exam. Indicated change raises the overall Discrimination Validity of the question.

LICENSEE:

Unable to add a third set of conditions that would be plausible and not overlap Question #44.

Changed Safety Injection Signal to Automatic Safety Injection in Distractor A and D.

Changed Safety Injection Signal to Manual Safety Injection in Answer Choice B.

Revised Answer Explanation and Plausibility Statements A and D.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 23 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 39 H

F 1

2 X

B U

S NRC:

LOD=1. Distractors B and C are not credible and can be easily eliminated. Why would an applicant believe that a Safety Injection would not actuated if the ESFAS Actuations listed in these Distractors had automatically occurred?

LICENSEE:

Added stem bullets Automatic Safety Injection (SI) fails to actuate and The operating crew manually actuates SI.

Changed the given CNMT pressure from 15 to 16.

Added ONE following Which and underlined automatically in the Question Statement.

Changed answer choices to CNMT Spray, CNMT Isolation, Steamline Isolation, or Feedwater Isolation.

Revised answer explanation and ALL plausibility statements.

After discussion with Chief Examiner, question LOK changed from H to F.

Question determined to be Satisfactory based on Chief Examiner evaluation.

40 H

F 3

B N

U S

2014 GINNA NRC EXAM NRC:

K/A Mismatch flaw.

LICENSEE:

Chief Examiner randomly reselected K/A A1.05; Question replaced.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

QUESTION DELETED based on review of post-examination comments.

41 H

3 B

S 2018 GINNA NRC EXAM

Page 24 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 42 H

2 M

E S

MODIFIED BANK (2019 GINNA NRC RETAKE EXAM)

(Previous 2 NRC Exams)

NRC:

Replace Distractor C, with Distractor C in RO Q43 from the 2019 GINNA NRC Retake Exam, which states:

HCV-110A, BA to BA Blender Flow Control Valve, fails OPEN during normal makeup.

Indicated change raises the overall Discrimination Validity of the question.

LICENSEE:

Changed Distractor C to HCV-110A, BA to BA Blender Flow Control Valve, fails OPEN during normal makeup.

Revised Distractor C plausibility statement.

Swapped Distractors C and D and associated plausibility statements for psychometrics.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 25 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 43 F

1 2

N U

S NRC:

Part 2 of the question, which is the actual tie to the K/A, is LOD=1, making it easy to eliminate Distractors A and C.

Part 1 of the question is borderline LOD=1. If possible, suggest writing a question to test applicant knowledge of the relationship between the number of HDT Pumps running at power > 70% and MFW Pump NPSH, using the guidance in AP-FW.1, Abnormal MFW Pump Flow or NPSH.

If unable to develop a discriminating RO level question for the assigned K/A without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

LICENSEE:

Both parts of the Question, as written, tie to the K/A. The High Pressure Feedwater Heaters are part of the Feedwater System and the first part of the question tests the physical connection to the Heater Drains on high level in the High Pressure Feedwater Heaters. Additionally, Part 1 provides adequate Discrimination Validity as there are five sets of secondary heaters that drain to different locations on normal and high level.

Changed Part 1 Question Statement to High-Pressure Feedwater Heaters (5A, 5B) drain directly to the Heater Drain Tank during ____ level conditions.

Changed Part 1 of answer choices to either normal or high.

Changed Part 2 Question Statement to During plant startup, the second Heater Drain Tank Pump is started prior to exceeding a minimum of ___ reactor power.

Changed Part 2 of answer choices to 50% and 70%.

Changed correct answer to C.

Revised answer explanation and Distractor plausibility statements.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 26 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 44 H

1 2

B U

S NRC:

LOD=1 for any applicant that has spent time in the simulator manipulating the AFW Pump controls and operating the system. Suggest testing applicant knowledge of other AFW Pump operational aspects such as:

o Whether both MDAFW Pumps should be run at the same time with the cross connects open.

o Operation of the MDAFW and TDAFW Pump Air Operated Recirc Valves.

o Auxiliary Feedwater Bypass Switch (Defeat position (Turbine Latched / Turbine Not Latched), Normal position) o SAFW Pump operation / interlocks / controls.

LICENSEE:

Question replaced.

Question determined to be Satisfactory based on Chief Examiner evaluation.

45 H

3 M

S MODIFIED BANK (2017 GINNA NRC EXAM)

LICENSEE:

• Changed Part 2 of Distractor C to energized from Offsite Power.

• Revised Distractors C and D plausibility statements.

46 H

3 N

E S

NRC:

Correct Answer B explanation on the Form 4.2-1 Question Worksheet is deficient. The explanation needs to incorporate the relevant aspects of:

a) Distractor A Part 1 explanation as it applies to the B MFW Pump, and b) Distractor B Part 2 explanation as it applies to the B MDAFW Pump.

LICENSEE:

Revised Answer C explanation and Distractors A and B plausibility statements.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 27 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 47 F

H 2

X B

M U

S 2017 GINNA NRC EXAM NRC:

Question does not meet the intent of the K/A, which states:

Ability to (a) predict the impacts of the following on the DC Electrical Distribution System and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operations: Loss of all AC.

Part 1 of the question is merely asking the applicants to recall the specific design capacity value of the Station 1E Batteries, instead of requiring them to predict / determine the actual impacts of the Loss of All AC on the DC Electrical Distribution System.

Last NRC Exam field on the Form 4.2-1 Question Worksheet lists the 2012 GINNA NRC Exam. Does not appear that this Bank Question was on that Exam.

LICENSEE:

Replaced Question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

48 F

3 B

S LICENSEE:

Added stem bullet to state An automatic reactor trip and Safety Injection occur.

Page 28 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 49 F

H B

E S

2012 GINNA NRC EXAM NRC:

Third bullet in the stem states In accordance with the procedure in effect Specify the actual procedure that is in effect, i.e., ATT-16.1, ATTACHMENT SGTL.

Level of Knowledge (LOK) is classified as Low Cognitive Level (LCL) (i.e., Fundamental) on the Form 4.2-1 Worksheet. Appears that the question should be classified as High Cognitive Level (HCL) based on the guidance in NUREG-1021, Appendix A, regarding the number and types of mental steps necessary to identify the correct answer. Explain the Low Cog assignment.

LICENSEE:

Changed the procedure in effect to ATT-16.1, ATTACHMENT SGTL in third bullet.

Deleted continue to in last stem bullet.

Changed Question Cognitive Level to Comprehension or Analysis.

Question determined to be Satisfactory based on Chief Examiner evaluation.

50 H

4 X

B E

S NRC:

Question appears to require the recall of knowledge that is too specific for the closed reference test mode (i.e., not required to be known from memory). Information required to answer the question appears on Page 61 of 113 of STP-O.17.2. Would this be considered RO knowledge at Ginna? Explanation required.

Question acceptance contingent upon Licensee confirmation of RO knowledge.

LICENSEE:

Added RO Justification Statement The Question is RO knowledge since it tests system knowledge of the Auxiliary Building Ventilation System. This information is also listed in Alarm Response Procedures. Additionally, Ginnas Facility Representative has determined that this question is RO knowledge and NOT minutia.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 29 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 51 F

3 N

E S

NRC:

The word following is used twice in the Part 1 Question Statement. Replace the first following with the word after to improve the readability.

LICENSEE:

Changed the first following to after in the Part 1 Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

52 H

3 N

E S

NRC:

The Part 2 Question Statement reads in accordance with the appropriate EOP. The explanation for correct Answer D identifies E-1, Loss of Reactor or Secondary Coolant, as the appropriate EOP procedure. Revise the Part 2 Question Statement to read in accordance with E-1, Loss of Reactor or Secondary Coolant.

LICENSEE:

Changed the appropriate EOP to E-1, LOSS OF REACTOR OR SECONDARY COOLANT in the Part 2 Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

53 H

3 B

E S

NRC:

Revise the second bullet of the stem to read:

A and B Instrument Air Compressors are in standby.

Replace the word secured with standby in the Part 2 answers of Distractors A and C.

LICENSEE:

Deleted secured in second bullet.

Changed secured to standby in Part 2 of Distractors A and C.

Question determined to be Satisfactory based on Chief Examiner evaluation.

54 H

3 B

S 2013 TURKEY POINT NRC EXAM

Page 30 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 55 F

2 B

S 56 H

3 X

B N

U S

NRC:

Question requires backward or reverse logic and is therefore Unsatisfactory. See ES-4.2, Section C.8 and Appendix B, Section G. See also Form 2.3-5 Instructions for Written Examination Review Worksheet, Item 4, last bullet (Logic).

Suggest, if applicable at GINNA, staying with the same concept by modifying the stem to provide the Annunciators that would be indicative of an RTD failure that results in a failure LOW of the Average-Tave input to the PRZR Level Control System, the consequence of which is Programmed PRZR Level failing to 20%. Answer choices would include the PRZR BU Heaters energizing (correct answer) and three plausible Distractors.

If unable to develop a discriminating RO level question for the assigned K/A without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

LICENSEE:

Replaced Question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

57 H

2 B

S

Page 31 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 58 H

3 M

B E

S 2012 SALEM NRC EXAM NRC:

Part 1 Question Statement tells the applicant that one of the IR Channels is under-compensated. Revise the question to have the applicant determine whether the IR Channel is under-compensated or over compensated, as in the referenced parent question from the 2012 SALEM NRC Exam. NOTE: Indicated change would most likely make the Question Source BANK vs MODIFIED.

Part 2 Answers for Distractors C and D state:

verify Source Range channels automatically energize Appears that this step is actually performed at Step 13.d of ES-0.1, Reactor Trip Response, prior to RNO Step 13.d, to Manually energize Source Range Detectors, which is identified as the correct answer response. Is there a more than one correct answer?

LICENSEE:

Changed - to negative in fourth and fifth bullets of the stem and answer A explanation.

Changed Part 1 Question Statement to IR Channel __

with the possible answer choices of N-35 is undercompensated or N-36 is overcompensated.

Changed Part 2 Question Statement to The Source Range channels __ with the possible answer choices must be manually energized or will automatically energize.

Revised answer explanation and plausibility statements.

Changed Question Source to Bank.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 32 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 59 F

2 B

M E

S 2017 GINNA NRC EXAM NRC:

Part 1 of Distractors A and B result in a Question Statement that reads:

NaOH enters the Containment Spray System via pumps at the suction of the Containment Spray Pumps.

This version of the Question Statement reads awkwardly (i.e., via pumps at the suction of the pumps) and diminishes the credibility of Part 1 for Distractors A and B, making these choices easy to eliminate and Part 1 of the question Borderline LOD=1. Revise the Part 1 answer for Distractors A and B to raise the credibility of Part 1.

LICENSEE:

Changed Part 1 Question Statement to NaOH enters the Containment Spray System at the ___ of the Containment Spray Pumps.

Changed Part 1 of answer choices to either suction or discharge.

Deleted inside Containment from Part 2 Question Statement.

Changed Part 2 answer choices to either Iodine levels inside CNMT or Ph levels in CNMT Sump B.

Changed correct answer to A.

Revised answer explanation and plausibility statements.

Changed Question Source to Modified Bank.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 33 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 60 F

2 B

E S

2013 GINNA NRC RETAKE EXAM NRC:

Revise correct Answer D to state: Overload, similar to the referenced Bank Question form the 2013 GINNA NRC Retake Exam. Removing the description Load Cell avoids any potential for cueing the applicant and is not necessary information to correctly answer the question.

Correct Answer D is the only choice preceded by a descriptive qualifier.

LICENSEE:

Removed Load Cell from answer choice D.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 34 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 61 H

2 X

N B

U S

NRC:

Part 2 of the question is a K/A Mismatch Flaw. ES-4.2, Section D.8, contains the following guidance:

Note: For multipart questions, both parts of the question must meet a topic/concept that is related to the associated K/A.

The applicant is not being tested on the failed valve position of the operator in Part 2 because the Steam Dump Valve is responding properly to Failed High STM HEADER PRESSURE XMTR PT-484. For Part 2, suggest testing applicant knowledge of failed valve position for the SDCS solenoid valve(s) that re-align to supply / block air to the Turbine Bypass Valves (TBVs) upon de-energization (i.e., loss of DC power to the solenoid).

As an example, the solenoid valve that is aligned to direct control air from the positioner to the TBV actuator if there is a demand for modulating the TBVs, is de-energized to accomplish this function. However, this same solenoid is energized when Steam Dump Demand exceeds a certain value. When energized, the solenoid aligns the valve ports so that the air supply bypasses the modulating positioner and provides full air pressure to the actuator to quickly trip open the TBVs. If there is a loss of DC power to the solenoid during this condition, the solenoid would reposition, preventing full air pressure from being supplied.

LICENSEE:

• K/A and Question replaced.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 35 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 62 F

3 N

E S

NRC:

Truncate the Part 1 Question Statement after the words SETTER value. The last part of the statement is too much information, is unnecessary, and can potentially cue the applicant. A minimally competent applicant that has trained for months in the simulator should know that the reason for maintaining the VALVE POSITION LIMIT a specified percentage above the setter, is to minimize the load increase in the event of a governor malfunction. No need to educate the applicants during the exam.

LICENSEE:

Removed to minimize the load increase in the event of a governor malfunction from Part 1 Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

63 F

2 B

S 64 F

2 B

S

Page 36 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 65 F

2 B

U S

2019 GINNA NRC RETAKE EXAM NRC:

Question does not meet the criteria for Tier 3. ES-4.2, Paragraph B.2.c states:

Ensure that Tier 3 questions focus on plant-wide generic K/As and not system-specific knowledge or knowledge of specific emergency / abnormal plant evolutions.

Question, as written, requires specific knowledge of various Plant Systems, EOPS, and setpoint values.

If unable to develop a discriminating RO level Tier 3 question for the assigned K/A without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

LICENSEE:

Added Tier 3 Justification Statement. The Question meets the Tier 3 requirements as it tests the Applicants knowledge of MANUAL ACTIONS in LIEU of AUTOMATIC ACTIONS from an Operations Department Guideline (administrative procedure), which are based on conservative decision-making actions for manually tripping the reactor. Furthermore, OP-AA-103-102-1001, STRATEGIES FOR SUCCESSFUL TRANSIENT MITIGATION, Step 4.1.2.2.A states Operators are expected to manually initiate a reactor trip or safety injection prior to a given parameter reaching the setpoint for automatic initiation of that design function, based on the observed parameter trend indicating an imminent automatic initiation. The question tests conservative actions based on guidance contained in Operations administrative guidelines/procedures and does NOT test Reactor Trip System setpoint values.

Chief Examiner discussed Question with the Program Office and determined that the Question is satisfactory as written.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 37 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 66 F

2 B

E S

2019 GINNA NRC RETAKE EXAM (Previous 2 NRC Exams)

NRC:

Replace SFAM with Operations Manager / Superintendent, or equivalent position at GINNA, to raise the Discrimination Validity of the question.

LICENSEE:

Replaced SFAM with Shift Operations Superintendent (SOS).

Revised Distractors C and D plausibility statements to include the SOS.

67 H

3 B

S 2018 GINNA NRC EXAM 68 F

2 B

S 69 F

3 B

S 2015 MCGUIRE NRC EXAM 70 H

3 B

S

Page 38 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 71 F

3 B

E S

NRC Early Review Question Reference provided:

Embedded Reference; THE NUCLEAR DESIGN AND CORE MANAGEMENT OF THE R.E. GINNA NUCLEAR POWER PLANT CYCLE 43, Figure 3-1, Critical Boron Concentration vs.

Cycle Burnup Note: Embedded pictures, sketches, drawings, and graphs, with exception, are considered to be references.

NRC:

Editorial: Delete the words point or in the second line of the Question Statement, given that each answer choice is actually a set of points.

Editorial: Include the title of Figure 3-1 in the Proposed references to be provided to applicants during examination Field of Worksheet Form 4.2-1.

LICENSEE:

Deleted point or from the Question Statement.

Added Figure 3-1 title in the Proposed references to be provided to applicants during examination Field of Worksheet Form 4.2-1.

Question determined to be Satisfactory based on Chief Examiner evaluation.

72 H

2 B

E S

NRC:

Delete the word stable in the third bullet of the stem.

Plausibility of Distractor A would be questionable without this change; i.e., how could SUR continuously rise if the HCO established a stable SUR?

LICENSEE:

Deleted the word stable in the third bullet of the stem.

Changed + to positive in third bullet of the stem and answer choices C and D.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 39 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 73 H

3 N

E S

NRC Early Review Question Reference provided:

Steam Tables NRC:

Change Distractor B to 350°F for the applicant that misapplies the Mollier Diagram by throttling to the Saturation Curve. Re-sequence the answers from lowest to highest, resulting in the correct answer being B and Distractor A the 350°F option. Suggested change raises the overall Discrimination Validity of the question given that an applicants use of the Main Steam Safety Valve setpoint of 1085 psig (1100 psia) is unlikely and therefore less credible.

LICENSEE:

Changed Distractor B to 350°F and revised the distractor explanation.

Swapped answer choices A and B for psychometrics.

Changed correct answer to B.

Question determined to be Satisfactory based on Chief Examiner evaluation.

74 F

3 B

S 75 H

3 B

S Reference provided:

Steam Tables

Page 40 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 76 H

3 B

E S

2018 GINNA NRC EXAM Reference provided:

Detached Reference (Separate Handout); EOP FIG-1.0, Figure Min Subcooling NRC:

Revise the stem bullet The STA observes: to state Following the transition to E-1, the STA observes:

Enhancement recommended to avoid any potential confusion regarding the information associated with Pressurizer Level in the stem. Similar approach was taken on the referenced Bank question from the 2018 GINNA NRC Exam; i.e., After the crew transitions from E-0 to E-1, STA observes:

LICENSEE:

Revised the stem bullet The STA observes: to Following the transition to E-1, the STA observes:

Question determined to be Satisfactory based on Chief Examiner evaluation.

77 H

2 B

S 2017 GINNA NRC EXAM

Page 41 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 78 H

3 N

E S

NRC:

Raise the Discrimination Validity of Part 2 by revising the question to test applicant knowledge of when a Faulted S/G would be isolated IAW FR-S.1 (i.e., Rx is NOT shutdown, making Choice B the correct answer). This approach more closely aligns Part 2 with the intent of the K/A, as it relates to the ATWS event. In the question as written, the ATWS no longer exists per the stem conditions.

Form 4.2-1 Worksheet explanation for correct Answer A incorrectly states that the reactor is locally tripped at Step 6 of FR-S.1. This guidance is actually provided in Step 7.a RNO.

LICENSEE:

Added stem bullet The Control Room is establishing Emergency Boration.

Deleted bullets under Subsequently: and added new bullet While checking if the reactor is subcritical, the HCO reports Intermediate Range Startup Rate is 0.1 DPM and stable.

Revised wording of question Part 2 to The procedure the US will perform to isolate A S/G is ____.

Changed correct answer to B.

Revised answer explanation and Distractor plausibility statements.

Question determined to be Satisfactory based on Chief Examiner evaluation.

79 H

3 M

S MODIFIED BANK

Page 42 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 80 H

2 X

B U

S NRC:

Question appears to be written at the RO License Level.

Would an RO at Ginna not be expected to know that 59.5 Hz is considerably below the acceptable value for Grid Frequency (59.9 Hz), requiring the start of both EDGs?

Separately, why would any RO applicant reasonably believe that the correct action for addressing a severely Degraded Voltage condition on the Safeguards Buses would be to prioritize a Normal Plant Shutdown over transferring these Buses to the EDGs in a timely manner?

Plausibility explanation for Distractor C explains why Part 2 is plausible. Part 2 of Distractor C is actually correct.

Revise the explanation accordingly.

LICENSEE:

Changed stem bullet for Grid frequency from 59.5 Hz and stable to 59.9 Hz and slowly lowering.

Changed Part 2 of Distractors B and D from O-2.1, NORMAL SHUTDOWN TO HOT SHUTDOWN to AP-TURB.5, RAPID LOAD REDUCTION.

Revised Part 2 of Distractor C plausibility statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 43 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 81 H

F 2

N B

E S

Reference provided:

Detached Reference (Separate Handout); EOP FIG-6.0, Figure Min RCS Injection NRC:

Correct Answer and Distractor Plausibility explanations are silent regarding why the number of SI Pumps specified in Part 2 (i.e., 1 or 2), other than to state that SI and RHR Pumps should be stopped as necessary to establish and maintain minimum required injection flow.

Part 1 of the Question is RO level knowledge (Reference provided). Suggest revising the Part 2 Question Statement to test applicant knowledge of the Number of ECCS Pumps running (i.e., Charging & SI), given that minimum Charging Flow must also be established for RCP Seal Injection per RNO Step 18.b of ECA-1.1. Suggested change intended to raise the overall Discrimination Validity of this SRO Question.

LICENSEE:

Recommend K/A replacement. Replaced with EPE W/E11 EA2.10.

Replaced question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 44 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 82 F

2 X

B U

S NRC:

RO level knowledge. A minimally competent RO would know, given the conditions specified in the stem, that the first action to insert negative reactivity would be to start both BA Pumps and Open MOV-350.

LICENSEE:

Changed - to negative and capitalized dpm in fourth stem bullet.

Changed last stem bullet to Two Control Rods are fully withdrawn.

Changed the applicable procedure to ES-0.1, REACTOR TRIP RESPONSE in Question Statement.

Changed correct answer to D.

Revised Distractor A plausibility statement and answer D explanation.

Added SRO Justification Statement The Question is SRO level since according to ES-4.2, Figure 4.2-3 it can NOT be answered solely from systems knowledge, it can NOT be answered solely by knowing immediate operator actions, it is NOT AOP or EOP entry conditions, it can NOT be answered solely by knowing the overall mitigative strategy, and it DOES require an assessment of plant conditions and then prescribing a section of a procedure with which to proceed.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 45 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 83 H

X X

B U

S Reference provided:

Detached Reference (Separate Handout);

O-2.2, Attachment 3, Full Range RCS Pressure Temperature Cooldown Curve NRC:

Part 1 is RO level Knowledge. Per NUREG-1021, ES-4.2, Section E.3.b, Facility Operating Limitations in the Technical Specifications and Their Bases [10 CFR 55.43(b)(2)], ROs are typically expected to know the LCO statements and applicability information (i.e., the information above the double line separating the ACTIONS from the LCO and associated applicability statements in the standardized TS).

Part 2 of the question provides cues to the correct answer and is leading. The Part 2 Question Statement specifies LCO 3.4.17, STEAM GENERATOR (SG) TUBE INTEGRITY. The correct Part 2 answer states propagate to a Steam Generator Tube Rupture (SGTR). The Part 2 Distractor, which has nothing to do with SG Tube integrity, states exceed the limits for secondary coolant activity.

LICENSEE:

Recommend K/A replacement. Replaced with APE037 AA2.17.

Replaced question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 46 of 56 84 H

2 4

N U

S NRC Early Review Question NRC:

Multiple issues:

Each of the answer choices is an awkwardly worded run-on sentence with items that are strung together using multiple ands in combination with the word then, resulting in answers where intent is not precisely expressed or clearly conveyed. For example, Choice A utilizes the conjunction and 3 times, Choice B 4 times, and Choices C / D each use and 5 times.

Each of the answer choices talks about the simultaneously referencing of procedures, rather than the parallel implementation of procedures.

The K/A explicitly states Knowledge of emergency and abnormal operating procedures implementation hierarchy and coordination with other support procedures or guidelines, such as operating procedures, abnormal operating procedures, or severe accident management guidelines, as applicable to the APE for PLANT FIRE ON SITE. The K/A is intended for development of a question that specifically tests applicant knowledge of procedure implementation hierarchy and coordination, not merely the referencing of procedures, which can arguably be done at any time that is convenient.

Note that EMERGENCY AND ABNORMAL OPERATING PROCEDURES USERS GUIDE (A-503.1), Section 5.2 (Instructions for Procedure Use),

Subsection A (Procedure Hierarchy), provided as an accompanying reference to the question, only discusses the implementation of procedures. There is no mention of referring to procedures.

Separately, would correct Answer A actually be correct, given that AP-CCW.2 guidance allows operation to continue for up to 2 minutes if CCW Flow to an RCP is interrupted for this length of time?

If unable to develop a discriminating SRO level question for the assigned K/A without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

LICENSEE:

K/A replaced with APE067 G2.4.6. Question replaced.

Question determined to be Satisfactory based on Chief Examiner evaluation.

85 H

2 N

E S

NRC:

Borderline LOD=1 as written. In Part 2, why would an SRO applicant reasonably believe if CET indications were

Page 47 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level unavailable, that T-Cold would be used instead of T-Hot to verify Natural Circulation? Distractor B and D plausibility explanations are silent as to why T-Cold would be credible, other than to say that T-Hot is used.

Change Part 2 answer options to Loop A T-Hot and Loop B T-Hot to raise the Discrimination Validity of this SRO Question. Loop A T-Hot is the correct answer per ER-FIRE.1, Attachment 5, Natural Circulation.

LICENSEE:

Changed Part 2 of answer choices A and C to Loop A THOT.

Changed Part 2 of answer choices B and D to Loop B THOT.

Revised Part 2 of Distractors B and D plausibility statements.

Question determined to be Satisfactory based on Chief Examiner evaluation.

86 H

2 B

E S

NRC:

RCP Seal Leakoff value of 9.2 GPM appears high, and as such, diminishes the overall Discrimination Validity of the question, given: (a) the 8.0 GPM threshold value in Step 2 of AP-RCP.1, and (b) the longer time considerations in Distractors C and D. Change 9.2 GPM to 8.5 GPM.

Indicated change does not impact the correct answer.

LICENSEE:

Changed RCP Seal Leakoff value from 9.2 to 8.5.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 48 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 87 F

H 3

N E

S NRC Early Review Question NRC:

Level of Knowledge (LOK) is classified as Low Cognitive Level (LCL) (i.e., Fundamental) on the Form 4.2-1 Worksheet. Appears that the question should be classified as High Cognitive Level (HCL) based on the guidance in NUREG-1021, Appendix A, regarding the number and types of mental steps necessary to identify the correct answer. Explain the Low Cog assignment.

LICENSEE:

Changed Question Cognitive Level to Comprehension or Analysis.

CHIEF EXAMINER Supplemental Comment for Early Review Question (8/16/2022):

Verified the change from LCL to HCL on the Form 4.2-1 Worksheet.

88 H

2 X

M E

S MODIFIED BANK (2010 GINNA NRC EXAM)

NRC:

Question is Borderline LOD=1 for an SRO. Part 1 of the Question is RO level knowledge. In addition, Stem, as written, provides too much information regarding the leak location and is effectively cueing the applicant that the leak is in the Aux Building. Revise the SUBSEQUENTLY section of the stem as follows to raise the overall Discrimination Validity:

o Truncate the first bullet after P113 by deleting the words in the Auxiliary Building.

o Delete the 3rd bullet that states Annunciator L-9, AUX BLDG SUMP HI LEVEL, is LIT.

Sufficient information exists without this additional detail for the applicants to successfully answer this question.

LICENSEE:

Deleted in the Auxiliary Building from the first Subsequent stem bullet.

Deleted the third Subsequent stem bullet.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 49 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 89 H

2 B

S Reference provided:

Detached References (Separate Handout);

Tech Spec LCO 3.8.4, DC Sources - MODES 1, 2, 3, and 4 Tech Spec LCO 3.8.6, Battery Cell Parameters

Page 50 of 56 90 F

H 2

X B

U S

NRC:

UNSAT. Part 1 of the Question is deficient in that one can reasonably argue there are two correct answers depending upon how the Question Statement is interpreted, given that ATT-27.0 directs the performance of ATT-3.0, thereby ensuring the performance of all required manipulations to achieve Containment Isolation.

Distractors B and D do not appear to be credible. For Distractor B, if ATT-27.0 performed all of the required manipulations to ensure Containment Isolation (Part 1),

then where is the plausibility in the next action being the performance of ATT-3.0 to ensure the isolations locally?

For Distractor D, if ATT-27.0 did not perform all of the required manipulations to ensure Containment Isolation (Part 1), then where is the plausibility in the next action being the performance of an ATT-27.0 subsequent action that does not address the failure of Containment to fully isolate?

Level of Knowledge (LOK) is classified as Low Cognitive Level (LCL) (i.e., Fundamental) on the Form 4.2-1 Worksheet. Appears that the question should be classified as High Cognitive Level (HCL) based on the guidance in NUREG-1021, Appendix A, regarding the number and types of mental steps necessary to identify the correct answer. Explain the Low Cog assignment.

LICENSEE:

Part 1 of the Question has only ONE correct answer. It is asking the Applicant whether ATT-27.0 contains the steps to perform ALL the required manipulations of the individual components. Though ATT-27.0 does contain a step to REFER to ATT-3.0, this step does not contain any manipulations, those are contained within ATT-3.0.

Similarly, one could not reasonably argue that E-0 isolates a faulted steam generator solely because it transitions the operator to E-2.

Changed notes to observes in 5th stem bullet.

Added for the given plant conditions to the end of the overhead Question Statement.

Changed required manipulations to required component manipulations in Part 1 Question Statement.

Changed Part 2 of Distractor B to check CCW system status.

Changed Part 1 of Distractor C to will NOT.

Changed Part 2 of Distractor C to place Letdown Orifice Valve switches to CLOSE.

Changed Part 1 of Distractor D to will.

Changed Part 2 of Distractor D to manually CLOSE affected valves from the MCB.

Revised plausibility statements for Distractors B, C and D.

Page 51 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level Changed Question Cognitive Level to Comprehension or Analysis.

Question determined to be Satisfactory based on Chief Examiner evaluation.

91 F

2 X

N U

S NRC:

Appears to be a K/A Mismatch flaw. Question is written to test SRO applicant knowledge of TS LCO 3.1.8 Physics Tests Exception requirements rather than precautions, limitations, notes or cautions associated with the Nonnuclear Instrumentation System. What is the connection to the Nonnuclear Instrumentation System?

As an informational note, Part 1 of the Question is RO level knowledge, which is acceptable given that Part 2 is written to test SRO level knowledge (i.e., TS Bases information) and is therefore not an issue in this question.

Per NUREG-1021, ES-4.2, Section E.3.b, Facility Operating Limitations in the Technical Specifications and Their Bases [10 CFR 55.43(b)(2)], ROs are typically expected to know the LCO statements and applicability information (i.e., the information above the double line separating the ACTIONS from the LCO and associated applicability statements in the standardized TS). The 530°F and 540°F values are above the double line TS information in LCOs 3.1.8 and 3.4.2 respectively.

LICENSEE:

Replaced Question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

92 H

2 N

E S

NRC:

Borderline SRO license level as written. Revise the question to test SRO applicant knowledge of conditions warranting the start of all 3 Auxiliary Feedwater Pumps to raise the SRO Discrimination Validity.

LICENSEE:

Replaced Question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 52 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 93 F

H 3

X N

B U

S NRC Early Review Question NRC:

Parts 1 and 2 of the question can both be answered using RO system level knowledge. If unable to develop a discriminating SRO level question for the assigned K/A without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

Separately, In Part 2 of Distractors A and C, an argument can be made that selection of 195 psig is not incorrect on the basis that 220 psig is > 195 psig. Per NUREG-1021, ES-4.2, D.15.d, this is a Specific Determiner that is to be avoided. The Specific Determiner associated with D.15.d states an option that includes another option, also called a subset (e.g., (A) less than 5; (B) less 3).

LICENSEE:

CE randomly reselected 033 A2.02. Question replaced.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 53 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 94 F

2 B

E S

2018 SALEM NRC EXAM NRC:

Borderline LOD=1. Enhance the question as follows to raise the SRO Discrimination Validity level:

(a) Revise Part 1 of the Question to test the MINIMUM hours aspect of the Quarterly shift requirement to maintain an SRO License, similar to the 2018 Salem Parent Question.

(b) Revise Part 2 of the Question to test applicant knowledge of performance of non-position related administrative tasks inside the Control Room.

In addition, change the Question Source to MODIFIED on the Form 4.2-1 Worksheet and include the 2018 Salem Parent Question with the Written Exam.

LICENSEE:

Changed Part 1 Question Statement to The quarterly shift requirement to maintain an SRO license is a MINIMUM of

__ hours to be completed with a combination of complete 8-and 12-hour shifts.

Changed Part 1 of answer choices to either 40 or 56.

Changed Part 2 Question Statement to During a credited shift to meet the quarterly shift requirement, the SRO __

perform non-position related administrative tasks inside the Control Room.

Changed Part 2 of answer choices to either may or may not.

Changed correct answer to C.

Revised answer explanation and Distractor plausibility statements.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 54 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 95 F

3 B

N E

S 2018 GINNA NRC EXAM NRC:

Question does not meet Tier 3 criteria; requires system specific knowledge. Coded as an Enhancement per ES-5.1 guidance due to previous use on the 2018 GINNA Exam.

LICENSEE:

Replaced Question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

96 F

2 B

E S

2015 TURKEY POINT NRC EXAM 2019 COMANCHE PEAK NRC EXAM NRC:

RO Question 68 on the 2015 TURKEY POINT Exam.

Coded as an Enhancement per ES-5.1 guidance due to previous use on the 2015 TURKEY POINT Exam.

LICENSEE:

Replaced question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

97 F

2 B

E S

NRC:

In Distractor B, replace site Senior Leadership with Station Duty Manager, as referenced in the associated plausibility explanation, to enhance the credibility of this answer choice.

LICENSEE:

Replaced site Senior Leadership with Station Duty Manager in Distractor B.

Revised Distractor B plausibility statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 55 of 56 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 98 H

3 B

E S

NRC:

Distractor B does not appear to be plausible and can therefore be easily eliminated. Why would an applicant reasonably believe that it would be acceptable for a GDT (i.e., GDT B) that had not been sampled to be released?

LICENSEE:

Changed Distractor B to The B GDT was sampled this morning at 0500. The current time is 1800 with a release permit for B GDT on the HCO desk.

Revised Distractor B plausibility statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

99 F

3 B

E S

NRC:

Distractor C does not meet the intent of a Tier 3 Distractor because it requires knowledge of a specific Emergency Plant Evolution.

LICENSEE:

Changed Distractor C to ECA-0.0 and the referenced FLEX Support Guidelines (FSG)s contain all the necessary actions from the FR Procedures to cope with a station blackout.

Revised Distractor C plausibility statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

100 F

2 B

N E

S NRC:

Distractor B is not credible. Why would any SRO applicant reasonably believe that the STA position, which does not have a Command and Control function, and may not be staffed by a Licensed SRO, would have the authority to escalate a declared emergency action level from an SAE to a GE?

LICENSEE:

Replaced Question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Page 56 of 56 Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1) Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2) Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3) Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4) Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5) Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6) Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7) Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8) Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9) Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.