ML23334A235

NRC Comments on Draft Licensee-Developed Written Exam

ML23334A235
Person / Time
Site:	Millstone
Issue date:	11/30/2023
From:	Joseph Demarshall NRC/RGN-I/DORS/OB
To:	Dominion Energy Nuclear Connecticut
References
EPID L-2023-OLL-0010
Download: ML23334A235 (1)

Text

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 1

H 2

X B

N E

S 2021 MILLSTONE 3 NRC EXAM (Q1 removed from the CERT Exam)

NRC:

Question, as written, is asking the applicant to determine which annunciators are NOT applicable to a Reactor Trip.

The following issues were identified with the Proposed Question:

o Excessive use of Double Negatives between the Question Statement and the answer choices.

Recommend eliminating the words should not be lit from all four answer choices. Question is confusing as written.

o Excessive use of negatives in Distractor A. Distractor presently contains three negatives (i.e., 2-NOTs and 1-No).

o Distractor B requires the applicant to make an assumption regarding the initial level of the DWST.

The 19.6 minute calculation time to receive the DWST LEVEL LO alarm in the plausibility explanation does not support why the Distractor is incorrect, given the 10-minutes after trip information provided in the stem.

o Distractor C does not appear to be plausible. How does MFP Recirc Valves opening on a Feedwater Isolation affect Feed Water Heater Levels?

LICENSEE:

Replaced question with new question that asks which annunciator IS expected on a reactor trip. Also ensured no negative statements are included in the answer choices, and removed DWST Low Level from the choices.

Supplemental LICENSEE response:

Changed the question statement to ask which annunciator would be lit (Based on NRC Branch Chief review comment).

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

2 F

2 B

S 2007 MILLSTONE 3 NRC EXAM

3 H

3 N

E S

NRC:

Correct Answer C and Distractor B are equivalent answer options for the conditions specified. The action in Correct Answer C is to Remove the D RCP from service per AOP 3554. This action is accomplished in accordance with Step 3 of the AOP, which states STOP affected RCP. As such, Remove and Stop are synonymous in terms of the action prescribed in Correct Answer C. The action in Distractor B is to Stop the D RCP and continue the shutdown per OP 3206. Note that the stem specifically states that a plant shutdown is in progress per OP 3206.

The fact that the plant shutdown in progress information is included In Distractor B and omitted from Correct Answer C, does not make the two answer choices (B & C) unique/different. Question is still in need of a 3rd plausible Distractor.

Explanations for Answers C and D indicate that the value for P-10 is 11%. Is this correct for MS3? Should it be 10%?

LICENSEE:

Modified B and C to make it clear that in one case the crew remains in OP 3206 without having to transition to AOP 3554, and in the other case, the crew is required to transition to AOP 3554 to stop the RCP. Rearranged all four distractors to fit the grammar of new distractors B and C, and organize them in a logical arrangement, making old distractor B the new distractor D.

Confirmed from Tech Spec Table 2.2-1 that P-10 setpoint is 11% actuation for power increasing (reset setpoint is 9%

on decreasing power).

CHIEF EXAMINER Supplemental Comment (8/9/2023):

Revise Distractor A to state Continue the shutdown per OP 3206 instead of Remain in OP 3206.

Distractor D does not appear to be plausible from the standpoint that the RCP is being tripped without the use of procedural guidance; i.e., without going to either E-0 or AOP 3554. Understand that the Distractor is written this way to distinguish it from Correct Answer C.

In addition, why would the applicants be told without going to E-0, after telling them continue the shutdown per OP 3206?

LICENSEE Supplemental Response:

Revised Distractor A to read, Continue the shutdown per OP 3206 Changed distractor D to read, Stop the D RCP per OP 3353.MB4B, 2-7, without going to AOP 3554.

Question determined to be Satisfactory based on Chief Examiner evaluation.

4 H

3 X

N E

S NRC:

Remove the T+5 stem line item that states The crew stabilizes the plant per AOP 3555, to raise the overall Discrimination Validity of the question. This information is not required to correctly answer the question and is leading. The applicants can easily eliminate Part 2 of Distractors C and D, i.e., Pressurizer level is decreasing, if they are told that the crew has stabilized the plant per AOP 3555. The information provided in T+6 and T+7 is sufficient for the applicants to determine that Pressurizer level has stabilized.

LICENSEE:

Modified the T+5 statement rather than removed it, since removing it makes it unclear whether actions have been taken per AOP 3555, and applicants are to assume no procedure actions are taken unless they are told the crew has taken actions. By simply removing the T+5 item, distractor C could potentially be correct. Modified the T+5 statement to say The crew takes initial actions per AOP 3555, and is not required to trip the plant. This removes the leading statement saying the crew has stabilized the plant, and now it is up to the applicant to understand what actions have been taken to avoid having to trip the reactor in AOP 3555.

Enhanced the justification section to support the changes.

Per discussion with NRC on 7/28/23, removed VCT level decreasing from Answers A and B, since it was deemed easy for an applicant to discern that VCT level would be decreasing with a RCS leak in progress. Changed part 2 of answers A and B to ask Pzr level, whether it is stable or decreasing.

CHIEF EXAMINER Supplemental Comment (8/9/2023):

Revise Item T+5 to eliminate the information that the reactor is not required to be tripped. If the applicants are provided this information, they are essentially being told that Pressurizer Level is Stable, making it easy to eliminate Distractors C and D. This information is unnecessary, given that the applicants should be able to determine, based on Item T+10, what actions were taken to stabilize the plant per AOP 3555.

In addition, suggest raising the Leak Rate to 20 GPM to raise the overall Discrimination Validity.

LICENSEE Supplemental Response:

Removed the crew is not required to trip the reactor from stem item T+5, since this can be determined from the remaining stem items.

Per discussion with Chief Examiner on 8/17/23, added the step number for stem item T+10 so applicants can determine where the crew is in the procedure.

Changed stem item T+7 to make the leak size 20 gpm.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 5

F 2

N E

S NRC:

Replace the word align with establish in the Part 1 section of the Answer Statement, such that it reads as follows:

The RO will establish the bleed path by _____

Recommended change enhances the plausibility of Distractors B and D given that allowing RCS pressure to increase to the PORV COPPS setpoint is not the same as physically manipulating a component for the purpose of aligning the bleed path.

LICENSEE:

Replaced the word align with the word establish in part 1 of the answer statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

6 F

2 B

S 7

H 3

B E

S NRC Early Review Question 2017 MILLSTONE 3 NRC EXAM NRC:

Replace the word from with the word to in Item 5 of the stem so it is clear that Item 5 is referring to the upstream Load Center Supply Breakers.

LICENSEE:

Changed the word from to the word to.

Question determined to be Satisfactory based on Chief Examiner evaluation.

8 H

2 X

X N

E S

NRC Early Review Question NRC:

Plausibility of Distractor A is questionable. Why would an applicant reasonably believe that any of the Feedwater Isolation signals would only result in closure of the Containment Trip Valves (CTVs)?

Revise Item 3 of the stem to state A SG NR level is 81%

and slowly rising.

Replace the words Assume the Reactor has not automatically tripped, with a PZR pressure value that has sufficient margin to the Low Pressure trip and indicate that the value is lowering. Include this information as Item 5 of the stem. As written, the use of the word automatically may be interpreted to mean that the reactor failed to automatically trip when required.

Remove the stem information that states No additional operator actions have been taken. The statement is unnecessary and can lead to confusion on questions where the same or similar statements have not been provided. NUREG-1021, ES-1.2, Paragraph B.8, states:

When answering a question, do not make assumptions about conditions that are not specified in the question unless they occur as a consequence of other conditions that are stated in the question. For example, you should not assume that any alarm has activated unless the question so states or the alarm is expected to activate as a result of the conditions that are stated in the question. Similarly, you should assume that no operator actions have been taken, unless the stem of the question or one of the answer choices specifically state otherwise.

Revise the question statement to read How did the Main Feedwater System automatically respond?

LICENSEE:

Changed the question and the distractors to ask which Main Feedwater component did NOT automatically respond the event. This removes the implausibility issue with distractor A.

Revised stem item 3 to state A SG NR level is 81% and slowly rising.

Replaced the words, Assume the Reactor has not automatically tripped with new item 5 that states RCS pressure is 2150 psia and lowering.

Removed the statement to Assume no additional operator actions have been taken.

Changed the question statement to read, Which of the following Main Feedwater System components did NOT automatically respond to this event?

CHIEF EXAMINER Supplemental Comment for Early Review Question (7/6/2023):

Remove Item #4 which states The BOP operator reports

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level that the Main Turbine has tripped, in light of the change made to the Question Statement requiring the applicants to now determine the MFW System responses that did NOT automatically occur. Item #4 information is not required to correctly answer the question and is potentially leading, given Distractor D, The A TDMFP (3FWS-P2A) tripped. The applicants should know that the Main Turbine tripped on high SG level based on the information provided in the stem; i.e., SG A NR level is 81% and rising.

LICENSEE Supplemental Response:

Removed item 4 from question stem.

Added The Feed Station Operator reports to item 3 from question stem, to be consistent with the stem items 2 and 4.

Question determined to be Satisfactory based on Chief Examiner evaluation.

9 F

2 X

X B

N E

S 2009 MILLSTONE 3 NRC EXAM NRC:

Question does not appear to be written at the RO license level. Associated Pedigree Sheet identifies the question as a Bank Q that was previously used on the Millstone 3 2009 NRC Exam. The Question on the 2009 NRC Exam was SRO Q78.

Note that OP 3272, EOP Users Guide, R14, Attachment 2, Section 7.0, Example 3, identifies specific components to which the guidance contained therein applies and is required to answer the question correctly. Would this specific level of detail, embedded in a 15-page Attachment of a 66-page EOP Users Guide be considered RO Knowledge? Confirm that the question is RO knowledge at Millstone 3.

Question answer choices are predicated upon whether the action taken by the crew, per the EOP Users Guide, IS or IS NOT REQUIRED. The referenced EOP Users Guide section makes no mention of what IS or IS NOT REQUIRED. Accordingly, a case can be made that there is at least two, and quite possibly three correct answers based on writeup in the Explanation Section.

The only IS NOT answer option is Correct Answer D.

Note that the Question Statement provides amplifying information in the form of IF NOT, WHY NOT. This is leading based on the construct of the question, with the potential to cue the applicants.

NOTE: Per ES-5.1, Question coded as an Enhancement (E) since it was used on a previous NRC Exam (2009 Millstone 3 Exam). Accordingly, the question must be either revised or replaced to address the identified flaw. ES-5.1, Section G (1) states:

Note: Do not count questions from past NRC examinations (at any facility) unless the facility licensees current use caused the current unsatisfactory status after the last NRC approval of the test item. (For example, the questions reference changed, but the question was not revised accordingly, or the question was previously associated with a different knowledge and abilities (K/A) statement and its current K/A pairing results in a K/A mismatch flaw.)

LICENSEE:

Replaced question with new question due to concerns with the question requiring SRO level knowledge of detailed requirements within an administrative procedure attachment. The new question focusses on RO level knowledge that can be answered by big picture procedure strategy and system knowledge.

CHIEF EXAMINER Supplemental Comment (8/9/2023):

Editorial - Explanation Section incorrectly references Group 2 in the following statement; should be Group 3:

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level A and B are wrong, since the MSIV closed lights are on Group 2 of the ESF Status Panel.

LICENSEE Supplemental Response:

Changed Explanation Section to reference Group 3 of the ESF Status Panel.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

10 H

3 B

E S

2000 MILLSTONE 3 NRC EXAM NRC:

Insert the acronym RCS between the words uncontrolled and cooldown in Distractor D such that it reads uncontrolled RCS cooldown.

Explanation for Distractor C states that C is plausible because it is the basis for Step 17 in FR-S.1. Confirm the accuracy of Distractor C as the basis for Step 17, given that the question was apparently written approximately 23 years ago.

Associated Pedigree Sheet identifies the question as a Bank Q that was previously used on the Millstone 3 2000 NRC Exam. Does not appear that this question was used on the 2000 Exam based on a search conducted by the Lead Examiner. Resolve this apparent discrepancy.

LICENSEE:

Added RCS between the words uncontrolled and cooldown in distractor D.

Checked current revision of FR-S.1, and changed the explanation section for distractor C to reference step 18.RNO of the current revision of FR-S.1. Also changed the wording of distractor C from shutdown the reactor to add negative reactivity to better match the basis document.

Researched pedigree of question, and verified on NRC.gov that it was used on Millstone 3 2000 NRC Exam (RO #45). The question has NOT been significantly modified from the 2000 NRC Exam version (distractors have been enhanced), but the Exam Bank number has changed, since a different exam bank numbering system was being used in 2000 (NUTIMS versus Vision). The exam bank number of the 2000 Exam was Modified Bank # 0069971.

Question determined to be Satisfactory based on Chief Examiner evaluation.

11 F

2 B

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 12 H

3 N

S 13 F

3 N

E S

NRC:

Delete the word only after Battery 3 in the Question Statement.

LICENSEE:

Per Chief Examiner comments on 7/28/23 phone call, moved the word only from the Question Statement to Distractors B and D Part 2, to avoid being subsets of A and C.

Question determined to be Satisfactory based on Chief Examiner evaluation.

LICENSEE:

Note: Changed distractor order to make A the new correct answer, to balance overall number of correct answers for each letter choice (25 each, +1) after making all changes based on NRC Review.

Supplemental LICENSEE response:

Changed the question statement to ask about the reason AOP 3563, Loss of DC Bus Power directs the crew to trip the Reactor to better match t KA Statement (Based on Don Jackson comment).

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 14 H

3 X

B M

E S

2007 MILLSTONE 3 NRC EXAM NRC:

Associated Pedigree Sheet identifies the question as a Bank Q that was previously used on the Millstone 3 2007 NRC Exam. The Question on the 2007 NRC Exam, RO Q17, appears to have been significantly Modified for use on the 2023 NRC Exam. Should the question source be Modified instead of Bank?

Questioning the plausibility of Distractor B. If an Instrument Air leak occurred, then how could the MFP Recirc Valves, which Fail Open on Loss of Air Pressure, have been unaffected? Plausibility Explanation is deficient.

There are two Distractor C Plausibility Explanations, the second of which is incorrect, as it refers to crew actions associated with the Loss of Feed Control; the subject of Distractor B.

LICENSEE:

Compared Question 14 to 2007 NRC Exam and current bank question 406972. The question has been significantly modified from both the 2007 exam and the bank question.

This question was modified on this exam to improve discrimination validity based on validator comments during exam validation process. Changing question pedigree to Modified.

The question pedigree statement now lists this question as Modified.

Enhanced Plausibility Explanation for Distractor B.

Distractor is plausible, since per stem item 5, some air operated valves have repositioned, but per item 6, the crew is not required to trip the reactor. This allows applicants to apply knowledge of (unstated) reactor trip criteria in AOP 3562 to determine feed control has not been lost. This is plausible, since air pressure has not decreased to zero before air pressure recovers, meaning all air operated components may not have repositioned.

Cleaned up Plausibility Explanation for Distractor C.

LICENSEE Supplemental Comment:

• Based on final validation results after the exam submittal, removed stem item 5, since by mentioning that some valves have repositioned, the validators were focused in on Distractors A and B, which included repositioning valves.

Removing this item does not force the applicant to assume valves did not move.

Question determined to be Satisfactory based on Chief Examiner evaluation.

15 H

3 M

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 16 H

3 B

E S

NRC:

Change the word into to in in the Answer Statement, such that it reads The RCS leak is/was in the ___

LICENSEE:

Changed the word, into to in.

Question determined to be Satisfactory based on Chief Examiner evaluation.

17 H

3 B

S 18 H

2 X

N E

S NRC:

Questioning the plausibility of Distractor D. What leads an applicant to reasonably select Distractor D (establish Main Condensate Flow) over Distractor C (establish Main Feed Flow) to feed at least one SG, given the conditions currently specified in the stem and the fact that Condensate is required for establishment of Main Feed Flow?

Suggest: a) changing B SG WR level to 42%, and b) providing indication that the Containment is Adverse, to raise the Discrimination Validity of the question.

LICENSEE:

Enhanced plausibility explanation for distractor D, describing how FRH.1 tries to establish Main Feed Flow (which requires Main Condensate flow), and if unsuccessful, the crew will then try to establish Main Condensate flow. This would be the case if there were a problem with the Main Feed System. Several actions are required if establishing Main Feed flow is unsuccessful, with the most significant being the SGs must be depressurized to below Main Condensate Pump shutoff head.

Per discussion with Chief Examiner (phone call, 7/28/23) changed SG WR levels to be significantly closer to bleed and feed requirements (2 of 4 SG Wide Range levels below Bleed and Feed criterion) without adding adverse containment conditions, since this may be getting closer to SRO level knowledge, beyond overall strategy of the procedure and beyond system knowledge.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 19 F

2 X

N U

S NRC:

For Sequence of Events Item 1, need to specify that rods are stepping OUT to comply with the K/A Statement.

Distractor B does not appear to be credible given that the DC Hold Bus was not in use.

Distractor D does not appear to be credible given that the event is a Continuous Rod Withdrawal and the RIL Computer has no control function.

LICENSEE:

Chief Examiner randomly reselected the K/A since Automatic Rod withdrawal capability has been permanently disabled at Millstone 3.

Replaced the question.

Based on NRC input, changed part 2 of the new question to ask if rod is trippable, due to overlap with operating exam.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

20 F

H 2

B E

S 2021 MILLSTONE 3 NRC EXAM (Q20 removed from the CERT Exam)

(Previous 2 NRC Exams)

NRC:

Level of Knowledge (LOK) is classified as Low Cognitive Level (LCL) (i.e., Fundamental) on the Form 4.2-1 Worksheet. Appears that the question should be classified as High Cognitive Level (HCL) based on the guidance in NUREG-1021, Appendix A, regarding the number and types of mental steps necessary to identify the correct answer. Explain the Low Cog assignment.

Note that the same question on the 2021 MS3 ILE was classified as HCL LICENSEE:

Changed level of knowledge to Comprehension or Analysis.

Question determined to be Satisfactory based on Chief Examiner evaluation.

21 F

3 N

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 22 F

H 2

X B

U S

NRC:

K/A Mismatch flaw. Question can be answered without any knowledge of the Fuel Handling Incident Event. Question, as written, only tests applicant knowledge of the communications that need to be established to support Fuel Handling Operations.

LICENSEE:

Wrote a new question that tests specific RO job functions during a fuel handling event, rather than generic RO responsibilities during fuel handling operations.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

23 H

2 X

M B

U S

2019 MILLSTONE 3 NRC EXAM (Previous 2 NRC Exams)

NRC:

Distractors A and B do not appear to be credible. Why would an applicant reasonably believe that insufficient Circ Water Flow or insufficient Gland Sealing Steam Pressure would be the cause of decreasing vacuum if Air Ejector (AE) Discharge Flow is 0 SCFM?

What makes it possible to eliminate Distractor C entirely?

Would not AE Discharge Flow be 0 SCFM if the AE Steam Supply was lost? Is it possible to eliminate Distractor C solely on the basis of elevated Suction Manifold Temperature, as noted in the Explanation? Plausibility Explanation for Distractor C is deficient in this regard.

LICENSEE:

Changed this modified question back to the original bank question per phone call with Chief Examiner on 7/28/23 (acceptable as long as the bank question is satisfactory).

CHIEF EXAMINER Supplemental Comment for Early Review Question (8/9/2023):

Provide a value for Gland Sealing Steam Pressure that is more Discriminating than 0.5 PSIG.

Suggest enhancing Distractor C by changing it to Degraded steam supply pressure to the SJAEs, to raise the Discrimination Validity.

Distractor A Plausibility Explanation states that Circ Water flow is abnormally low. Enhance the Plausibility Explanation to include an explanation of why this is the case based on the conditions provided in the stem. What is the normal Circ Water System configuration and corresponding flows values?

Distractor C Plausibility Explanation states that the Distractor is wrong since SJAE flow is greater than normal. Enhance the Plausibility Explanation to include an explanation of why this is the case based on the conditions provided in the stem. What is normal SJAE Flow?

LICENSEE Supplemental Response:

Changed Gland Sealing Steam (GSS) pressure to 1.0 psig (Chose not to raise it higher, since even though this is still well out of spec, it is not known how much actual GSS pressure would be required to prevent air inleakage.

Enhanced the Distractor A Plausibility Explanation to include the normal Circ Water System configuration and flowrate.

Changed Distractor C to say, Degraded steam supply pressure to the SJAEs.

Enhanced the Distractor C Plausibility Explanation to include the normal SJAE flow.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 24 F

H 1

X N

B U

E S

NRC Early Review Question 2017 MILLSTONE 3 NRC EXAM NRC:

LOD=1. The three Distractors are closed cooling water systems that are not remotely associated with the Liquid Radwaste Release activity.

LICENSEE:

Unable to write a discriminating question for the APE 59 and AK1.06 combination due to limited RO job function.

RO will understand Circ Water provides dilution flow for liquid waste discharges, and inform the US if a Circ Pump is lost during a discharge.

Chief Examiner Randomly Reselected a new APE and KA.

Replaced the question.

CHIEF EXAMINER Supplemental Comment for Replacement Question (7/7/2023):

NRC Lead Examiner:

Question determined to Satisfactory; however, the following Plausibility Explanation associated with Distractors C and D is incorrect:

C and D are wrong, since the crew is allowed to have immediate boration continue from the RWST on a Reactor trip.

Plausibility Explanation should be revised to state:

C and D are wrong, since the crew is allowed to have immediate boration continue through its normal path on a Reactor trip.

See Question 30 on the 2017 MS3 NRC Exam.

Replacement question COG Level is HCL per the Pedigree Sheet Replacement Q Source is Bank.

LICENSEE:

Changed the Explanation as to why C and D are wrong, to say, the crew is allowed to have immediate boration continue through its normal path on a Reactor trip.

Will update ES Form 2.3-4 Written Examination Quality Checklist for all questions after the Exam is approved.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 25 F

1 2

N U

S NRC:

LOD=1. Silencing the local alarm horn and resetting the fire alarm are standard capabilities that provide little, if any insight, into the ability of an applicant to operate and/or monitor Plant Fire Zone Panels. Correct Answer B is the obvious choice, even if uncertain, given the other answer options. Recommend testing one of the other Zone Panel capabilities listed in the Explanation Section.

LICENSEE:

Redrew KA with Chief Examiner, due to difficulty writing a discriminating question with plausible distractors since there is limited zone panel capabilities.

Replaced the question.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

26 H

2 X

N E

S NRC:

Distractor D does not appear to be credible, as there is no RED Path associated with FR-Z.2.

Revise the question to make Distractor A the correct answer, in order to raise the overall Discrimination Validity of the question. Suggest a Containment Pressure value of 58 psia and stable, with no Containment Spray Pumps running to support this change.

LICENSEE:

Not able to change to 58 psia with no Containment Spray Pumps running since this overlaps with Operating Exam.

Per discussion with Chief Examiner on 7/28/23, discerning Containment Sump Level color is acceptable for RO portion of exam, and changed Containment pressure to 61 psia to raise discrimination validity of the question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

27 F

3 N

S 28 H

3 B

S 2017 MILLSTONE 3 NRC EXAM

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 29 H

F 2

X X

B U

S NRC:

K/A Mismatch flaw. Question tests applicant knowledge of automatic Makeup System flowrates (i.e., Boration and Primary Grade Water), not automatic operation of VCT level control.

Questioning the plausibility of Distractors C and D, given that Primary Grade Water Flowrate is 80 GPM for both. If Total Flow controls at 80 GPM for Automatic Operation, then how could the combined Boration and Primary Grade Water Flowrates exceed 80 GPM total, as is the case for each of these Distractors?

LICENSEE:

Replaced question with a new question that better tests monitoring of automatic operation of VCT level control.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

30 F

2 B

E S

2002 MILLSTONE 3 NRC EXAM NRC:

Operation History Field on the Pedigree Sheet incorrectly references the MS3 2001 ILE. Question was Q42 on the 2002 ILE.

Use the MS3 2002 ILE version of the question (i.e., Q42),

to raise the overall Discrimination Validity. The applicants should be required to determine which valve, the Suction or the Discharge, must be closed first, as on the 2002 ILE.

The 2023 ILE Question, as written, tells the applicant upfront that the Discharge Valve is required to be closed prior to the Suction Valve.

Procedure OP 3304A, Charging and Letdown, was not included with the References provided to the NRC.

Confirm that the reason the Discharge Valve is closed prior to the Suction Valve is covered/addressed in the Precaution Section of OP 3304A, as alluded to in the stem.

LICENSEE:

Changed question to match 2002 version.

Changed question pedigree to state last used on Millstone 3 2002 NRC Exam Verified Precaution 3.3.8 still exists and provides the listed basis in the current revision of OP 3304A (Rev. 47).

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 31 H

2 X

B E

S 2007 MILLSTONE 3 NRC EXAM NRC:

Distractor C does not appear to be plausible. If reducing RHR HDR FLOW from 4,000 gpm to 3,200 gpm as stated in the stem, then how is it possible to have an increase in flow through FCV-606 with NO change in flow through FCV-618 (i.e., Remains the same)? Where does the overall reduction in RHR HDR FLOW come from? The combined Plausibility Explanation for Distractors A & C is reasonable for Distractor A because of the Decreases option in Part 1 of the answer. Conversely however, it fails to sufficiently address the credibility of Distractor C because of the Increases option in Part 1.

Per Chief Examiner (CE) evaluation, this question, as written, satisfies the criteria for being significantly modified from Q31 on the 2007 NRC ILE because: a) Part 2 of the answers is based on RHR Pump Recirc Flow instead of RHR HX Bypass Flow, and b) the Part 2 answer options are different (see 3rd criteria for what constitutes a Significant Modification in ES 4.2, Page 4 of 29; i.e., CE determination).

LICENSEE:

Verified the question matches the current bank question, so it is considered Bank, not modified. Also verified the bank question has changed significantly since it was used on the 2007 NRC Exam, so removed the reference to 2007 NRC Exam on the pedigree sheet.

Added Recirc Valve position at the start of the event as being closed. This creates the potential for the recirc valve opening (at discharge of heat exchanger) as overall flow decreases, which would cause heat exchanger flow to increase.

Modified Explanation section to explain why C is plausible based on decreasing flow potentially opening the recirc valve.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 32 H

3 X

N U

S NRC:

K/A Mismatch flaw. K/A states:

Ability to (a) predict the impacts of the following on the Emergency Core Cooling System and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operations: ECCS valve failure mode.

While the question is technically correct, there is no impact to the running ECCS Pumps that would require the applicant to demonstrate the ability to use procedures to correct, control, or mitigate the consequences of the spurious closure of 3SIL*MV8804A during Cold Leg Recirculation.

LICENSEE:

Chief Examiner randomly reselected a new K/A due to difficulty writing a discriminating question related to part 2 of the KA. If a valve fails, the crew will attempt to reposition the valve.

Replaced the question.

Based on Chief Examiner input, changed part 2 of the new question to test actions on single train inadvertent SIS.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

33 F

2 B

E S

2002 MILLSTONE 3 NRC EXAM NRC:

Question is confusing as written, given that Correct Answer C is based on what RCS pressure is doing, rather than a determination of RCS subcooling and/or Pressurizer Level, in accordance with Step 14 of ES-1.2 In addition, the question on the 2023 NRC ILE appears to be markedly different than version (Q28) on the 2002 NRC ILE, which requires a determination of both RCS Subcooling and Pressurizer Level. Recommend replacing the question with the version on the 2002 Exam, which is referenced on the Pedigree Sheet as the question used.

Ensure the Correct Answer and Distractor Explanations contain the necessary supporting information, as the question was developed over 20 years ago and No Distractor Explanations were provided for the 2002 Exam.

LICENSEE:

Changed the 2023 version of the exam to more closely match the 2002 version, especially by changing distractor D to match the 2002 version.

Changed stem item 3 to state the RO reports RCS pressure, RCS subcooling, and Pzr level are all initially decreasing, to improve plausibility of new distractor D, which is based on RCS subcooling and Pzr level.

Kept the existing correct answer, which matches the current EOP user guide and the current revision of ES-1.2.

Ensured the Explanation section includes justification and plausibility information for each of the distractors.

CHIEF EXAMINER Supplemental Comment for Early Review Question (8/10/2023):

Potential exists for two correct answers given the changes made to Distractor D.

LICENSEE Supplemental Response:

Changed Correct Answer C and Distractor D to the following:

c)

Monitor RCS pressure. Allow pressure to stabilize or increase prior to evaluating whether adequate subcooling exists to stop the "B" SIH pump.

d)

Monitor RCS subcooling. Ensure RCS subcooling stabilizes at or above its SI Reinitiation Criterion value. If not, restart the B Charging Pump.

This change was made to make D more clearly wrong, and keep C around the same length as D.

Question determined to be Satisfactory based on Chief Examiner evaluation.

34 H

3 X

B U

NRC Early Review Question

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level F

S NRC:

K/A Mismatch Flaw. K/A states:

Knowledge of the operational implications or cause and effect relationships of the following concepts as they apply to the Pressurizer Relief Tank/Quench Tank System: Method of forming a steam bubble in the PZR.

Question, as written, does not test applicant knowledge any operational implication or cause and effect relationship that exists between the PRT/Quench Tank and the method of forming a steam bubble in the Pressurizer. The PRT/Quench Tank is merely used as a Distractor in the question.

LICENSEE:

Millstone 3 does not normally draw a Pzr bubble by routing water to the PRT. Chief Examiner randomly reselected a new K/A; Question replaced.

CHIEF EXAMINER Supplemental Comment for Replacement Question (7/11/2023):

NRC Lead Examiner:

Questioning the plausibility of Distractors C and D. Why would an applicant reasonably believe that selecting Block would allow for the Automatic function to occur?

Has a Block feature ever been designed that would serve to enable an Automatic function?

LICENSEE Supplemental Response:

Changed the question statement from Selecting Block allows for 3DGS-AV8031 to____ to Selecting Block allows for 3DGS-AV8031 to be manually _____. This allowed removing the automatic function from distractors C and D.

Changed the distractors to grammatically flow with the new question statement. A and B now say, (manually) opened when, and C and D now say, (manually) closed when Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 35 F

3 B

E S

NRC:

In the As Written question, what is the purpose of the stem item that states CIA fails to actuate? Based on this information, suggest revising the Question Statement to read something similar to the following: Which RPCCW valves repositioned in response to the sequence of events identified?

LICENSEE:

The purpose of the stem item is to raise the difficulty of the question, since CIA is normally generated from any SIS signal. By stating CIA fails to actuate, applicants are required to know which specific signal (SIS or CIA) causes the components to reposition. Added this information to the beginning of the Explanation section.

Changed the question statement to read, Which RPCCW valves received an automatic signal due to this event?

Question determined to be Satisfactory based on Chief Examiner evaluation.

36 F

H 3

X N

B U

S 2009 MILLSTONE 3 NRC EXAM NRC:

K/A Mismatch flaw. Question, as written, tests applicant knowledge of the physical connections between the Pressurizer System and the CVCS System. Question does not test the physical connections or the cause and effect relationships between the Pressurizer Pressure Control System and the CVCS System.

LICENSEE:

Chief Examiner randomly reselected a new K/A, since there the direct physical connections and cause/effect relationships between the CVCS System and the PPC System are very limited.

Replaced the question.

Supplemental LICENSEE Comment:

Based on final post exam submittal validation, corrected a typo in stem item 1, changing A to An.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 37 H

2 B

E S

2009 MILLSTONE 3 NRC EXAM NRC:

Enhance Distractor D by expressing the bistable status light coincidence combination per loop for the Low Reactor Coolant Flow Trip.

Enhance the overall Discrimination Validity of this question by requiring the applicants to evaluate the status of both the P-10 and P-13 indications in determining whether the P-7 permissive is met (Rx Pwr <10%), instead of just telling the applicants that Rx Power is stable at 7%.

Change Distractor C to Two bistable lights lit for PZR LEVEL HI, instead of three.

LICENSEE:

Changed distractor D to, Three lit RCS Loop Flow Low bistables per RCS loop. Changed four to three since each loops low flow trip is a 2 of 3 coincidence. Also added for the C SG to distractor A, to maintain similar distractor lengths.

Changed distractor C to Two bistables lit.

Based on Chief Examiner comment on 7/28/23, added one NIS channel fails high to question stem.

Changed Explanation section to support changes to question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

38 H

3 X

B E

S NRC:

Distractor D does not appear to be plausible as the High Pressurizer Pressure Trip is 2/4 coincidence.

Explanation Section is confusing in terms of why Answer A is correct and the Distractors are implausible. Explanation incoherently strings together a series of informational items that are not clearly or logically connected, making them difficult to interpret.

LICENSEE:

Distractor D is plausible, since it requires the applicant to determine which bistables were received from each of the two instrument failures, and which bistables were required to be tripped per the AOP based on the temperature instrument failure. Both instruments feed in to multiple Bistables, and had the Pressure instrument failed low instead of high, the reactor would have tripped.

Enhanced the Explanation Section to improve readability, and better explain why distractor D is plausible.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 39 F

X B

E S

NRC:

Remove the words if at all from the Question Statement.

Questioning the plausibility of Distractor D. Why is it plausible that All Emergency ACUs and Emergency Fans would start on the manual start of a single ECCS Pump?

Explanation for Distractor D provides no insight as to why this would be credible.

LICENSEE:

Changed question stem to state, What is the response of the ESF Building HVAC System?

D is considered plausible, since most building ventilation systems (e.g., Main Steam Valve Building, Control Building, Aux Building) have two modes, normal and emergency, where the entire system realigns during an emergency.

Enhanced the plausibility description for distractor D in the Explanation section.

Question determined to be Satisfactory based on Chief Examiner evaluation.

40 H

3 N

S Reference provided:

Detached Reference (Separate Handout); Electrical Drawing ESK-6BC (R10); Elementary Diagram 480V US CNTMT AIR RECIRC FAN [3HVU-FN1C]

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 41 H

2 X

B E

S 2004 MILLSTONE 3 NRC EXAM NRC:

Per EOP-35, GA-8, Step 2, both SI and CDA are procedurally required to be Reset before performing Step 3 to Stop the Quench Spray Pumps. The actions that would be performed by an RO in the plant under the postulated conditions would be to Reset both SIS and CDA. Question was last on the Millstone 3 2004 NRC Exam. While Resetting only the CDA signal is a technically correct answer, is it fair to test the applicants on the conduct of a task that would be performed differently in accordance with station operating procedures? Note that ES-1.2, Guidelines For Taking NRC Examinations, Section B, Paragraph 8, which the applicants are briefed on by the Lead Examiner, states in part:

Finally, answer all questions based on actual plant operation, procedures, and references.

LICENSEE:

Changed question to ask what actions are required per GA-8, Stopping Containment Spray?. This changed the correct answer.

Changed the Explanation section to support the new correct answer.

Supplemental LICENSEE Comment:

• Based on final post-exam submittal validation changed the question statement to say, GA-8 will direct the RO to reset, since the validator was unsure if the question was asking about what the GA directed, or what was required to allow stopping the pump.

Question determined to be Satisfactory based on Chief Examiner evaluation.

42 H

2 N

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 43 H

2 N

E S

NRC:

Distractor D does not appear to be plausible. Why would an applicant believe that the Condensate Pump Recirc Min Flow Valve would automatically open after the MFP Recirc Valve has Failed Open, diverting additional flow to the Main Condenser?

LICENSEE:

• Changed distractor D to, MSR Drain Pump 3DSM-P1A trips.

• Changed the Explanation section to support the new Distractor D.

CHIEF EXAMINER Supplemental Comment (8/10/2023):

Distractor D Replacement, MSR Drain Pump 3DSM-P1A trips, does not appear to be credible, especially given the fact that Distractor B is a Trip of 4th Point Heater Drain Pump 3HDL-P1A.

LICENSEE Supplemental Response:

• Replaced Distractor D with Condenser Desuperheating Spray Valve 3CNM-PV99 opens.

• Updated Explanation Section for Distractor D.

Question determined to be Satisfactory based on Chief Examiner evaluation.

44 H

3 N

E S

NRC:

Is there a procedure to accomplish the task of satisfying the interlocks to stop the MDAFW Pump and realign the suction path to the CST from MB5? If so, it should be included/referenced in the Question Statement.

LICENSEE:

Added new stem item 3, which states, The US directs the BOP Operator to use GA-4, Transfer AFW Pump Suction and Fill DWST to transfer the A MDAFW Pump suction to the CST.

Question determined to be Satisfactory based on Chief Examiner evaluation.

45 H

2 N

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 46 H

2 X

B E

S NRC:

The single phase ground information in the stem, together with the Part 2 answer stating that there are three ground detection lights, with the grounded phase appearing dimmer than the other two, is leading, making it possible for the applicants to easily eliminate the Part 2 answers for Distractors B and D. The grounded phase appearing dimmer than the other two is more information than is necessary and is too descriptive in this case.

Suggest raising the Discrimination Validity of this question by revising the Part 2 answers read as follows:

(2) ground detection lights (Choices A & C)

(2) a volts to ground meter (Choices B & D)

LICENSEE:

Revised part 2 answers to choose between ground detection lights and a volts to ground meter.

Question determined to be Satisfactory based on Chief Examiner evaluation.

47 F

3 B

S 2011 MILLSTONE 3 NRC EXAM 48 H

3 B

E S

2002 MILLSTONE 3 NRC EXAM NRC:

Q75, Distractor B, on the 2002 NRC Exam, specifies starting the Fuel Oil Transfer Pumps from 3EGS*PNLA.

Q48, Distractor B, on the 2023 NRC Exam specifies starting the B Train Fuel Oil Transfer Pump from 3EGS*PNLA. This is the only difference in the two questions. Why was the change made to identify only the B Train Fuel Oil Transfer Pump for Distractor B on the 2023 Exam? Concern is the potential for inadvertently cueing the applicants, given that B Train power is not available.

LICENSEE:

Stem item 6 states that although (A Train) power is available, neither A Train Fuel Oil Transfer Pump can be started. Based on this, it improves readability to make the change to discuss the B Train Transfer Pumps.

Also changed the question and the distractors to more clearly focus on how electrical power will be provided, since that is the KA the question is testing. This change was not based on the original NRC comments, but the Chief Examiner agreed with this change on the 7/28/23 phone call.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 49 H

3 B

S 50 F

2 N

E S

NRC:

Confirm that Stepping the Filter causes 3CMS-RE22 indication to drift upward while a new background calculation is performed, as stated in Correct Answer D.

The Answer Explanation is silent regarding the impact on 3CMS-RE22 indication.

LICENSEE:

OP 3362, Note prior to step 4.16.1 states the rad monitor will be inoperable for up to two hours before the indication returns to normal (implying the indication will be abnormal) while the background calculation is performed. The fact the indication drifts upward during this process was confirmed by an SRO validator who is involved with work control. This information has been added to the Explanation section of the question.

Supplemental LICENSEE Comment:

• Based on post exam submittal validation, removed the direction of the trend from correct answer C and distractor D, since the rad monitor will drift, but it could drift in either direction. This change makes these choices better match what the procedure Note states.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 51 H

3 B

N U

S NRC:

K/A Mismatch flaw. K/A states:

Ability to (a) predict the impacts of the following on the Service Water System and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operations: Controller and positioner failure.

Although technically correct, there is no aspect of the question that requires the applicants to predict the impact of, and demonstrate the ability to use procedures to correct, control, or mitigate the consequences of a Service Water System Controller/Positioner failure.

LICENSEE:

Chief Examiner randomly reselected a new K/A due to difficulty writing discriminating way to test the second part of the KA. There are limited controllers and positioners in the SWP System, and procedural actions are also limited.

Replaced the question.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

52 H

2 B

S 53 F

3 N

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 54 F

2 X

B U

S NRC Early Review Question 2009 MILLSTONE 3 NRC EXAM NRC:

What does the acronym ADTS stand for?

Distractors A and B do not appear to be credible. Why would the ADTS have to be consulted if there was a predetermined time (i.e., 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> or 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />) for placing the CTMT Purge System. Recommend using Hydrogen Concentration percentages for all of the answer options.

For Distractor D, 10 % appears to be a rather high value before taking any action to reduce Hydrogen Concentration.

LICENSEE:

ADTS stands for Assistant Director of Technical Support, which is a Emergency Response Organization person who provides technical support to the crew.

KA is testing knowledge of a specific step late in E-1 after the crew has aligned for cold leg recirculation. It is minutia to know the Ctmt hydrogen concentration from memory that requires operator action with Containment Purge.

Chief Examiner Randomly Reselected a new KA.

Replaced the question.

CHIEF EXAMINER Supplemental Comment for Replacement Question (7/12/2023):

Question History updated to reflect 2009 MILLSTONE 3 NRC EXAM on the Pedigree Sheet.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 55 H

3 B

E S

2019 MILLSTONE 3 NRC EXAM (Previous 2 NRC Exams)

NRC:

Pedigree Sheet indicates that this question was used on the 2019 Millstone 3 NRC Exam. However, review of ADAMS record documentation shows that the question was in fact not used on the 2019 Exam. Rather, the 2023 NRC Exam version of the question served as the Source /

Parent Question for Modified Q18 on the 2019 Exam.

Correct this item on the Pedigree Sheet.

LICENSEE:

Removed 2019 NRC Exam reference from pedigree sheet Question determined to be Satisfactory based on Chief Examiner evaluation.

56 F

3 N

S 57 F

H 2

X N

B U

S 2001 MILLSTONE 3 NRC EXAM NRC:

K/A Mismatch flaw. Question does not meet the intent of the K/A, which states:

Knowledge of the operational implications or cause and effect relationships of the following concepts as they apply to the Nuclear Instrumentation System: Maximum disagreement allowed between channels.

Question, as written, tests applicant knowledge of a) the Power Change Channel Deviation value that corresponds to the associated annunciator setpoint, and b) the desired target band for all NIS Channels after adjustment is made.

Question does not test applicant knowledge of the operational implications or cause and effect relationships of the Maximum disagreement permitted between NIS channels.

LICENSEE:

Chief Examiner randomly reselected a new K/A, given that the Operational Implications of reaching the maximum disagreement between NIS channels are limited, other than having to adjust the channel that is reading out of spec.

Replaced the question.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

58 H

2 X

N M

U S

NRC:

K/A Mismatch flaw. Question does not meet the intent of the K/A, which states:

Knowledge of the effect that a loss or malfunction of the Containment Purge System will have on the following systems or system parameters: CNT.

There has been no loss or malfunction of the Containment Purge System in this question. The question, as written, tests applicant knowledge of the Containment Purge System response to the failure of Fuel Drop Radiation Monitor 3RMS*RE42 (fails into Hi Alarm) with the Containment Equipment Hatch Open. Is 3RMS*RE42 considered part of the Containment Purge System?

LICENSEE:

Chief Examiner randomly reselected a new K/A due to excessive overlap with the operating exam.

Replaced the question.

Supplemental LICENSEE Comment:

1. Initial version of the new question was viewed as a KA mismatch, so the question was changed.

2. The following changes were made to the new version of the replacement question based on final post exam submittal validation comments.

• Changed the trend for the first two radiation monitors listed in the question stem to state, radiation levels are rising, since the validator wasnt sure if the trend as previously written was trying to describe a continuously rising trend, or a step increase in the past.

• Changed top two bulleted items in the question stem to state, The post-core offload fuel shuffle has been completed.

A PEO has alternated the in-service Fuel Pool Purification System (SFC) prefilters.

• The first bullet was changed since the previous wording was ambiguous.

• The second bullet was changed to change the word filters to prefilters, since these are the only filters that can be swapped.

• Changed the same wording in distractor C to also say prefilters, since the validator considered C to possibly be a second correct answer as previously written. This change makes C more clearly wrong.

• Updated the question explanation section with an updated description of distractor C.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

59 H

3 B

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 60 H

3 N

S NRC:

Question K/A (A2.03) is a coupled K/A per ES-4.2, Section C.2, and reads as follows:

Ability to (a) predict the impacts of the following on the Steam Generator System and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those abnormal operations: Pressure/level transmitter failure.

ES-4.2, Section C.2, states:

When selecting or writing questions for K/As that test coupled K/As (e.g., the A.2 K/A statements in Tiers 1 and 2 and a number of generic K/A statements such as 2.4.1, in Tier 3), try to test both aspects of the K/A statement. If that is not possible without expending an inordinate amount of resources, limit the scope of the question to that aspect of the K/A statement requiring the highest cognitive level (e.g., the (b) portion of the A.2 K/A statements) or substitute another randomly selected K/A.

Tier 2 Question 60, as written, appropriately tests that aspect of K/A statement requiring the highest cognitive level (HCL). In this case, the (a) portion of K/A A2.03 satisfies the HCL criteria; i.e., predict the impacts of a Main Steam Pressure Transmitter failure on the SG System. It was not possible to test the (b) portion of the coupled K/A statement without expending an inordinate amount of resources; as determined by the Licensee in concert with the Chief Examiner (CE).

Q60 determined to be Satisfactory based on 1) CE evaluation of the question, and 2) Program Office confirmation of CE interpretation of the aforementioned guidance in ES-4.2, Section C.2. Program Office confirmation received on 7/13/2023.

Update the Pedigree Sheet to include the highlighted information.

LICENSEE:

Updated pedigree sheet with the highlighted information.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 61 F

2 N

E S

NRC:

Question is Borderline LOD=1 as written. Suggest the following to raise the overall Discrimination Validity of this question:

Leave Part 1 as is. Recommend using Part 2 to test applicant knowledge of system operation that is more substantive in nature; for example, system flow rates upon initiation (i.e., Supply, Exhaust, Recirc, etc). Saw several values in the References provided (e.g., 800, 1200, 1400, 4000, 20000 scfm). There may be other aspects of system operation that would work as well. System flow rates were provided as an example.

LICENSEE:

Left Part 1 as is. Changed part 2 to test which of the two Emergency Filters is expected to be in service.

Question determined to be Satisfactory based on Chief Examiner evaluation.

62 F

2 X

B E

S NRC Early Review Question NRC:

Is it necessary to specify DAS in Correct Answer A?

Concern is the potential for cueing the applicant.

Is it necessary to specify outside the ESF Building in Distractor B.? Concern is the potential for cueing the applicant.

Given the plausibility explanation for Distractor D, i.e., D is plausible, if a high radiation alarm is received, this Sump will divert to the TPCCW Sump, which drains to the LWS System, could this answer choice also be correct?

LICENEE:

Removed the three and four letter system designators from the distractors.

Kept the outside the ESF Building specifier, since this is in the Radiologically Controlled Area (RCA), and there is another SRW sump outside the Emergency Diesel Enclosure, and which is a less plausible distractor.

Did not change distractor D, since the question stem states all systems are in their normal lineups, and a high radiation alarm in the Turbine Building is not normal.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 63 F

2 N

E S

NRC:

Distractor B, per the Plausibility Explanation, is a characteristic of Geiger-Mueller detectors, but not an advantage, making it technically correct. The case can therefore be made that there are two correct answers to this question. Replace Distractor B.

LICENSEE:

It was difficult to come up with a fourth plausible answer, so the question statement was changed to say, The Low Range Area Radiation Monitors are designed to operate in the Geiger-Mueller region of the gas amplification curve since a specific advantage of operating in this detector operating region is.

This change aligns with ES-4.2, item D.15.j (ES-4.2, Page 17 of 29), which states, When appropriate, use distractors that are generically correct statements, but do not correctly answer the question Per this guidance, existing distractor B is now clearly wrong, since even though it is a generically correct answer, it is not an advantage of this type of detector, so it does not correctly answer the question.

Question determined to be Satisfactory based on Chief Examiner evaluation.

64 F

1 2

B U

S NRC:

LOD=1 as written. Correct Answer A is obvious given the other choices.

LICENSEE:

Changed question to another bank question covering the same KA.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

65 H

3 B

S LICENSEE Comment:

Changed the word VIOLATE in the question statement to more appropriately state, NOT comply with, based on NRC Branch Chief review comment.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 66 H

2 B

E S

NRC Early Review Question NRC:

Plausibility Explanation for Distractor A incorrectly states that Answer D is plausible and that the action taken for Distractor D was taken due to ALARA concerns. D is the correct answer.

LICENSEE:

Changed D to A at both locations in the justification.

Question determined to be Satisfactory based on Chief Examiner evaluation.

67 F

2 B

S 68 F

2 N

S 69 F

3 B

S

70 F

2 X

B N

U S

NRC:

Question is confusing as written. The applicant is initially told that rods are withdrawn several steps while observing an initial increase in Source Range Counts and SUR, then rod withdrawal is stopped.

The ensuing Question Statement, reads as follows:

Assuming the Reactor is close to, but not yet critical, how will source range counts continue to respond after rod motion?

Do the words Assuming the Reactor is close to, but not yet critical mean prior to the rod withdrawal or after the rod withdrawal? This matters, because if the words are intended to mean prior to rod withdrawal, then without any amplifying information, the reactor may either remain subcritical or achieve criticality. In the later case, Distractor D would be a correct answer. If the words are intended to mean after to rod withdrawal, then Distractor D is not credible because the Question Statement is telling the applicant that the state of the Reactor is subcritical (i.e.,

Reactor is close to, but not yet critical), which clearly eliminates Distractor D.

Separately, Question does not sufficiently test applicant knowledge of the concept of Subcritical Multiplication and is therefore UNSAT. All the applicant needs to know to correctly answer this question is that the closer to criticality, the higher the neutron population and the longer it takes to reach stabilization, without any basic understanding as to the reasons why or what Subcritical Multiplication actually is. The K/A specifically states:

Explain the concept of Subcritical Multiplication.

For example, the applicant is not required to demonstrate the understanding that Subcritical Multiplication is the process of utilizing source neutrons and fission neutrons to maintain a constant neutron population with Keff < 1.0 OR an understanding that the number of neutrons produced from fission in subsequent generations is less than the number produced from fission in the immediately preceding generation in a subcritical reactor (Keff < 1),

where the number of generations it takes to reach a stable/constant neutron population depends on the value of Keff. The closer Keff, is to criticality (1.0) the more generations are required to reach stabilization and therefore more time. Conversely, the further Keff is from criticality, fewer generations are required to reach stabilization and therefore less time.

LICENSEE:

Changed question to include the theory behind subcritical multiplication, not just the effect of it.

Updated the Explanation Section.

Question determined to be Satisfactory based on Chief Examiner evaluation.

71 F

2 X

N E

S NRC:

Question addresses the Integral Rod Worth (IRW) aspect of the K/A only. Enhance question to also address the Differential Rod Worth (DRW).

RCS Boron and Xenon Concentrations are essentially the same answer; i.e., both are poison concentrations.

Question still requires a third Distractor.

Distractor D does not follow grammatically from stem; i.e, due to the effects of Fuel temperature profile.

Readability is awkward for this Distractor which can cue the applicant that this choice is incorrect.

LICENSEE:

Added Differential Rod Worth to the existing Integral Rod Worth in the question body and statement.

Changed the question statement into a two-part question, to allow testing of aspects of both the IRW and DRW curves.

Changing the question into a two-part question (to test both IRW and DRW) solved the issue with needing a third distractor, since it allowed the removal of RCS Boron as one of the distractors.

Changed Fuel Temperature Profile to the Fuel Temperature Profile to improve readability of these distractors.

CHIEF EXAMINER Supplemental Comment (8/10/2023):

Appears that Part 1 of Distractors C and D would require the applicant to make an assumption regarding the Xenon concentration in the core, given that it will vary depending on the operational condition. If we are talking about Integrated Rod Worth, and Part 1 of Choices A and B pertains to Rods, how is Xenon concentration below 115 steps a credible alternative to an answer containing Rods?. Choices A and B seem kind of obvious.

Explanation Section provides no insight into the credibility of Part 1 for Distractors C and D.

Question Statement is with respect the slope of the IRW Curve below 115 steps, whereas the Explanation Section is with respect the slope of the IRW Curve below 110 steps. Clarification required.

LICENSEE Supplemental Response:

Per phone call with Chief Examiner (8/11/23), changed part 1 of new distractors C and D from Xenon concentration to boron concentration to keep applicants from having to make assumptions about Xenon conditions in the core.

Changed Explanation section for distractor D to match this change. This change included correcting the rod height to 115 steps, to match the rod height in the question statement.

Made the following changes based on post exam submittal final validation comments:

Changed part 1 of distractors C and D to say, of the boron concentration to grammatically flow with the question statement.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level Changed part 2 of distractor D to say, Axial Neutron Flux Distribution to match Correct Answer B.

Question determined to be Satisfactory based on Chief Examiner evaluation.

72 F

2 N

S 73 H

3 N

E S

NRC:

Remove the 1100 psia parenthetical information in the stem. Applicants should know to convert psig to psia when using the Steam Tables.

LICENSEE:

Removed the 1100 psia parenthetical information from the stem Question determined to be Satisfactory based on Chief Examiner evaluation.

74 H

3 B

E S

NRC:

Explanation for Distractor A states:

A is wrong since QSG = mh, and artificially low feed temperature means the calorimetric will assume the SGs are having to heat up the feedwater a lesser amount to reach saturation temperature, Statement should read artificially high.

LICENSEE:

Changed artificially low feed temperature to artificially high feed temperature in the Explanation section for distractor A Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 75 H

1 3

M N

U S

NRC:

The stem states:

Complete the following statement, assuming adequate natural circulation cooling is occurring.

Distractors B and D are not credible and can easily be eliminated, making this question LOD=1. If the assumption is that Natural Circulation is occurring (specified in the stem), then Part 2 of these Distractors cannot possibly be correct if CETC temperatures are rising.

Plausibility Explanation for Distractor D incorrectly states:

D is plausible, since CETCs will be decreasing.

Part 2 of Distractor D is rising. CETCs decreasing would be indication that Natural Circulation is occurring. Appears to be a Cut-and-Paste error.

LICENSEE:

Added RCS pressure to the question stem, and changed the second part of the question to ask the maximum Core Exit Thermocouple temperature allowed.

Supplemental LICENSEE Comment:

Made the following changes based on final post exam submittal validation.

Changed the question statement to ask the minimum subcooling required per ES-0.1, Reactor Trip Response, Step 9, since ES-0.2 adds more restrictions on subcooling.

Added information on this change to the Explanation section.

Question determined to be Satisfactory based on Chief Examiner evaluation.

76 H

4 X

N U

E S

NRC:

Distractors C and D are not credible. E-1, Step 5, ensures that either the PORV or its associated Block Valve is closed with RCS pressure < 2350 psia. The applicants are told that RCS pressure is 2200 psia and dropping rapidly with the PORV Open.

In addition, as it pertains to the credibility of Distractor D, why would any SRO applicant reasonably believe that the required action would be to intentionally leave the PORV Open with RCS pressure 2200 psia and dropping, and then transition to ES-1.1, SI Termination?

Separately, question can be answered without any knowledge of, or recognition of the fact, that the event is a Pressurizer Vapor Space Accident.

NOTE: Ensure that any revision to, or replacement of this question, does not overlap with RO Q20, as it describes plant conditions indicative of a Pressurizer Vapor Space Event.

LICENSEE:

Changed the question to improve the plausibility of distractors C and D by changing the question to ask how the crew will isolate the vapor space break through the PORV, rather than ask if they are required to isolate it.

Changed the question to improve the plausibility of the procedure transitions by asking which section of E-1 they will transition to rather than which procedure they will transition to.

Changed the question to ensure the applicant is required to recognize action is required to mitigate a vapor space break through an open PORV.

CHIEF EXAMINER Supplemental Comment (8/11/2023):

Revise Item 2 of the stem to state the following:

Per E-1, the US provides the RO direction to mitigate the event.

LICENSEE Supplemental Response:

Revised stem item 2 to state, Per E-1, the US provides direction to the RO to mitigate the event.

Supplemental LICENSEE Comment:

Made the following changes based on final exam validations (missed by six of six validators) which indicated that the question was borderline LOD 5 as written.

Revised question to clarify the event timeline that the two parts of the question are asking about.

Added stem item 4, describing the current step being performed in E-1.

Changed part two of the question statement and all four answer choices to describe the procedure to be transitioned to, rather than the next step in the current

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level procedure (Based on NRC Branch Chief review comment).

Question determined to be Satisfactory based on Chief Examiner evaluation.

77 H

3 N

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 78 H

3 N

B U

S 2013 MILLSTONE 3 NRC EXAM Reference provided:

Detached Reference (Separate Handout); Tech Spec 3/4.4.4, Relief Valves, Page 3/4 4-12 NRC:

Item 5 of the Stem states:

Engineering reports A PORV is leaking by, but the leakage is considered NOT excessive.

Explanation Section states that the question requires the applicant to apply surveillance requirements to determine if a PORV with seat leakage is considered OPERABLE.

This statement is inaccurate. The applicant is not required to apply SRs to determine whether the PORV is Operable with seat leakage, in answering this question.

The applicant, having been provided the TS Reference (LCO 3.4.4), and told in Item 5 of the stem that the seat leakage is NOT excessive, is easily able to determine:

o per ACTION a, that the PORV is NOT INOPERABLE due to excessive seat leakage, and o that ACTION d, requiring the PORV Control Switch to be placed in the CLOSE position, is therefore applicable and the correct answer.

Part 2 of the Question becomes an exercise in Connecting-the-Dots, and is essentially a Direct Lookup with the TS Reference and Item 5 information.

Explanation Section incorrectly states:

A and C are plausible, since per Action b, if the PORV were considered INOPERABLE, no Action with the Block Valve would be required.

This statement is confusing since, per Actionb, if the PORV were considered INOPERABLE, Action to close the associated Block Valve and Remove Power would be required.

LICENSEE:

Chief Examiner randomly reselected a new K/A due to difficulty writing SRO level question about the effect of RCS pressure change on components.

Replaced the question.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

79 F

2 X

B E

S 2007 MILLSTONE 3 NRC EXAM NRC:

Distractor D is too obvious and can easily be eliminated; i.e., extending the AOT for an EDG to 14 days and concurrently allowing the performance of 3 days of elective maintenance in the switchyard that could challenge offsite power availability? Replace Distractor D.

Is Correct Answer B accurate? Is there a 1-hour requirement for the SBO Diesel Generator to be available in the case of a 14-Day AOT? TS 3.8.1.1, Condition B.4 states:

(Applicable only if the 14 day allowed outage time specified in ACTION Statement b.5 is to be used).

Verify the required Millstone Unit No. 2 diesel generator(s) is/are OPERABLE and the Millstone Unit No. 3 SBO diesel generator is available within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> prior to or after entering this condition, and at least once per 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> thereafter. Restore any inoperable required Millstone Unit No. 2 diesel generator to OPERABLE status and/or Millstone Unit No. 3 SBO diesel generator to available status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or be in at least HOT STANDBY within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and COLD SHUTDOWN within the following 30 hours3.472222e-4 days <br />0.00833 hours <br />4.960317e-5 weeks <br />1.1415e-5 months <br />.

Pedigree Sheet indicates that this question appeared on the 2007 Millstone 3 Exam. Was unable to find the question on the 2007 Exam.

LICENSEE:

Changed distractor D to test knowledge that a contingency plan to supply Charging Pump area ventilation is required.

Verified accuracy of Correct Answer B. ACTION b.4 requires the SBO Diesel to be verified available within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> prior to entering the condition with one EDG inoperable, but verification by test performance is required within 30 days prior to allowing the EDG to be inoperable for greater than 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. Clarified answer B by saying The SBO Diesel has been verified available within the past hour, and was last verified to be available by test performance 15 days ago.

To improve balance and plausibility, re-worded each of the four answer choices to state one condition, followed by the word although, followed by another condition.

Checked pedigree of question and verified it was on the 2007 NRC Exam (Question #79, Question Source listed as New).

CHIEF EXAMINER Supplemental Comment (8/11/2023):

Appears that Distractor C should specify the A Charging Pump and A CCE Pump, based on the associated Plausibility Explanation, i.e., "C" is plausible, since the running Charging Pump is aligned to the operable EDG.

Change the 15 days to 30 days in Correct Answer B.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level LICENSEE Supplemental Response:

Changed Distractor C to specify the A Charging and CCE Pumps have been verified to be in operation.

Per discussion with Chief Examiner on 8/17/23, kept 15 days in Correct Answer B, since the exact number of days in the basis section of Tech Specs is not required from memory.

Supplemental LICENSEE Comment:

Based on final validation after exam submittal, changed correct answer B to say the SBO Diesel was last run by surveillance test performance to clarify what had actually been performed.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 80 H

2 B

E S

2013 MILLSTONE 3 NRC EXAM NRC:

Distractors C and D appear to be technically the same.

Explain how these Distractors are technically different.

Distractor B is awkwardly worded and is different than the version of Distractor B on Q80 of the referenced Millstone 3 2013 NRC Exam, which reads as follows:

The EDG IS still OPERABLE, since loss of a Service Water Pump is covered by the SWP LCO, which has a more restrictive ACTION time than the EDG LCO.

Plausibility Explanation for Distractor D (below) is confusing, as it relates the TRM to the Distractor.

Clarification/Enhancement required.

D is plausible, since this is true for Service Water per the TRM, where it is assumed one SW Pump is supplying Core and CTMT heat loads, while the other Pump is required to supply the Spent Fuel Pool.

LICENSEE:

Changed D to say, per the TRM to make it clearly different from C, which refers to Tech Specs.

Enhanced the Explanation section, adding more detail about why C and D are technically different.

Distractor B is different from the 2013 NRC Exam since it was changed during the validation process in response to an SRO validators comment that pointed out the distractor was not clear as to whether the crew was logging into the SWP LCO.

Added and cascading is not required to distractor B to improve readability.

Enhanced the plausibility description for distractor D in the Explanation section.

Question determined to be Satisfactory based on Chief Examiner evaluation.

81 H

3 M

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 82 H

3 X

N E

S NRC:

Steps 1, 3, and 4 are confusing in terms of what was done when, for example:

STEP 1: The crew completes all actions in EOP 3503 STEP 3: The US directs the required actions to be taken per EOP 3503 to restore RCS subcooling STEP 4: The crew complete all other actions in EOP 3503 If all actions in EOP 3503 were completed per STEP 1, then why is the performance of additional actions in EOP 3503 being directed in STEP 3, and the completion of all other actions in EOP 3503 noted in STEP 4? What other actions (per STEP 4) would be outstanding if all actions were completed per STEP 1? Confusing read.

Explanation for Correct Answer C states, per EOP 3504, Step 6, that the crew will be directed to align a head vent path to the PRT. The head vent path will be aligned to Excess Letdown in this question.

In addition, the version of EOP 3504 provided to the NRC, shows that the step referenced should be Step 4 instead of Step 6. Confirm which step of EOP 3504 is correct.

LICENSEE:

Clarified Stem Steps 1-5, to clarify that the crew completed all actions in the Control Room prior to evacuating the Control Room, and which step in EOP 3503 directed the actions to address the low subcooling.

Changed the justification for correct answer C in the Explanation section to say the crew was directed to align head vent excess letdown.

Confirmed correct revision of EOP 3504 is Rev. 9-2, and corrected the step number referenced in the Explanation for correct answer C to EOP 3504, step 4.

Question determined to be Satisfactory based on Chief Examiner evaluation.

83 H

3 B

S

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 84 H

3 N

E S

NRC:

Item 4 of the stem states The crew isolates the C SG per E-2, Faulted Steam Generator Isolation.

Should Item 4 instead read The crew confirms isolation of the C SG per E-2, Faulted Steam Generator Isolation, given that the steps to successfully isolate the SG were previously performed in E-3?

LICENSEE:

Discussed with the Chief Examiner on 7/28/23. No change was made to item 4 of the question, since the step in E-2 that confirms faulted SG isolation is titled, Isolate Each Faulted SG.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 85 H

3 B

E S

NRC Early Review Question 2000 MILLSTONE 3 NRC EXAM NRC:

Recommend replacing Functional Response Procedure with Functional Restoration Procedure, as referenced in the Answer Explanation section.

LICENSEE:

Changed Functional Response Procedure to Functional Restoration Procedure.

CHIEF EXAMINER Supplemental Comment (7/17/2023):

Question: Would not Distractors A and B essentially be the same step/answer given that the depressurization is currently in progress?

Unable to confirm that the Last NRC Exam was the 2002 Millstone 3 Initial License Exam, as identified on the Pedigree Sheet. Ensure the accuracy of this information.

LICENSEE Supplemental Response:

Differentiated Distractor A from Distractor B by having A suspend the current SG depressurization step, and B complete the current SG depressurization step.

Changed the Explanation section based on the changes to Distractors A and B.

Located Question on the Millstone 3, 2000 NRC Exam, Question #15. Changed pedigree statement to reference the 2000 NRC Exam, rather than the 2002 NRC Exam.

CHIEF EXAMINER Supplemental Comment (8/11/2023):

Suspect that the 3rd bullet in the stem should read The crew has just commenced depressurizing The word depressurizing appears to be missing.

LICENSEE Supplemental Response:

Added the word depressurizing to the 3rd bullet in the stem.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 86 H

2 B

E S

2011 MILLSTONE 3 NRC EXAM)

NRC:

Should Correct Answer C require the RO to adjust 3CHS*FCVI21 instead of 3RCS-LK459 to decrease charging flow? The Correct Answer for previously used question Q94 on the 2011 NRC Millstone 3 Exam states:

The US will direct the RO to adjust Charging Flow Control Valve 3CHS*FCVI21 to decrease charging flow.

How would adjusting Letdown Isolation Valve 3RCS-LK459 decrease Charging Flow? Appears that adjustment of this valve would only affect Letdown Flowrate.

For Distractors A and B, recommend using the words:

with T across the Pressurizer Spray nozzle in excess of TRM limits from Q94 on the 2011 Exam, instead of with excessive delta T across the Pressurizer Spray nozzle.

LICENSEE:

Changed C to say, Charging Flow Control Valve 3CHS*FCVI21 to decrease charging flow.

Adjusting the letdown controller would not affect Charging flow, but it would change letdown flow, and this is an allowed procedural way to decrease overall mass addition rate to the RCS and combat the Pzr insurge event.

Clarified this in the Explanation section.

For distractors A and B, changed wording to say, with T across the Pressurizer Spray nozzle in excess of TRM limits Supplemental LICENSEE response:

Deleted redundant decrease charging flow from correct answer C due to inadvertently duplicating the wording a second time during last update to the question (NR Branch Chief review comment).

Question determined to be Satisfactory based on Chief Examiner evaluation.

87 H

2 X

N E

S NRC:

Part 2 of Distractors A and C does not appear to be credible. Why would an SRO applicant reasonably believe that the correct answer would be to rack down and then rack up the failed RPCCW Pump Breaker? This would require the applicants to make an assumption that the A RPCCW Pump Breaker is not faulty (i.e., not the cause of the pump trip).

NUREG-1021, ES-1.2, Paragraph B.8, states:

When answering a question, do not make assumptions about conditions that are not specified in the question unless they occur as a consequence of other conditions that are stated in the question. For example, you should not assume that any alarm has activated unless the question so states or the alarm is expected to activate as a result of the conditions that are stated in the question. Similarly, you should assume that no operator actions have been taken, unless the stem of the question or one of the answer choices specifically state otherwise.

Explanation Section references the incorrect Train in the following statement (should be Train A):

A and B are wrong, since the crew will be required to use Attachment A to align the C (swing) RPCCW Pump to the B Train.

LICENSEE:

Changed Part 2 of Distractors A and C to use a breaker dedicated to the C pump on the affected train. This is a misapplication related to the swing Charging Pump, which uses a breaker from the affected train.

Changed the Explanation Section for the description of why Distractors A and B are wrong to reference the correct A Train.

Made editorial enhancements to the question statement and all of the distractors to grammatically accommodate new distractors A and C.

Changed the Explanation Section to accommodate the above changes.

CHIEF EXAMINER Supplemental Comment (8/11/2023):

Changes made to Part 2 of Distractors A and C have resulted in Distractors that do not follow grammatically in Part 2 of the Answer Statement. This can potentially cue the applicant that these Distractors are incorrect. Part 2 of the Answer Statement now reads as follows for Distractors A and C:

The electrical alignment requires the C RPCCW Pump breaker to be verified to already be in its cubicle on Bus 34C, and then racked up at Bus 34C.

LICENSEE Supplemental Response:

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level Changed the second part in the Question Statements to say, The electrical alignment requires the crew to (2) its cubicle at Bus 34C. The crew will then rack the breaker up at Bus 34C.

Changed the grammar in distractors A and C to say, verify the C Pump breaker is already in Changed the grammar in correct answer D and distractor B to say, rack down the C Pump breaker at its cubicle at Bus 34D and move it to Question determined to be Satisfactory based on Chief Examiner evaluation.

88 H

3 M

E S

NRC:

Distractor A does not appear to be credible. How would it be remotely plausible to verify the adequacy of Sump Recirc Flowrate with one Charging Pump in ES-1, when Cold Leg Recirculation has yet to established?

Modification justification on Pedigree Sheet states:

Distractor A has been changed from aligning a hot leg injection path to verifying adequate sump flow through the running Charging Pump.

While Distractor A has been changed to verifying adequate sump flow through the running Charging Pump, the Parent Question version of Distractor A does not specify aligning a hot leg injection path. Ensure the accuracy of this information.

LICENSEE:

Modified Distractor A to say, after initial sump recirc alignment is complete to clarify the verification is to occur after the recirc alignment has been completed.

Changed the plausibility statements for Distractor A in the Explanation section to match new Distractor A.

Changed modification justification to say, Distractor A has been changed from verify two Charging Pumps are running to verify adequate sump recirc flowrate with one Charging Pump running.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 89 H

3 N

E S

NRC:

Second sentence of the Explanation Section incorrectly states:

Per LCO 3.8.3.1, ACTION b(2), there is a 2-hour requirement to energize a deenergized VIAC.

Should be LCO 3.8.3.1, ACTION b(1).

LICENSEE:

Changed second sentence of the Explanation section to say LCO 3.8.3.1, ACTION b(1)

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 90 F

H 1

3 X

X N

B U

S 2011 MILLSTONE 3 NRC EXAM)

Reference provided:

Detached Reference (Separate Handout); Tech Spec LCO 3.7.1.1, with Tables NRC:

LOD=1. It is common knowledge for both RO and SRO applicants that the function of the RHR Suction Relief Valves is to protect the Integrity of the RCS pressure boundary (Item 4) because the low pressure piping of the RHR System is not rated for RCS pressure. Consequently, Distractors B, C, and D can be easily eliminated by all license-level applicants.

K/A Mismatch flaw. Is there a Secondary Containment System in Westinghouse PWRs? The K/As identified for Safety Function (SF) 5, Containment Integrity, in a Westinghouse PWR, are intended to test applicant knowledge of the Primary Containment Systems. There are no K/As in the SF5 section of the K/A Catalog that support Secondary Containment Systems. Note that: a)

Item 1 in the stem is apparently associated with Secondary Containment, and b) the Question Statement is written to encompass both Primary and Secondary Containment Systems.

Suggest eliminating the word Breaker from Item 3 to avoid any potential ambiguity. Should read the same as the TRM, i.e., The Reactor Coolant Pump Motor Power Feeder Overcurrent Protective Device.

The bulleted Explanation Statement under Correct Answer A, pertaining to the basis for verifying the Containment Recirc Pump Casing is full of RWST water, appears to be a Cut-and-Paste error. None of the answer choices are associated with this information.

LICENSEE:

Chief Examiner randomly reselected a new K/A due to overlap with the operating exam.

Replaced the question.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 91 H

4 3

M B

E S

NRC:

Editorial - Correct Answer D contains the word ACTION after LCO 3.0.3. Remove the word ACTION.

LICENSEE:

Removed the word ACTION since Correct Answer D is entering Section 3.0 LCO 3.0.3, which has only one LCO ACTION Supplemental LICENSEE Comment:

Replaced the question based on post exam submittal validation results, given that all six SRO validators missed the question during the exam validation process, indicating that the question was borderline LOD 5.

Replacement Question determined to be Satisfactory based on Chief Examiner evaluation.

92 H

3 N

U E

S NRC Early Review Question NRC:

If OP 3353.MB2B, 3-9, Rad Hi, directs the crew to AOP 3573, then it would appear that Distractor A and Correct Answer B would both be correct, given that direction to place the Degasifier in Standby is accomplished in AOP 3573.

LICENSEE:

Changed the question statement to have the applicant identify the specific procedure that actually contains the direction to place the Degasifier in Standby, CHIEF EXAMINER Supplemental Comment (7/18/2023)

Revise the Question Statement to read as follows:

The _(1)_ closed; and the US will direct the crew to place the Degasifier in Standby using __(2)___.

Supplemental LICENSEE Response:

Revised the Question Statement to read as follows:

The (1) closed; and the US will direct the crew to place the Degasifier in Standby using (2).

Question determined to be Satisfactory based on Chief Examiner evaluation.

93 H

3 N

S Reference provided:

Detached Reference (Separate Handout);

TRM 3.7.12.2, pg. 3/4 7-21 and 22, Fire Suppression Systems; Spray and/or Sprinkler Systems

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 94 H

2 N

S 95 H

3 B

S 96 F

2 B

S 2019 MILLSTONE 3 NRC EXAM (Previous 2 NRC Exams) 97 F

1 2

X X

N U

E S

NRC:

LOD=1. Discrimination Validity of this question is low.

Distractor A is the obvious choice, especially given the nature of the Distractors. Replace Correct Answer A with a more discriminating reason for rejecting the Work Order.

Distractors C and D do not appear to be credible. Why would a Work Order be rejected for the conduct of Minor Maintenance or Urgent Work? How else would the work for these items be accomplished if not done under the Work Order process?

LICENSEE:

Replaced correct answer A.

• Replaced distractors C and D.

• Per Chief Examiner comment (phone call, 7/28/23),

removed the word reject to in the question statement.

Changed the statement to The work can be issued without requiring a Work Order...

CHIEF EXAMINER Supplemental Comment (8/11/2023):

Credibility associated with Distractor D, involves a Reactivity Management issue, is Low. Replace this Distractor.

LICENSEE Supplemental Response:

• Per discussion with Chief Examiner on 8/17/23, Changed Distractor D back to, is classified as Minor Maintenance (and added quotation marks to correct answer A Tool Pouch to match distractor D), since this distractor has higher plausibility than a Reactivity Management issue.

• Changed Justification Section for Distractor D to match tis change.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 98 H

3 N

E S

NRC:

Rename the answer choices as follows:

o Turbine Building Stack Monitor increasing trend o

ESF Building Ventilation Exhaust Monitor increasing trend o

Liquid Radwaste Effluent Monitor in ALERT o

D Steam Generator Tube Leak Remove the word abnormal from the Question Statement.

CE Comment: Question is a precise exact K/A Match.

Question written in a way that complies with Tier 3 criteria while still meeting the intent of the K/A.

LICENSEE:

Renamed the answer choices as follows:

o Turbine Building Stack Monitor increasing trend o

ESF Building Ventilation Exhaust Monitor increasing trend o

Liquid Waste Effluent Monitor in ALERT o

The D Steam Generator Tube leak Removed the word abnormal from the Question Statement.

Question determined to be Satisfactory based on Chief Examiner evaluation.

99 H

F 2

N B

U S

NRC Early Review Question 2015 MILLSTONE 3 NRC EXAM NRC:

Question does not meet the criteria for Tier 3. ES-4.2, Paragraph B.2.c states:

Ensure that Tier 3 questions focus on plant-wide generic K/As and not system-specific knowledge or knowledge of specific emergency / abnormal plant evolutions.

Question, as written, requires specific knowledge of Functional Restoration Procedures (FR-P.2 & FR-S.2).

If unable to develop a discriminating SRO level Tier 3 question for the assigned K/A without expending an inordinate amount of time and resources, it may be necessary to substitute another randomly selected K/A.

LICENSEE:

Unable to develop an operationally valid and discriminating Tier 3 question at the SRO License level to test the selected K/A (G2.4.22).

CHIEF EXAMINER randomly re-selected Generic K/A 2.4.16 for Q99 from the Emergency Procedures/Plan portion of Generic K/A Section 2.

CHIEF EXAMINER Supplemental Comment for Replacement Question (7/18/2023):

2015 MILLSTONE 3 NRC EXAM Plausibility of delaying implementation of E-0 following a Reactor Trip, for the sole purpose of completing an in-progress AOP step, is extremely low. Revise Distractors A and D to begin with After completing all Immediate Actions, In addition, for Distractor A, replace RCS temperature Control with RCS Pressure Control, to raise the overall Discrimination Validity.

LICENSEE:

Changed Distractors A and D to begin with, After completing all immediate actions,.

Made additional changes to distractors A and D, since the change in the first bullet was made, they could now be considered correct if they didnt delay implementation of ES-0.1.

Kept RCS temperature control in Distractor A, since there isnt a step in AOP 3563 that takes actions to restore RCS pressure control.

Supplemental LICENSEE response:

Based on final validation after submitting exam, added After exiting E-0 to Answer C, since the other three choices include the status of E-0. Validator was not sure if this answer choice implied the crew directly entered ES-0.1 without entering E-0.

Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level Question determined to be Satisfactory based on Chief Examiner evaluation.

100 F

2 X

N E

S NRC:

Replace Correct Answer A, Provide current plant status to the ADTS, with Recommend corrective actions to the ADTS, to raise the overall Discrimination Validity.

Remove the second now from the Question Statement such that it reads:

Now that the turnover with the EOF-DSEO is complete, which of the following is a responsibility of the MCRO per EPI-FAP01, Control Room Emergency Operations?

Concern is the potential for inadvertently cueing the applicants.

Provide the names of the acronyms in the Explanation Section.

LICENSEE:

Replaced correct answer A with Recommending corrective actions to the ADTS.

Removed the second now from the question statement.

Added ing to each of the Distractor action verbs, to better match the grammar of the question statement, e.g.,

changed Recommend to Recommending.

Provided the names for the acronyms used in the question in the Explanation section.

Question determined to be Satisfactory based on Chief Examiner evaluation.

Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1) Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2) Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3) Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4) Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5) Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6) Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7) Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8) Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9) Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.