ML24120A192

Ile Form 2.3-5 Written Examination Review Worksheet

ML24120A192
Person / Time
Site:	Point Beach   (DPR-024, DPR-027)
Issue date:	04/05/2024
From:	NRC/RGN-III/DORS/OB
To:	NextEra Energy Duane Arnold
B. Bergeon
References
Download: ML24120A192

Text

Form 2.3-5 Written Examination Review Worksheet

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 1

H F

H 3

X B

U S

2019 Callaway NRC:

1)

Providing the -1/3 DPM in the stem is cueing, as this is essential knowledge as to understanding the subsequent decay in reactor power. Bank question did not include this information in the stem.

2)

Question is low-cog - applicant simply needs to recall when SR detectors energize and determine the amount of time required for those to do so, based on decay.

3)

Change CFR 55 reference to 55.41(6) and list only one.

Licensee:

1)

Removed the -1/3 DPM decay from the note and state something along the lines of Assume reactivity affects other than decay are negligible (check terminology)

2)

Changes listed above upgraded this to a high cog question.

3)

Done NRC:

1)

Question change to high cog due to changes made. Question is sat as revised.

2 F

3 B

E 2011 Millstone 3 NRC:

1)

Change distractor C from drain the pressurizer to 80% using letdown to lower pressurizer level to 80% using letdown.

2)

Change CFR 55 reference to 55.41(10) and list only one.

Licensee:

1)

Done

2)

Done NRC:

1)

Question is sat as revised.

3 H

3 M

E 2017 Callaway NRC:

1)

Answer justification for answer B states insufficient RCS inventory to support natural circulation and therefore there is no natural circulation in progress. However, initial conditions in the stem support natural circ per EOP-1.2 Att 1.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

2)

Reference what subcooling actually is in the answer justification and show how each of the Att 1 criteria is met.

3)

Change CFR 55 reference to 55.41(5) and list only one Licensee:

1)

Answer justification has been updated. The stem numbers have been altered slightly (610-580F).

2)

Done

3)

Done NRC:

1)

Question is sat as revised.

4 H

3 N

S NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

Done 5

H 2

B E

S Free Look 2012 Point Beach NRC:

1)

Could not find this question used on the 2012 PB exam (or any NRC exam at PB).

Is Bank correct?

Licensee:

1)

Question is from the 2009 Harris Exam NRC:

1)

How would the indicated conditions be correct for LT-428 failing high in distractor D, but not for LT-427?

2)

Is the initial condition providing a report that the charging pump relief valves lifting necessary? Can we provide an indication or an alarm providing that information (for the applicant to make a determination rather than provide the cue)?

3)

Change CFR 55 reference to 55.41(8) and list only one Licensee:

1)

LT-428 (Blue) is normally the controlling channel. This is the normal configuration. If the Pzr Level Defeat switch is set to Defeat 3 it will defeat the channel but this is done in an event that there is a malfunction is no

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level the norm. This has been added to the justification.

2)

Procedurally the two ways to determine this is VCT level and local indication where an operator would be dispatched. In this case, it would be expected that an operator would provide this indication.

3)

Done NRC:

1)

Question did not require any revisions and is sat as submitted.

6 F

3 B

S 2012 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

Done 7

H 3

X M

E S

2003 Point Beach NRC:

1)

Part 1, while not necessarily cueing, can imply that AOP-24 must be used if the operator is going to take the actions. If AOP-24 could not be used, the operator would take actions IAW another procedure that isnt listed. The stem wording can imply cueing. It would appear that asking what procedure would mitigate the transient in part 1 (and having another procedure and AOP-24) and then leaving part 2 as is, would be a cleaner look to the question.

2)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

While there are other procedures operators can enter, this procedure is the only one that allows actions from memory per OM 3.7. Justification was discussed.

2)

Done NRC:

1)

Based on discussion with licensee surrounding OM 3.7 from memory actions, this question was sat as submitted.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 8

H 2

X B

E NRC:

1)

While not part of the free look questions, this was submitted due to potential overlap with Q9. While the overlap is not a concern, the question is SRO knowledge.

Knowing entry conditions into an EOP is RO knowledge. Assessing plant conditions and selecting a procedure to mitigate the transient is SRO knowledge. Testing EOP-3 entry conditions or overall mitigative strategy would be RO knowledge.

Recommend rewording transitions to to enter

2)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

Swapped the words transitions to to enter. Change was discussed prior to submittal.

2)

Done NRC:

1)

Question is sat as revised.

9 H

2 X

X N

U E

Free Look NRC:

1)

While knowing entry conditions into an EOP is RO knowledge. Assessing plant conditions and selecting a procedure to mitigate the transient is SRO knowledge.

Testing the EOP-2 entry conditions or overall mitigation strategy would be RO knowledge.

2)

K/A match is questioned - how is the ability to identify & interpret diverse indications to validate the response of another indicator met by assessing plant conditions and determining what procedure to transition to?

Licensee:

1)

Due to the requirements of the original KA a valid RO Tier 1 question is not available due to lack of discernment required when following the procedure. Due to this and various other factors, a new KA and question has been developed.

2)

See above.

NRC:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Typo in answer A and distractor B -

generator.

2)

Errant space in distractor C between RCPs and continue

3)

Question stem has (1) and (2) for the two part question, but answer/distractors do not. Add (1) and (2) to answers/distractors.

See Q8

4)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

Done

2)

Done

3)

Done

4)

Done NRC:

1)

Question is sat as revised 10 H

2 N

E NRC:

1)

Use same terminology for all answer choices - shut or close.

2)

Remove CFR55.45(7) reference Licensee:

1)

Done, changed to shut

2)

Done NRC:

1)

Question is sat as revised.

11 H

3 B

S NRC:

1)

Remove CFR55.45(5) & (6) references.

Licensee:

1)

Done 12 H

3 X

B N

U E

Free Look 2012 Point Beach NRC:

1)

Questionable K/A match to tier 1. While the stem puts the applicant in AOP-0.1, the question is asking for a system response based on a LOOP. Simply knowing that 13.8kV breakers have no UV trip protection is sufficient to answer the question. Very different than Q5 tier 1 match. Need to better tie this to AOP/EOP guidance/actions.

Licensee:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Question was removed and a new question was added.

NRC:

1)

Question stem has (1) and (2) for the two part question, but answer/distractors do not. Add (1) and (2) to answers/distractors.

See Q8

2)

Add CFR 55 reference 55.41(10)

3)

Add to answer justification that EDGs would start and load the respective ESF busses - this would help support ECA-0.0 being incorrect due to the ESF busses remaining energized.

Licensee:

1)

Done

2)

Done

3)

Done NRC:

1)

Replacement question is sat as revised.

13 H

3 X

N U

S NRC:

1)

Part 2 - manually divert or makeup to maintain VCT level is very generic. While that is the procedural direction, with rising VCT level due to constant auto makeup, the applicant would divert to a HUT away from the VCT and secure auto MU. Part 2 can be more pointed.

2)

From OR-01 page 38, it states At 56%

Divert valve begins modulating to maintain level at 56% on LT-112 failing low. Since AOP-1D states manually divert or makeup to maintain VCT level, this answer could be partially true. The applicant could secure auto MU and allow LT-141 to divert 1CV-112A automatically. Then, this part 2 would be partially correct.

3)

Remove CFR55.43(5) & 45(13) references and add 41(7)

Licensee:

1)

Changed the (2) distractor to manually maintain VCT level

2)

While the valve does modulate open, it does not keep up with the inputs including makeup. Water level will continue to rise slowly past 56%. Manual action is required to maintain level.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

3)

Done NRC:

1)

Question is sat as revised.

14 H

3 B

E NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one

2) 1A52-57 noun name can be added in italics below the stem and removed from distractor C, for consistency.

3)

AOP-0.0 Att A step 3 says to close contactor 1B52-391 for D-07, however, the question answer/distractors all reference 1B42-391. No 1B42 contactor exists in AOP-0.0.

Licensee:

1)

Done

2)

Done

3) 1B52-391 supplies power to 1B42-391 which is the D-07 Battery charger contactor. The step in A3 is rather vague but it requires the operator to close the contactor (1B42-391) and if that does not work, ensure the supply breaker is closed (1B52-391). The wiring diagram and procedure step is now referenced in the justification.

NRC:

1)

Question is sat as revised.

15 H

3 M

B S

2007 Point Beach NRC:

1)

Remove CFR 55.45(13) reference

2)

Question is not modified from original bank question.

Licensee:

1)

Done

2)

Done. This question was not contributing to the modified/new question numbers. NO impact on metrics.

16 H

3 X

X B

E S

2019 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one

2)

How is part 1 of cycling RHR pumps a plausible distractor? It would be more

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level plausible to manually cycle RH heat exchanger bypass valve or similar.

3)

Cannot find in AOP-5B where it directs controlling AFW flow via the TDAFW pump discharge valves. Question stem asks IAW AOP-5B - if the procedure doesnt direct it, the answer is not correct. Validate AOP-5B compliance.

Licensee:

1)

Done

2)

As listed, RHR could be used for DHR but it is not procedurally allowed. Furthermore, there are instances when intermittently operating the RHR pumps would be required for runout concerns within the procedure.

3)

The foldout page states Monitor and maintain minimum AFW discharge flowP-29 minimum flow - GREATER THAN 75 GPM.

NRC:

1)

Based on licensee discussion, question is sat as submitted.

17 H

3 M

S 2011 South Texas Project NRC:

1)

Change CFR 55 reference to 55.41(7) and list only one Licensee:

1)

Done 18 F

2 B

E S

NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one

2)

Consider changing pump cubicle drain valve to the noun name referenced in ECA-1.2: RHR pump drain to sump control valves Licensee:

1)

Done

2)

This is the term used in 1C20 A 4-4. IT 45 also refers to them collectively as the cubicle drain valves before going on to list the valve number and nomenclature. This has been validated as a term used by operators.

NRC:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Based on discussion with the licensee, question is sat as submitted.

19 H

2 X

B U

S Reference provided 2021 Point Beach (previous 2 exams)

NRC:

1)

Overlap with scenario 2, event 7. Scenario event is testing emergency boration for 4 stuck rods.

2)

K/A mismatch - question is asking EOP-0.1 actions for emergency borating for stuck rods. K/A requires using RPI to determine inop/stuck rods. While this is done to determine two rods are stuck in the stem of the question, the question is asking boration requirements for stuck rods (K/A AK3.01).

3)

Change CFR 55 reference to 55.41(10) and list only one NRC:

1)

Question has been replaced and discussed.

NRC:

1)

Question is sat as revised.

20 H

3 B

E 2009 Point Beach NRC:

1)

Question stem should reference AOP-3 since the plant is in a tube leak. AOP-3 can be referenced to justify that 1RE-231 is used to identify the affected SG (step 4) -

tie to tier 1.

Licensee:

1)

Added and the crew has entered AOP-3 to the second bullet.

NRC:

1)

Question is sat as revised.

21 F

3 X

B U

S 2019 Point Beach NRC:

1)

Remove CFR 55 reference other than 55.41(10)

2)

K/A mismatch - K/A is knowledge of the process for controlling equipment configuration or status" with regards to a plant fire. The question is asking RCS level and temp control during a plant fire.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Licensee:

1)

Done

2)

The process for controlling plant status and configuration when abandoning the control room due to a fire is AOP-40A, Control Room Abandonment Due to Fire and the actions therein. Matches the K/A by requiring the operator to determine PZR level once the control room has be evacuated and recalling actions to maintain subcooling and PZR level. At a tier 1 level the operators use procedures as part of the process for combating casualties such as fires. Operators must perform the procedure steps to control the configuration and status of the plant.

NRC:

1)

Agree with licensees response to #2, question is sat as submitted..

22 H

3 B

S 2021 Point Beach (previous 2 exams)

NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

Done 23 F

3 N

S NRC:

1)

Change CFR 55 reference to 55.41(7) and list only one Licensee:

1)

Done 24 F

3 X

B E

2017 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one

2)

While the MSIVs are shut in step 3, which makes the steam dumps nonfunctional, step 4 of BG AOP-10 states the controllers are set for this value [1005 psig] to maintain the RCS at no load Tavg. It would appear that distractor D is partially true based on the background document.

Licensee:

1)

Done

2)

Discussed changing 1005 psig to 1050 psig (the normal position) making the distractor

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level more incorrect. Change has been implemented.

NRC:

1)

Question is sat as revised.

25 H

3 N

S NRC:

1)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

Done 26 F

2 B

E 2019 Callaway NRC:

1)

Stating No RCPs are running after stating that a natural circ cooldown is in progress due to a LOOP, is unnecessarily redundant.

2)

Change CFR 55 reference to 55.41(10) and list only one Licensee:

1)

Bullet removed

2)

Done NRC:

1)

Question is sat as revised.

27 F

2 B

E 2003 Point Beach NRC:

1)

Answer/distractor justifications do not specify which one is A, B, C or D.

2)

Remove CFR 55 references except 55.41(7)

Licensee:

1)

Added A-D designators.

2)

Done 28 H

3 B

S 2007 Point Beach NRC:

1)

Remove CFR 55 references except 55.41(5)

Licensee:

1)

Done 29 H

3 M

B S

2003 Point Beach NRC:

1)

Question is not modified sufficiently to be considered modified. Only change was a loss of IA rather than only 1CV-110A failure

- Answer & distractors remain the same.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

2)

Remove CFR 55 references except 55.41(6)

Licensee:

1)

This question was credited as a modified question and impacts the new/modified count. Currently this is fine.

2)

Done 30 F

2 B

S NRC:

1)

Change CFR 55 reference to 55.41(5) and list only one

2)

Is there anything that defines refueling height for cavity water level?

Licensee:

1)

Done

2)

TS 3.9.6: Cavity level shall be maintained greater than or equal to 23 ft above the top of reactor vessel flange. This reference has been added to the reference list.

31 H

3 B

E NRC:

1)

Remove CFR 55 references except 55.41(5)

2)

Stem states that OS is directing you (the applicant), however, the answer/distractors state the CO. Need consistency in who is performing the action.

3)

Consistency in currently is and is currently in choices B & C Licensee:

1)

Done

2)

Removed you from the 4th bullet and added the CO

3)

Changed C and D to is currently lower and is currently higher respectively.

NRC:

1)

Question is sat as revised.

32 H

2 B

S 2011 Diablo Canyon NRC:

1)

Remove CFR 55 references except 55.41(3)

Licensee:

1)

Done 33 H

3 X

N U

S NRC:

1)

Tier 4 fundamentals question. This question can be answered without any

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level system level knowledge. Question does meet the K/A, but not the tier. Could attempt to meet the tie by incorporating surge tank level and auto makeup actions, for example.

2)

P&ID referenced does not support answer justification as to why indicated level would lower if the variable leg were to leak.

Fundamentals lesson plan? System lesson plan?

Licensee:

1)

Modified this question to be a 2x2 question.

The first part is a question of what indicated level would do. The second part is what the alarm setpoint is for the surge tank

2)

General Fundamentals lesson plan NRC:

1)

Question is sat as revised.

34 H

3 B

E NRC:

1)

Add complete valve numbers for PORVs in choices A & B (as done in D).

2)

Remove CFR 55 references except 55.41(5)

Licensee:

1)

Done

2)

Done NRC:

1)

Question is sat as revised.

35 F

2 B

E 2013 Callaway NRC:

1)

Remove CFR 55 references except 55.41(5)

2)

Choices A & B assume that the applicant entered the procedure (remain in..), which is not given. Making all choices enter the appropriate procedure would be more consistent Licensee:

1)

Done

2)

Added enter and remain in. We dont want them to transition out of those procedures in the distractor NRC:

1)

Question is sat as revised.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 36 H

3 B

S 2015 Point Beach NRC:

1)

Remove CFR 55 references except 5.41(6)

Licensee:

1)

Done 37 F

2 B

S 2012 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(6) and list only one Licensee:

1)

Done 38 H

3 B

S Free Look 2012 Point Beach NRC:

1)

Remove CFR 55 references except 5.41(7)

Licensee:

1)

Done 39 H

3 M

S Reference provided (embedded) 2019 Point Beach NRC:

1)

Remove CFR 55 references except 5.41(7)

Licensee:

1)

Done 40 F

2 B

E NRC:

1)

Remove CFR 55 references except 5.41(7)

2)

Change has manually actuated in the stem to was manually actuated.

3)

Discrepancy in noun name for SI-860C/D between Q40 and Q41. Which is correct?

Licensee:

1)

Done

2)

Done

3) 40 used a more generalized term. Pulled in the nomenclature from 41 as its more precise.

NRC:

1)

Question is sat as revised.

41 H

3 N

S NRC:

1)

Remove CFR 55 references except 5.41(7)

2)

Discrepancy in noun name for SI-860C/D between Q40 and Q41.

Licensee:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Done

2)

Discrepancy has been resolved. See Q40.

42 H

3 X

B U

S Reference provided (embedded) 2017 Point Beach NRC:

1)

Remove CFR 55 references except 5.41(5)

2)

How is the (b) part of the K/A met in this question? How are procedures used to correct, control, or mitigate the transient?

The (a) part of the K/A is met, but the (b) part must be met, at a minimum. The operator action portion of the answer/distractors are given and dont require any procedural knowledge of how to mitigate the transient (ex requires operator actions to close for both C & D).

Question is a good question, but not for an A2 K/A.

Licensee:

1)

Done

2)

KA must be approached at a tier 2 level without upgrading to a tier 1. Question answer/distractors require the examinee to know what action to control/mitigate the issue would be taken without upgrading to tier 1 by entering procedures etc. Question asks what big picture action would be taken to control and mitigate the issue.

NRC:

1)

Based on discussion with the licensee, the question is sat as submitted. Requiring specific procedure knowledge would make this a tier 1, not a tier 2 question as required. The (b) part of the K/A is met as the applicant must determine what the system impact was (part a), but also requires overall procedure mitigative strategy (see distractor A & B).

43 H

2 B

E NRC:

1)

Remove CFR 55 references except 5.41(7)

2)

Recommend new distractor for either A or D, as both can be eliminated simply by knowing suction pressure (given in both distractors) is unaffected.

Licensee:

1)

Done

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

2)

Changed D to "Turbine Runback will automatically occur on Low Steam Generator Level.

NRC:

1)

Question is sat as revised.

44 H

3 X

B U

S 2019 Point Beach NRC:

1)

K/A mismatch - K/A is knowledge of electrical power supplies for AFW power-operated valves. Question is asking power supplies to AFW pumps (K/A K2.02).

Question would be sat for K/A K2.02, but does not match K/A K2.01.

Licensee:

1)

Question has been replaced and discussed.

NRC:

1)

Question is sat as replaced.

45 H

3 M

E Reference provided 2023 Point Beach NRC:

1)

Remove CFR 55 references except 5.41(5)

2)

Recommend swapping P-32C in distractor C with another piece of equipment, as the pump alone will exceed the 2000 hr load limit on the EDG given the current load.

This eliminates A & C without even needing to validate the other piece of equipment listed.

Licensee:

1)

Done

2)

Swapped P-32C with 1P-14B at 183 KW which gives a combined total higher than allowed and no single component exceeding the value for the C answer.

NRC:

1)

Question is sat as revised.

46 F

3 B

S NRC:

1)

Change CFR 55 reference to 55.41(7) and list only one Licensee:

1)

Done

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 47 H

3 X

B U

S 2012 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(8) and list only one

2)

K/A mismatch - applicant needs to know nothing about battery discharge rate.

Applicant simply needs to know alarm at TS voltages to answer the question. If they know 105V is the TS min and 127 is the alarm setpoint, part 2 is a given, leaving the applicant to need to know nothing about battery discharge rate to answer the question; thereby not meeting the K/A of monitoring battery discharge rate from the MCR (or locally, since battery voltage is not present in the MCR).

Licensee:

1)

Done

2)

This question has been discussed and rated as sat.

NRC:

1)

Based on discussion with licensee, this question is sat as submitted. While not an ideal K/A match, the question does require the applicant understand battery discharge and key battery voltage values.

48 H

3 B

S 2017 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(8) and list only one Licensee:

1)

Done 49 F

3 N

S 50 F

3 B

E Reference provided (embedded) 2017 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(7) and list only one

2)

In answer C, is this true for an SI on EITHER unit?

3)

Recommend changing distractor D wording from Unit 2 Safety Injection to Unit 2 SI signal to match B & C wording.

Licensee:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Done

2)

Yes, Safeguard Sequence Logic 883D195 SH. 8 reflects this.

3)

Done NRC:

1)

Question is sat as revised.

51 F

2 B

M B

S 2021 Point Beach (previous 2 exams)

NRC:

1)

Change CFR 55 reference to 55.41(4) and list only one

2)

Question is sufficiently modified from 2021 exam.

Licensee:

1)

Question is a bank question. Old question that was attached was an earlier revision.

This has been updated.

2)

See above NRC:

1)

Question is a bank question. Sat as submitted.

52 F

2 B

E NRC:

1)

Remove CFR 55 references except 5.41(7)

2)

Answer C is confusing - manually open while locally gagging open IA-3047. Why does this read differently than distractor D locally gag open..? Will they have to hold the valve C/S open while the valve is gagged?

Licensee:

1)

Done

2)

Yes. Valve receives two signals and since we cannot reset CI for 1 of those trains, the valve we receive a constant shut signal.

Distractor C and D were swapped to prevent a high number of identical answers being in a row.

NRC:

1)

Question is sat as revised.

53 F

3 B

S Free Look NRC:

1)

Remove CFR 55 references except 5.41(4)

Licensee:

1)

Done

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 54 H

H 2

3 X

B E

NRC:

1)

Change CFR 55 reference to 55.41(5) and list only one

2)

Overlap with Q9. In both questions, applicants have to diagnose primary plant conditions to determine a faulted SG inside containment. Q9 was a free look question and overlap not able to be determined at that time. Question is sat and meets the K/A, but must be modified due to overlap.

Marked as a cue flaw, due to overlap.

Licensee:

1)

Replaced with a question from 2012 Point Beach and discussed.

2)

See above.

NRC:

1)

Question is sat as replaced.

55 F

2 B

S 2013 ANO1 NRC:

1)

Remove CFR 55 references except 5.41(3)

Licensee:

1)

Done 56 H

H 3

3 X

M B

M U

S NRC:

1)

Remove CFR 55 references except 5.41(7)

2)

Question should be bank since theres minimal modification to stem &

answer/distractors.

3)

Can eliminate is and are from part 1 answer/distractors.

4)

K/A mismatch - question is not testing the knowledge of a loss or malfunction of NIS will have on the fuel handling system.

Question is asking operability requirements of NIS (source range) during refueling.

Licensee:

1)

This question has been replaced with a modified version of 2017 NRC Exam Q54 and determined to be SAT.

NRC:

1)

Question is sat as replaced.

57 H

3 B

E NRC:

1)

Change CFR 55 reference to 55.41(8) and list only one

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

2)

Errant space in bullet 1X04 locks out.

This bullet can be combined with the prior bullet - fire has occurred.causing a lockout of 1X04

3)

Can the applicant assume the jockey pump will maintain header pressure with several sprinkler heads open. Nothing in the justification discusses jockey pump capacity.

Licensee:

1)

Done

2)

Done

3)

No, the jockey pump will not be able to keep up with the demand. It has a different purpose to fill the accumulators at 55 gpm and make up for small leaks but not several sprinkler heads.

NRC:

1)

Quest is sat as revised 58 H

2 X

N U

S NRC:

1)

Having SFP in the question stem is awkward with the answer only the RWST and. Question may be cleaner to construct with 1) SFP, 2) RWST, 3) HUT and have the answer/distractors be only 1, only 1 and 2, only 1 and 3, and 1,2,3,

2)

K/A mistmatch. K/A is asking a impact on the SFP due to filters/demin issues.

Question is asking whats impacted due to a high SFP filter dp, beyond the SFP (which is given). Knowledge of the impact on the SFP is not required, as its given.

Licensee:

1)

Question has been modified as suggested and has been discussed.

2)

See above.

NRC:

1)

Question is sat as revised.

59 H

3 N

S NRC:

1)

Change CFR 55 reference to 55.41(4) and list only one

2)

Is providing the C-350 noun name in the stem necessary (not done for C-03)?

3)

Errant space prior to (1) and (2) in distractor C

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Licensee:

1)

Done

2)

Added name for C-03 and fixed the space issue

3)

Done NRC:

1)

Question is sat as revised.

60 H

2 X

B E

NRC:

1)

Distractor A is awkwardly written given the stem: which of the following is the minimum required level in T-24A - line up is not allowed. Would be an easy choice to eliminate as it doesnt make sense in terms of the question stem.

2)

Change CFR 55 reference to 55.41(4) and list only one Licensee:

1)

Replaced with the Low Low Alarm set point 4 ft (9,400 gal)

2)

Done NRC:

1)

Question is sat as revised.

61 H

3 B

S 2009 Point Beach NRC:

1)

Remove CFR 55 references except 5.41(7)

Licensee:

1)

Done 62 F

3 B

E 2012 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(7) and list only one

2)

List valve numbers in answer/distractors, since theyre given in the stem (beyond supply valve will shut - change to 1VNPSE-3244 will shut

3)

Answer justification needs enhancement -

fans are interlocked how? Answer explanation simply says all fans and valves will be secured after a CVI signal.

This does not specify its an auto isolation/trip or manual action required.

Licensee:

1)

Done

2)

Discussed. No change required.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

3)

Added the following to the justification:

When a CVI is actuated, in this case from 1RE-305 HIGH ALARM, the purge supply and exhaust valves will close. The purge supply and exhaust fans are interlocked with the purge supply and exhaust valves and will therefor trip.

NRC:

1)

Question is sat as revised. Valve numbers not added due to the convoluted appearance of the answer/distractors -

distracting to the licensed validators.

63 F

2 N

S NRC:

1)

Change CFR 55 reference to 55.41(7) and list only one

2)

In the stem, need a comma between Pump, the Licensee:

1)

Done

2)

Done 64 H

3 B

S 2011 Point Beach NRC:

1)

Change CFR 55 reference to 55.41(7) and list only one Licensee:

1)

Done 65 F

2 N

E S

NRC:

1)

Distractor C justification references caution tags when the distractor is referencing danger tags.

2)

Remove CFR 55 references except 55.41(10)

Licensee:

1)

Plausible as Caution Tags can be used as status control and to provide information.

This is not true of danger tags however.

Plausible if the student confuses danger tags and caution tags for status control.

2)

Done NRC:

1)

Question is sat as submitted.

66 F

F 2

2 X

B N

U S

Free Look 2017 Point Beach Q68 NRC:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Question does not meet tier 3 generic requirements (question should remain generic in nature) - question is an extension of tier 2 systems knowledge.

While the question is a good K/A match, it does not fit the tier. Justification states tier 3 is met as the K/A is met. This is not a valid tier 3 justification. ES-4.2 B.2.c.

Licensee:

1)

Question and KA swapped for another.

NRC:

1)

Remove CFR 55 references except 55.41(10)

Licensee:

1)

Done 67 F

2 B

S NRC:

1)

Remove CFR 55 references except 55.41(10)

Licensee:

1)

Done 68 F

2 B

S NRC:

1)

Change CFR 55 reference to 55.41(13) and list only one Licensee:

1)

Done 69 H

F 3

2 X

X X

B N

U U

S Free Look 2019 Point Beach NRC:

1)

Question does not meet tier 3 generic requirements (question should remain generic in nature) - question is an extension of tier 1 EOP & tier 2 systems knowledge. Bank question was used in tier

2. ES-4.2 B.2.c.

2)

Based on the knowledge of setpoints/interlocks/actions associated with EOPs/AOPs, this is a very difficult tie to a tier 3 generic. Recommend reselecting generic section 4 EOPs/E-Plan K/A.

Licensee:

1)

Question has been replaced.

NRC:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Per OM 3.7 note prior to step 6.18.3, licensed control room personnel may perform the initial assessment of status trees until the STA assumes monitoring responsibilities. Question stem does not specify that the initial pass was completed, or the STA has assumed duties (this would be cueing) and the applicant can argue that there is no correct answer.

2)

OM 3.7 step 5.9 states the STA is responsible for informing shift supervision of challenges to the critical safety functions. Is the OS not shift supervision? Need to clearly ensure that C is incorrect. OM 3.7 does not appear to specify shift manager in this role.

3)

Q87 stem provides cueing to the answer of part 1 of this question.

Licensee:

1)

Added assume the control room is fully staffed. Also added IAW the EOP network to the stem.

2)

This portion is referring to procedural action steps to inform SM of equipment malfunctions.

3)

Comment has been fixed. See Q87 for response.

NRC:

1)

Replacement question is sat as revised.

70 H

2 B

S 71 F

2 B

S Reference provided (embedded) 72 F

2 B

S 73 F

2 B

S 74 F

2 B

S 75 F

3 B

S NRC:

1)

Change CFR 55 reference to 55.41(1) and list only one Licensee:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Done 76 H

3 B

S 2007 Point Beach 77 H

3 N

E Reference provided NRC:

1)

It is unnecessary to include and its associated note in the stem. Step 14 directs determining time to boil IAW Att C, that includes the note. The applicant should know how to read and apply notes, without being directed to do so.

Licensee:

1)

Removed the information.

NRC:

1)

Question is sat as revised.

78 H

3 B

S 2017 Point Beach 79 H

3 B

E 2014 Comanche Peak NRC:

1)

Is it necessary to state rad levels are rising very slowly? Slowly and rapidly are used throughout the exam, but very slowly or very rapidly is not. Does not appear that very has any impact on the question; recommend removing it for consistency.

Licensee:

1)

Done NRC:

1)

Question is sat as revised.

80 H

H 3

3 X

B B

U S

Free Look 2012 Point Beach 2007 Point Beach Reference provided NRC:

1)

K/A is ability to diagnose and recognize trends in an accurate and timely manner using appropriate MCR reference material.

Question is using plant conditions to determine procedure transition criteria, not diagnosing & recognizing trends using reference material.

Licensee:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Question was replaced.

NRC:

1)

Replacement question SAT.

81 H

3 B

S 2009 Seabrook 82 H

3 X

B E

2011 Point Beach NRC:

1)

Answer/distractor justification can be improved. Recommend adding which SI termination criteria not met (PZR level) and reference EOP-0 step 14 in justification for distractor A. Reference step 18 & criteria for ECA-1.2 transition.

2)

Distractor B does not appear to be plausible, as there is no procedural path from EOP-0 to EOP-1.2. Stem states EOP-0 is in effect - would EOP-1 be a better distractor?

3)

Change CFR 55 reference to 55.43(5)

Licensee:

1)

Added EOP-0 step 14 to distractor A and discussed ECA-1.2 step 4 and indications for entering ECA-1.2.

2)

Plausible as this procedure is a procedure one would use for a LOCA. As this is part of the overall mitigative strategy it is plausible an examinee would select it.

3)

Done NRC:

1)

Agree with licensee justification for #2.

Question is sat as revised.

83 H

3 M

E 2012 Point Beach NRC:

1)

Answer explanation appears to be contradictory to procedure guidance.

Explanation; the procedure cautions operators on ensuring proper shutdown margin is maintained via boration, but no boration is required. The procedure states: boration should continue to obtain adequate shutdown margin during subsequent actions. So while it may or may not be required, it should continue.

The explanation simply says no boration is

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level required. This is not entirely true, as the procedure states.

2)

What does the background document for CSP-S.1 step 15 say? This is not referenced in the answer explanation.

Licensee:

1)

Discussed.

2)

Reference background information in the justification NRC:

1)

Based on the inclusion of the confirmation of subcriticality from the CSP-S.1 background document into the answer justification (addressing comment #2),

boration should be terminated under the given conditions. Therefore, comment #1 is adequately addressed and the question is sat as revised.

84 H

3 M

S 2003 Point Beach NRC:

1)

Correct (1) (2) numbering in answer justification for distractors B, C, D Licensee:

1)

Done 85 H

3 N

E NRC:

1)

Recommend rephrasing part 1 of A & B to automatic reactor trip should not occur.

This would maintain consistency with the automatic turbine trip should occur.

Licensee:

1)

Done NRC:

1)

Question is sat as revised.

86 H

3 N

E NRC:

1)

Correct numbering in answer explanation (1) (2).

2)

Answer explanation should also reference AOP-1D step 33 RNO for part 2.

Licensee:

1)

Done

2)

Explanation updated.

NRC:

1)

Question is sat as revised

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 87 H

2 B

E 2009 Point Beach NRC:

1)

Stem provides cueing for Q69 (STA informs OS2 that conditions exist for a red path for heat sink - stem is providing that STA should be the one monitoring CSPs).

Either modify the bullet in the stem to remove the STA informing OS2 or modify Q69.

Licensee:

1)

This has been changed to The OS2 is informed that NRC:

1)

Question is sat as revised.

88 H

2 M

S 89 H

3 B

E 2021 Point Beach (previous 2 exams)

NRC:

1)

Part 1 of the stem asks the action to be taken IAW the SOP. The direction to align D-305 and D-09 to D-03 comes from AOP-0.0 step 21 RNO, referencing the SOP.

Consider changing the wording, as the SOP does not necessarily direct to perform that lineup, the AOP does.

Licensee:

1)

Made the change.

NRC:

1)

Question is sat as revised.

90 H

3 X

B U

S NRC:

1)

K/A mismatch. K/A is knowledge of the purpose and function of major system components and controls. Question is asking knowledge of TS based off of system component manipulation, indirectly requiring some knowledge of system component function. Question is a match to K/A G2.2.38 or G2.2.40.

Licensee:

1)

Question pertains purpose/function of the cross connect and the consequence/

requirement of its operation.

NRC:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

1)

Agree with the licensees contention that knowledge of the function of the EDG starting air system is necessary to determine the impact (TS impact).

Question is sat as submitted.

91 H

2 X

B U

S NRC:

1)

Knowing the procedure flowpath is not using procedures to correct, control, or mitigate the consequences of those abnormal operations. To satisfy part (b) o the K/A, the question needs to use those procedures to correct, control or mitigate the transient; this goes beyond simply knowing the procedural flowpath.

Licensee:

1)

Originally considered unsat. However, after discussion this was considered sat.

Question meets the tier 2 requirements for this KA.

NRC:

1)

Agree with the licensees contention that this is a tier 2 question that meets the K/A.

Question hits both the (a) and (b) parts of the K/A by requiring the applicant to diagnose transient and also understand how the transient is mitigated through the EOP flowpath. Question is sat as submitted.

92 H

H 3

3 X

X X

N N

U U

S Free Look NRC:

1)

Question does not necessarily meet the K/A. Notification is required in regarding to the LOOP, not the main generator.

Question isnt asking what notification to make or why, its asking what EAL is required. EAL may require making a plant announcement per the site procedure, but the question isnt asking that (also, that has nothing to do with main generator).

2)

Why are applicants required to know EALs (UE vs SAE for LOOP) from memory? This is not something I would expect applicants to know from memory (no reference to be provided per pedigree)

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

3)

All similar distractors have met after 15 minutes and make notifications. Since this is the same, this can be eliminated.

4)

Would be preferable as a standard 2x2 question - part 2 has UE in 2 of the 4 (with the SAE only in 1 of the 4); this typically points to the UE being the correct answer.

Provides some cueing due to the difference.

5)

Answer pedigree is incomplete - SA1.1 is not met in this condition because both EDG.. Both EDGs are operating?

Licensee:

1)

Question has been replaced.

NRC:

1)

Change CFR 55.43 to (5).

2)

All answer/distractors include tripping the turbine. This can be put into the stem to simplify the answer/distractors.

3)

Not SRO-only. Question requires knowing the impacts of the transient, including whether P-9 is in and if the reactor trips, along with procedure entry conditions.

Applicant just needs to know status of P-9 and procedure entry conditions to answer the question, not any plant conditions to determine what to do in the procedure.

Licensee:

1)

Question has been replaced and determined to be SAT.

NRC:

1)

Replacement question is sat.

93 H

3 M

E NRC:

1)

Answer explanation specifically states LCO 3.3.5 is not met (wording needs changed -

TS 3.3.5 to be not operable; the TS is not met and the system/function is inoperable),

but does not specifically state LCO 3.7.9 is not met. Per B3.7.9-3, (g) states CREFS is capable of being automatically and manually initiated in the emergency mode of operation. If RE-101 is OOS, then auto actuation is non-functional and therefore CREFS is inop and LCO 3.7.9 is not met, correct? Would this be Condition C filter

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level trains inoperable for reasons other than Condition D?

2)

Remove CFR 55.43(b) links except (b)(2)

Licensee:

1)

This question was discussed with the NRC and facility representative. Question was changed to Mode 4 and asks what the status of CREFs Instrumentation was.

NRC:

1)

Question is sat as revised.

94 F

2 B

S 95 H

3 N

S Reference provided NRC:

1)

Note A stating hydrogen levels shall be >

15 cc/kg but may be < 25 cc/kg for up to 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> after criticality without instituting Action Level 1 - is this stating that Action Level 1 may be implemented, but is not required to be?

Licensee:

1)

Question was discussed by the facility representative. Operators are trained to not willing enter actions levels unless required.

The question is asking which action level is required.

NRC:

1)

Question is sat as submitted.

96 H

3 X

B U

S Reference provided 2009 Point Beach NRC:

1)

Question is an equipment control Admin JPM than a written question; extension of the operating test. Question requires system level knowledge (understanding of system alignment to use drawings to evaluate & determine component positions, which is tier 2)

Licensee:

1)

Question was replaced and determined to be SAT.

NRC:

1)

Replacement question is sat.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 97 H

2 3

B E

S Free Look Reference provided NRC:

1)

While the question is not direct L/U with the reference provided, it is very simple given that the times (1200) are all the same.

Recommend modifying the inop times (very easy to look and see 24, 48, & 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> in the stem)

Licensee:

1)

Times modified as suggested.

NRC:

1)

Question is SAT as revised 98 H

F 2

B S

NRC:

1)

OI 140B may need clarification to step 3.9.

Consider modifying the step (after the exam) to clarify that liquid discharges can be started/initiated within 24 of the approval

- the step only discusses discharges that are interrupted may be resumed. If the discharge is not initiated, does the 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> still apply, as the question asks?

2)

Change CFR 55.43 reference to 43(5).

Licensee:

1)

Comment has been understood and will be evaluated. Facility representative discussed this. No changes needed.

2)

Done NRC:

1)

Question is sat as submitted.

99 H

2 X

B E

Reference provided NRC:

1)

The question pedigree states none for proposed reference to be provided to the applicants during examination, however, the EAL wall charts are included for Q99 in the handouts provided. If they are not provided, this is LOD=5 and unsat.

Additionally, Question stem does not state "reference provided as other questions with provided references do.

2)

Additional actions beyond the EAL are unnecessary, as simply determining a SAE would make answer C correct; the applicant does not need to know any of the

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level actions required. If the intent to determine if the applicant understands procedural actions, this should be a 2x2 type question, however, then this may not be a tier 3 question, but a tier 1 AOP/EOP question.

Licensee:

1)

Changes made. The EAL chart was intended to be provided.

2)

Changed the stem and answers to ask the highest emergency classification level for the event. Also changed that the Site VP was taken hostage in a SSB Office.

NRC:

1)

Question is sat as revised.

100 F

2 B

S 2015 Waterford

Form 2.3-5 Written Examination Review Worksheet Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.