ML23143A205

4-PV-2023-05 Form 2.3-5 Draft Written Examination Comments

ML23143A205
Person / Time
Site:	Palo Verde
Issue date:	05/11/2023
From:	Heather Gepford Operations Branch IV
To:	Arizona Public Service Co
References
Download: ML23143A205 (1)

Text

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 1

H 3

X N

E

-Answer B: you dont necessarily know that RCPs are running, just that they have power available based on board alarms provided. Edit language.

-GENERIC: All worksheets say Level of Difficultly - edit. - Fixed throughout I would say that since the cause of the trip was an RPS failure and the crew has not been told that any RCP malfunction has occurred, nor than any RCP has been stopped by the crew,that they have to assume the RCP configuration is as it was pre-trip unless the alarms indicate that RCPs would not be running. But I am okay editing if desired.

NRC - OK.

2 L

2 N

S Ok.

3 L

3 X

N E

Technically a 20 GPM leak is not a SBLOCA. Can you increase size to slightly beyond the capacity of makeup to make it a SBLOCA and match the K/A better?

I can, but Q9 is basically asking what the limit for makeup is so since a 20 gpm leak would still require transition to the LOCA EOP, I stayed away from the 150 gpm range intentionally.

Changed to 200 gpm.

4 H

3 X

N E

-Add tripped from 100% power to stem. - Done

-Confirm that the stem has sufficient info for applicant to determine that H2 Analyzers are already aligned and BAMPs are available. - H2 analyzers are aligned, they just have to be started up. Nothing in

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level the stem indicates that BAMPs are not available and they normally would be so I dont think any additional info on those pieces of equipment is necessary.

NRC - OK.

5 L

2 B

S 2020 Q26 OK 6

H 4

X N

E

-Answer A.2: Is there anything actually in writing that says class equipment should be used before non-class eqpt? Common sense but is it documented?

Yes. It is in the LMFR lesson plan. However the revision I uploaded was rev 8 and the information was added in rev 9. Added rev 9 of the lesson plan to the EOP procedure folder in Box. Also updated the pedigree on this question to include the information from the current revision of the lesson plan.

-Isnt B.2 actually true though? It will prevent elevated dose rates. - No, since the NC flow is only running through a HX to provide cooling. Absent a leak in the EW HX, there should be no change in dose rates outside the Aux Building. If thought that reactor coolant was flowing to the NC HX, then yes, but that is not how the flow path works.

NRC - OK.

Stem rearranged to change answer from A to C 7

H 3

B S

2022 Q53 Ok.

Stem rearranged to change answer from A to D 8

L 3

X N

E

-First bullet of stem: manually should be manual.

- Fixed

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

-Distractor D explanation: Im not quite seeing how MG OUTPUT BKR opening first would prevent a Fast Bus Transfer - the early b contacts on NAN-S01/2 would still be shut so a FBT should still occur regardless of the order MT and MG are tripped - is that right? - Distractor D doesnt talk about a MT or MG trip, it talks about opening the MG output breakers. The breakers being opened prior to the generator being tripped will prevent FBT from actuating because a generator trip is one of the FBT requirements. Additionally, FBT wont actuate because the synch check relay will not be satisfied (opening the output breakers will result in an immediate loss of voltage to NAN-S01/2).

Additionally, D(1) is incorrect.

NRC - OK.

9 H

3 N

S Ok.

10 H

3 X

B U

KA mismatch - doesnt test on consequences of a SLR on ATWS, just ATWS response actions.

Procedurally, there arent any differences between how to mitigate an ATWS with or without a concurrent event. At least not until the Reactor is tripped or all reactivity control contingency actions are completed. The way I tried to match the KA here was to use the ESD to create a situation in which the MSIS could be desired to stop the steam leak and determine if the applicant knew whether or not that would come before or after completing the reactivity control actions. If you want to replace this question, probably need to change the KA.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC 3/7: If unable to write a question that meets K/A, replace with AK1.07 Effects of feedwater introduction on dry S/G IR=4.0. Update outline. -

Outline updated.

UPDATE - Upon further review, I dont think this is a good KA replacement as we dont re-initiate feed to a dry SG during ESD events (we only feed the least affected SG, which would not be the dry SG). We do feed a dry SG, but only in a loss of feed event.

Recommend a different KA to replace with or stick with the original question.

Replaced KA with 040 AK1.09 per CE approval and replaced question.

NRC-OK.

11 H

3 N

S Ok.

12 H

3 N

S OK.

13 H

3 X

N E

Answers C and D - you dont need to include ONLY because they are not a subset of another answer. - Done 14 H

3 N

S OK.

15 H

3 N

S Ok.

16 H

3 N

S OK.

17 H

3 X

N E

For better plausibility, I prefer the following combination of answer choices:

A. 1 ONLY.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level B. 1 and 2 ONLY C. 1 and 2 and 3.

D. 1 and 3 ONLY.

Reason is that I dont think its plausible not to include

1.

Changed, however answers are (for the sake of sequence):

A. 1 ONLY B. 1 and 2 ONLY C. 1 and 3 ONLY D. 1, 2, and 3 NRC-OK.

18 H

3 X

N E

Answers C and D - you dont need to include ONLY because they are not a subset of another answer. - Done

-Stem: replace to relay and de-energize with to de-energize on protective relay actuation. - Done 19 H

3 X

N E

Add a final bullet: Group 5 CEAs moved 10 steps prior to CEDMCS being taken out of AUTO SEQUENTIAL. - Done Replace the question in (2) with, Which procedure will actively mitigate the event? This is a more direct way to ask the question. Entry into 40AO-9ZZ11 CEA Malfunctions would be justified by entry conditions, but would not actively mitigate the event.

- Done, but I rephrased to Which procedure should be used to mitigate the event since that is how I would write it and I like the phraseology to be consistent. I can change to the way you wrote if you want.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC-OK.

20 H

3 X

X N

E Part (1) distractor C/D: The IA leak could impact other components in the IA loop, and this AOP wouldnt necessarily be inappropriate to enter at least initially. Id rather test on how this valve responds on a loss of air. I think the KA will still be met by part 2.

Part (2) rephrase to: Restoration of letdown flow will be accomplished (from the control room only vs from both the control room AND local operations in the field.)

Modified to incorporate both comments. Answer changed from A to D.

21 L

3 X

N E

Part (2): modify to Under these plant conditions, changing ventilation (IS/IS NOT) expected to result in significant changes to SFP level. - Changed 2nd part as described, however said, Under these plant conditions, changing ventilation _____ expected to result in noticeable changes to SFP level. I think noticeable is less subjective than significant. I guarantee students will ask to clarify what I consider significant. Im open to suggestions on that though. I also wonder if asking about the impact of changing ventilation in Part 2 implies that C or D is correct (they are not). Just a thought.

Answer changed from A to B Final: retained significant changes to SFP level in part (2), as determined to be less subjective than noticeable.

NRC-OK.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 22 L

3 N

S OK.

23 L

3 X

N E

Stem, should say UNinhabitable in 2nd bullet. -

Done Add Per this procedure to beginning of question. -

Done 24 H

3 N

S Add Per this procedure to beginning of question.

Done 25 L

3 X

N E

I dont like which ARP should you enter FIRST questions unless there is an explicit color-coded hierarchy between the relevant alarm windows directing one to be prioritized over the other. You could often argue multiple ARPs can be entered in parallel and theres no detriment to pulling a procedure and quick-referencing.

Rephrase part 2 to Under this condition, 40AL-9RK3A IS/IS NOT also expected to alarm. This implicitly tests knowledge of the input for this monitor.

Rephrased. Part 2 now says, The HIGH alarm on RU-155D WILL/will NOT cause 3A10B, LD PROCESS MON TRBL, to annunciate.

NRC Ok 26 L

2 X

B E

What is the bank source? - PV exam bank. Any question I mark as bank without citing a source (or annotating previous NRC exam) is from the PV exam bank but Ill add that note to each of those on the exam. Didnt know we were supposed to do that unless it was from another station or GFES bank.

What is the setpoint for CTMT RAD Monitor HI alarm on RM-148/149, or another detector? If its closer to 1E8 mR/hr it may be a better distractor than 1E3

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level mR/hr due to a direct relationship to CTMT. From Branch Chief: The range for the containment high range monitors is 1 R/h to 1E7 R/h. Thus they are barely registering at 1 R/h; if the CHRMs are one of the determiners for when harsh containment conditions exist if the applicant knows the range of the CHRMs then 1R/h wouldnt be plausible.

Both good points. The HIGH alarm for RU-148/149 is 1 x 10E4 mR/hr. That is a little closer so I think that would be a better distractor. Changed A2/C2 to that value. Side note, the setpoint for RU-148/149 is not listed in the ARPs as this is a variable setpoint, however the maximum alarm setpoint is 10E4 mR/hr per the UFSAR and TS Bases and this what the high alarm is set to.

NRC-OK 27 L

2 X

N E

Part 1: Modify to ask WHICH RCPs provide driving head (1A/1B vs 2A/2B), slightly more discriminatory. -

-Done This is a lower cognitive level systems knowledge question, not higher. - Done 28 H

3 N

S ok 29 L

3 X

N U

KA mismatch, does not test on a loss or malfunction of system. Normal ops knowledge only.

I understand the comment. The reason I wrote this the way I did is that CS pumps are not required when RCS pressure is < 385 psia per LCO 3.6.6. As such, I make the LPSI Pumps both unavailable in the stem (a defacto loss of the normal RHR pumps) to ask about the requirements to use the CS Pumps for

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level RHR instead (requirements are different than they are for LPSI). Happy to replace it, but not sure if you have something else in mind or would prefer a different KA for this question.

NRC 3/7: OK I can see that now, I do think that the question as written meets the K/A acceptably.

30 L

3 X

N S

Stem part 1) delete comma after Injection. - The comma is there since Hot Leg Injection is the title of the appendix. I can delete if you want. IMO, its a little awkward with or without the comma.

Part 2 capitalize Hot Leg Injection. - Its not capitalized in part 2 since its not used as the title of a procedure but I can do that if you want. I know it seems a little odd to be capitalized in part 1 but not part 2 but it makes sense to me. No biggie, just let me know if you still want it changed.

Modified per CE recommendation.

NRC Ok 31 H

3 B

S 2018 Q34 ok 32 L

2 M

S 2021 Q16 Spell out Nuclear Cooling. Done 33 L

3 N

S OK 34 L

3 B

S 2016 Q10 OK 35 L

3 N

S OK 36 H

2 X

B U

Parts 1 and 2 questions, add LCO after Technical specifications. - Done

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level K/A M/M: This isnt testing of effects of a malfunction, its testing on memory of TS LCO setpoints. If the K/A is too simple to write a discriminatory Q to, we can replace the K/A. - Yeah, I get what you are saying. To say that containment temperature (or pressure) would do anything other than rise following a loss of containment cooling seemed non-discriminatory so I focused on the impact of the loss, i.e. when will a limit be reached. KA replacement would be an option here.

NRC 3/7: Replace with K/A 022 A3.01 Ability to monitor automatic features of the Containment Cooling System, including: Initiation of ESFAS mode of operation 4.2 - Done Note that this is a different category (A3 vs K3) so update the count in the outline. - Count updated 2018 Q40 used for new KA with CE approval.

NRC Ok.

37 L

3 X

N E

1st bullet, add.. ESD inside Containment 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> ago. This is for distractor D plausibility. - Done 38 L

3 X

N E

Im not a fan of distractor C as it doesnt actually exist on the MSLs. I understand what you are saying but I dont know that there is a more plausible distractor to replace it with.

Replaced C with the size difference between the Main Steam header and cross-tie piping per CE recommendation.

NRC Ok.

39 L

2 X

N E

Distractor A is implausible - its the low end of normal operating speed. Use 5889 rpm as it is the speed limiter setting per the lesson plan. - Done

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 40 L

3 N

S OK Stem rearranged to change answer from A to D 41 H

3 M

S 2022 Q28 OK Stem rearranged to change answer from A to C 42 H

3 N

S OK 43 L

2 M

S 2022 Q40 OK 44 H

3 X

B E

2016 Q52 Specify Indicated radiation levels will begin -

Done 45 L

2 B

S GFES P624 OK 46 H

3 X

N E

Modify stem to Compressor should automatically trip on ___1___ and IA pressure will begin to recover

- Done 47 L

2 B

S 2018 Q22 ok 48 H

3 N

S OK 49 H

3 N

S OK. Clarify in explanation that Chill Water (WC) is not included in 9ZZ03 response actions. - Done 50 L

2 N

S OK.

51 H

3 X

M E

2016 Q43 Stem: With NO operator action, how DOES the AFAS - Done. Good catch 52 H

3 X

M U

2021 Q44 B and D dont make sense as distractors. The logic used to justify means the applicant calculates the correct answer and then makes an illogical assumption.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Use 8gpm as the basis for one of the distractors which is the SBOG max consumption rate. Then make the other distractor based on 920+1055GAL in case the applicant thinks they can use fuel from the A EDG FODT.

Im fine with the change, I just prefer the symmetry of the answers as is. Change made. Answer choices are now 35, 115, 140, 300.

53 L

3 X

N E

I dont prefer the only choice which isnt correct to be a standalone choice. Rearrange to:

A) 2 ONLY.

B) 1 and 2 C) 1 and 3 D) 2 and 3 Done 54 H

3 B

S 2018 Q4 OK.

55 H

3 N

S OK - KA changed from 001 K2.03 to 001 G 2.2.3 per CE request 56 L

3 N

S OK 57 L

2 X

N S

OK, but consider if excess flow check valve is a better alternated for C/D part 1). - Im fine with either as a distractor. The SFP actually has an auto makeup system so I think that makes more sense but Ill change if you want.

LOD=2. - Done Left as is per CE recommendation.

NRC Ok.

58 L

2 X

N E

B/D part 2) distractor is not credible. Test on the most limiting vessel component for thermal stress

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level instead, if that is RO knowledge. - I disagree. The cooldown rate is routinely discussed post-event when training on EOPs and the operational concern is cooling down faster than the Charging Pumps can handle due to the shrinkage in the RCS. Knowing the most limiting vessel component may be RO knowledge (not sure) but I think asking that instead of the basis for the cooldown rate is less operationally relevant.

NRC Ok after demonstrating relevance of Charging Pump makeup for 60°F/hr 59 H

3 X

N E

I dont find 3 - Main turbine Intercept Valve position indication to be credible. Replace with Main Turbine 1st Stage Pressure and the answer becomes 2 and 3. - First stage pressure does change, but the change is so slight I dont believe an operator could use that as a viable indication to determine the potentiometer is in control of turbine load. 1st stage pressure is read on a peanut gage and reads in very small increments (20 psig/line). I ran this evolution in the simulator and (using digital indication on the simulator computer) saw that 1st stage pressure changed from 704 psig to 702 psig. This is imperceptible to the naked eye. I can try to find another distractor but I dont think 1st stage pressure works.

UPDATE - After speaking with the Ops Rep about this, he said that 1st stage pressure is monitored and even a 0.5 psig change is considered confirmation that the potentiometer is in control of turbine load.

Although not really visible to the naked eye using the gage, this is monitored digitally on a computer and is

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level a valid answer. Change made. Answer change from C to D.

NRC OK 60 H

3 M

S 2021 Q63 OK 61 H

3 X

N E

3 does not seem credible. At least 2 is a radiological system. - It is not a radiological system, however since the turbine building sumps can be realigned to the LRS, it seems plausible that it would be plausible to send oil there as opposed to running the risk of spilling oil to the environment. Given the financial and political costs of an oil spill, it seems reasonable to me to send to a different storage system as opposed trying to contain the oil in a non-sealed sump inside a building that is not water tight.

Used replacement option per CE recommendation.

NRC Ok.

62 L

3 X

N U

3 is not an Area Radiation Monitor it is a Process Monitor. (So is 4 but its not part of the answer)

Good catch. I will try to find two other plausible area RMs and make it a 1x4 or at least one other and make it a choice out of 3.

Used replacement option per CE recommendation.

NRC Ok.

63 L

3 X

N E

Do you have additional procedural refs / training on this you can provide?

C/D distractor 1) - I do not see how increasing load to improve vacuum is credible.

KA changed to 075 A2.12 and new question used. Outline updated.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC OK.

64 L

2 B

S 2021 Q67 OK 65 L

2 X

N E

Dist A - add and rising to end. - Done Tripping the Reactor is too obvious an answer. Make the correct answer CSAS with 2/4 channels > 8.5 psig, or any other ESF actuation. - Changed from Rx trip to initiating a CIAS at 1800 psia RCS pressure (setpoint is 1837).

Not seeing that this is Higher Cognitive Level.

Changed to low cog.

66 L

2 B

S 2020 Q69 OK 67 L

3 B

S 2018 Q71 OK 68 L

2 M

S 2019 Q72 OK 69 H

3 B

S What is the bank source? - PV exam bank 70 H

3 X

X N

U Part 1): can you just say MTC of reactivity instead of change in reactivity per degree change is RC Tavg?

Part 2) is too simple, yes I know its GFE. Change the question to MTC affects reactivity due to [change in density of moderator as RCS Tavg changes, vs Change in vapor concentration as RCS Tavg changes]. Distractor is Void Coefficient of Reactivity.

Sounds reasonable. Ill make that change and run it through a couple of validators.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level FYI, the concept of Tcold vs Tavg change from low to high power is a common misconception at PV.

Although Tcold lowers, Thot rises at a faster rate, therefore Tavg rises. People focus on Tcold since that is what we control to and therefore some operator miss questions like this. Seems easy to me too but exam history tells a different story.

NRC OK.

71 H

2 B

S GFES P1654 OK 72 H

3 B

S OK 73 L

3 N

S OK 74 H

4 N

S This is a really good, integrated operations GFES question. - Thanks Provide a standalone comment in the explanation stating how lowering frequency is causing a reduction in RCS flow. - Done 75 H

3 X

N U

Part 2 is not generic. Good question otherwise.

I agree that part 2 is not generic but my understanding was that the GFES questions were supposed to incorporate GFES knowledge as it applies to the specific plant. Does the concept that is applied to SRO questions not apply to GFES questions?

I can change the 2nd part though. Perhaps for the second part I can ask about the reason for the power reduction (to protect the integrity of the fuel vs to

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level ensure the applied penalty factors dont result in a reactor trip?)

2nd part is related to OE from 2013 Unit 3 rod drop in which power reduction NRC Ok 76 H

3 N

S OK 77 H

3 X

N U

A/C part 2) distractor, 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> is not credible that someone would think LCO 3.0.3 when the stem directly asks, per LCO 3.4.9. The justification for why someone would think 3.0.3 would be entered is also unconvincing.

I see your point but I think that with the TS low level at 27% and the heater cutout at 25% (which would make both class heater banks inoperable and thus in an unanalyzed condition per 3.4.9) that 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> is reasonable.

If the question was reworded to remove LCO 3.4.9, and they just needed to know if the 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> action applied or the condition would result in 3.0.3, would that be acceptable?

Changed Part 2 from per LCO 3.4.9 to per Technical Specifications NRC OK.

78 H

3 N

S OK. I agree that this question meets the intent of the K/A and it is desired to test on station-specific OE and recent procedure changes.

79 L

3 X

N E

NRC 2021 Q80

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level How was this modified? - Realistically, this should be a new question. The idea was inspired from Q80 on the 2021 exam but both parts of the question are significantly different than the original. I believe when I first wrote the question for the 2023 exam it was closer to the original and has morphed over the past few months and is really no longer a modified question. -ok Reading the explanation for Part 2) Im not sure the section provided addresses this particular situation.

The discussion is about ensuring offsite agencies are updated at least once that a release has occurred, which the stem says HAS happened. At what point SHOULD the SM notify offsite agencies that the release has been terminated? I spoke to the EP manager about this a couple of months ago when developing this question and he agreed that you would not declare the release terminated but the words in our procedure do not explicitly state that. I spoke to the EP Section Leader this morning and we tried to modify the question to make it more appeal proof. I think this will work better while keeping the same general idea of the question but aligning better with our site procedures.

NRC 3/7 - Ok, document the changes made in the explanation.

NRC Ok 80 L

3 X

N E

To make this more SRO-aligned, modify to involve procedure selection or actions if 650F is NOT met.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Changed Part 2 of the question to ask if transition should be made to the FR or SAG-1 if the parameter in Part 1 exceeds 650°F and is slowly rising.

NRC Ok 81 L

3 N

S OK Q96 is a better use for the concept on Q81 so I used this concept on Q96 and generated a new question for Q81 Used new Q81 per CE recommendation.

NRC OK 82 H

3 N

S OK 83 H

3 B

S NRC 2018 Q82 OK 84 L

3 X

N E

Applicant is going to want to know when 75 gpd leak rate was confirmed.

Modify the format of this question to provide 3 data points for Leak Rate:

1100 - 65 GPD 1200 - 75 GPD 1300 - 85 GPD Part 1) becomes the whole question. Answer choices are:

A) 1800 B) 1900 C) 1200 next day D) 1300 next day This format tests on understanding of App F. Note that 75GPD is sustained >1HR at 1300 BUT the time to commence shutdown starts when LR first exceeded 75GPD at 1200. - Wrote the question, but

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level I think the correct answer would be 1300, not 1200.

The criteria of Appendix F is not met until the 75 gpd has been sustained for an hour, which would be 1300. Also added that the leak rate remains stable at 85 gpd. It sounds to me like you believe 1200 the next day is the correct answer.

UPDATE - I was discussing this question with the Ops Rep and explaining how you got it wrong then saw the note explaining the clock starts at the time 75 gpm is reached. His opinion, and I agree, is that the level of detail about the App F required to answer this question closed book is not reasonable. We did agree that the question would be appropriate if App F was provided as a reference. I checked the remainder of the exam and found nothing that could be cued if App F was provided. Let me know what you think.

MODIFY ORIGINAL TO MAKE CLEAR THAT WERE LOOKING FOR THE SHUTDOWN ACTION, NOT THE PRECISE TIME MODE 3 MUST BE ENTERED.

Question modified with CE approval NRC OK 85 H

3 N

S OK 86 H

3 B

S NRC 2019 Q86 OK - Add Reference Provided to top of question.

Generic for all Ref Provided Qs. - Done Replace term rupture with break at time 0130. -

Done

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 87 H

3 B

S NRC 2019 Q89 OK 88 H

3 N

S OK 89 H

3 B

S NRC 2018 Q87 OK 90 H

3 N

S OK 91 H

3 N

E Make parts 1) and 2) two separate sentences. The loss of NNN-D12 RESULTS in a loss - Done Rephrase Par 2) to Under this condition, B EDG is

[OPERABLE/INOPERABLE]. - Done 92 L

3 X

N E

Stem: Consider changing 1600 lbs to 1550 lbs so that both values are not exactly the setpoints, but still acceptable. - Done 93 H

3 N

S Stem: tweak to within the first four hours or less

- Done 94 L

3 X

N E

1200-1300: Should we include the qualifier Temp Relief for the 1200 meeting with the Ops Director? -

I dont think so. The CRS is allowed to leave without being relieved so long as CR minimum manning is maintained (perhaps by the SM remaining in the CR while the CRS is at the meeting). This is how it would really happen so I would leave question as is.

NRC OK.

95 L

2 N

S OK. I agree that this meets Tier 3 requirements.

40OP-9ZZ10 is the procedure you go to generically at the completion of any optimal EOP. So while the distractors sound system-specific, the answer is really based on generic knowledge of procedural transitions.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 96 L

3 X

N U

2018 Q99 K/A mismatch. Doesnt test on variations between procedural actions between units, it tests on sitewide E-Plan implementation. The original 2018 question was a different, E-Plan specific K/A.

I understand the comment since the question is geared towards E-Plan. I was focusing on the variations in procedural actions between units at a facility portion which leaves it pretty open. I can move to Q99 and try to come up with something different here, but on an SRO exam, Im not sure that there is anything that is generically different between the units. Perhaps management of switchyard operations NRC 3/7: Yes, move this to Q99, and then test on the differences between rod control systems between the units if you can make it SRO-only. I know that sounds like system-specific, but the K/A is explicitly asking for variations in control room layouts, systems, instrumentation, or procedural actions between units at a facility. The rod control differences is high visibility so this is appropriate and meets the intent. - I made a new question for Q96 that would test unit differences on Rod Control but it overlapped with Q81. Wrote a new question for Q81 so I could use the Rod Control differences question on Q96, but Q81 replacement may (I dont think so) require a KA replacement.

NRC Ok with new question.

97 H

3 B

S NRC 2022 Q100 OK

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 98 L

3 B

S NRC 2022 Q94 OK Stem rearranged to change answer from C to B 99 L

2 B

E Id rather you use the Q96 bank question that I told you was a K/A M/M for this one, as it fits the K/A. - If you disagree with my assessment of Q96, Ill remove this question and replace with Q96.

NRC 3/7: Yes use Q96 here. - Done Moved original Q96 to Q99 per CE request NRC OK 10 0

L 2

X N

U This is Emergency/Abnormal plant evolutions specific knowledge of SBO, not Tier 3 generic.

Generic exit conditions from EOPs seem like RO knowledge. Entry into the ELAP is only done from Blackout but is also the entry path into a severe accident type procedure. Given the decision to exit Blackout and operate from FLEX or to remain in Blackout and perform FLEX in conjunction is more SRO level, although I acknowledge it is not as generic as a Tier 3 question would ideally be. Ill look and see if there is another direction to go with this.

SEE POTENTIAL REPLACEMENT OPTION -

WRITTEN 3/24, NO VALIDATION YETIf G2.4.22 (or similar) is used, consider asking about the time requirement for SFSCs and the basis for transitioning out of the EOP?

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Q#

1.

LOK (L/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level KA changed to G 2.4.14 and question replaced with CE approval NRC ok.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes. ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Form 2.3-5 Written Examination Review Worksheet Generic Comments:

Last 2 NRC exams: 2022, 2021 On future submittals, be more specific with numeric designators for the learning objectives. I.E. Enabling Objective ###, etc.

It would be helpful on future submittals to explain directly in the Explanations box why the correct answer is correct, not just why the distractors are incorrect.

RO exam is overbalanced towards A answers. Rearrange 6-8 of the A answers into D, and ensure explanations are modified accordingly. - I figured. Just didnt want to move stuff around until I knew what questions would need work. When all questions are acceptable, Ill balance the answers.

RO exam: Re-arrange 3 of the C answers into B as a correct answer. - Assuming you mean SRO exam, but I will do that.

You did a good job making the GFE questions into site-specific questions.

Form 2.3-5 Written Examination Review Worksheet - PV-2023-05

[NRC NOTE: Question Status column is left as originally assessed for historical purposes.

ALL QUESTIONS ARE ASSESSED TO BE IN FINAL SAT STATUS AS OF EXAM APPROVAL.]

Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.