ML22214A605

4_CNS-2022-06 Form 2.3-5 Draft Written Examination Comments (Final)

ML22214A605
Person / Time
Site:	Cooper
Issue date:	06/21/2022
From:	Heather Gepford NRC/RGN-IV/DORS/OB
To:	Nebraska Public Power District (NPPD)
References
Download: ML22214A605 (18)

Text

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 1

H 3

N S

Pre-review.

Ok No changes 2

F 3

N S

Ok No changes 3

H 3

X N

E S

Rephrase the last statement in the stem as a question. For example, Per the alarm cards associated with the above, which of the following If the reactor is going to FULL SCRAM, is verification of a Rod Block per the alarm cards really something the crew is going to be concerned about? Recommend rephrase the stem to read, which of the following automatic actions will occur?

Changed to suggestion.

Bolded complete loss of 24VDC NRC OK.

4 H

4 X

N E

S Part 2 distractor is not plausible. Moisture impingement on condenser tubes? An applicant believing the MSR exhaust communicates directly with Condenser is not plausible.

Changed distractor to the basis of the isolation for a load reject vice the high level in the moisture separator.

Rev.2 - Distractor on Rev.1, did not fix question, changed the second question to which turbine (low/high) is the primary concern for this condition based on needing another distractor during NRC Validation Week review.

NRC OK.

5 H

3 N

S Pre-review.

Ok No changes 6

F 3

X N

U S

Negatively phrased question is a psychometric flaw disallowed by NUREG 1021. Need to rephrase to a positively phrased question.

Changed to a positively phrased question.

Rev. 2 - Minor editorial changes to stem based on CE comments during NRC Validation week.

NRC OK.

7 F

2 M

S What is modified about this question?

Added the original bank question to the justification and explained differences.

NRC OK.

8 F

3 M

S Modified questions should explain what was modified from original.

Added what changed on the question for it to be modified.

Changed alignment to direct B lead, A 1st backup, C 2nd backup. The tendamatic position was removed from the procedure but left on panel as an operator aid. Could be challenged so used procedure guidance for stem.

NRC OK.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 9

H 3

N S

Pre-review.

Reference Provided Ensure figures 1, 2, and 3 are all provided so the applicant has to choose.

Reference added and changed one of the distractors to align with one of the other figures.

NRC OK.

10 F

3 N

S No changes 11 H

3 B

S NRC 2021-06 Reference Provided No changes 12 F

2 N

S No changes 13 H

3 N

S Pre-review.

Reference Provided No changes 14 H

3 X

N E

S STEM: protects AGAINST exceeding the Added the word AGAINST in the STEM as requested.

NRC OK.

15 H

3 N

S Reference Provided No changes 16 H

3 M

S This is Higher Cognitive Level Changed the COGNITIVE level to Comprehensive/Analysis.

17 H

4 B

S No changes 18 H

3 X

N E?

S What is the purpose of use actual values in the stem?

The TS value for this isolation is 49 mrem so this is testing the actual value. This is one of the few things that we have ensured they are trained on both actual and TS values.

NRC OK.

19 F

2 X

N U

S Part 1 is not discriminatory. Why have a Fire Alarm if it wouldnt be used for a Fire?

Changed first question to which emergency procedure entry conditions have been met. 5.4TURBINE-FIRE versus 5.1INCIDENT Added the word Direct to the start of the second sentence to prevent potential challenges per CE directions.

NRC OK.

20 H

4 X

N E

S Add only to the single option answers to eliminate partially correct answers.

added only to the single option answers.

NRC OK.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 21 H

3 X

N E

S Part 1 distractor - why is it plausible that an automatic trip setpoint would be reached before a manual trip setpoint?

If the stem was changed to 23.8 inches and degrading at 0.2 inches per minute, then the automatic trip would occur first.

They need to do the math to see if the 5-minute timer occurs before exceeding the turbine trip action point per 2.4VAC.

No changes at this time but if the CE requests, I can modify this question so the automatic would occur before the required manual trip.

NRC OK.

22 F

3 N

S Pre-review.

No changes 23 H

3 N

S No changes 24 H

4 X

B E

S NRC 2014-07 Distractor C: for plausibility, when would you start only one SGT train, if both failed to start?

Added the reason why an operator could believe Answer C is correct.

NRC OK.

25 H

3 B

S NRC 2015-12 No changes 26 F

3 X

N E

S Add FIRST required to stem, to preclude partially correct answer.

Added FIRST required to the stem.

NRC OK.

27 F

2 N

S No changes 28 H

3 B

S No changes 29 H

3 X

N E

S Can the second bullet of the stem simply state, A SWBP trips? The applicant should implicitly recognize from that that service water is then lost to A RHR HX.

It should be sufficient to ask, in part (2), if the procedure requires bypassing the RHR HX first, or starting another SWBP first, without discussion of boiling tube side of HX.

Reading through 2.4SDC is a bit odd, nowhere does it actually say to start another SWBP, Attachment 1 simply says to bypass RHR HX until SW flow becomes available.

Are you sure there is no justification for entry into 5.2SW on this condition?

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Changed second bullet to A SWBP trips. Does not meet any entry conditions to 5.2SW so no justification.

NRC OK.

30 F

2 B

S No changes 31 H

3 N

S No changes 32 H

2 N

S No changes 33 H

3 X

N E

S Its not necessary to write one (1).

Removed the (1) from the stem.

NRC OK.

34 F

3 M

S No changes 35 F

2 X

B E

S Add per 4.1.1 Source Range Monitoring System, Section 4 to the stem to make distractor D more explicitly wrong. The discriminatory value of just testing on which one takes one manipulation vs 4 single manipulations is kind of iffy.

Added procedure to the stem.

NRC OK.

36 H

3 N

S Pre-review.

No changes 37 F

3 X

X N

E?

S Spell out IPTE. Changed Confirm that applicants are required to know which section of 2.1.1 each of the listed evolutions is contained in. Concerned this may be considered minutiae otherwise.

Per SKL012-43-01 (ops cold S/U to 100% power in Event 8 (2.1.1)the candidates discuss for section 4 and 5 all IPTE briefs that are required to be given and for what.

NRC OK.

38 F

3 B

S NRC 2021-06 No changes 39 H

3 N

S No changes 40 H

3 N

S No changes 41 F

3 N

S No changes 42 H

3 X

B E

S Its not necessary to include the bank question in the explanation if its not modified. Only required if a modification is made to a bank.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Specify the section of 2.2.60 being used in the Stem (12 I believe)

Removed the bank question in justification. Added Section 12 and title in bullet in the stem and added Section 12 in the first question.

NRC OK.

43 F

3 N

S No changes 44 H

3 N

S No changes Minor enhancements on stem based on validation comments.

Gave time frame for the main generator to ensure the ABT had time to transfer to prevent the thought the second during the transfer.

NRC OK.

45 H

3 N

S Provide an example in the explanation of a normal-seeking ABT in use at CNS, for the part (2) distractor.

Added an example of a normal seeking ABT at CNS in Part 2 of the explanation.

NRC OK.

46 H

4 N

S No changes 47 H

3 N

S No changes Enhancement to stem to ensure only one correct answer based on validation comments.

NRC OK.

48 F

3 X

N U

S B not plausible that 4160V 1G remains energized with loss of DG2 and ESST.

C and D not plausible that 1GE would close in with DG2 TROUBLE alarm in and DG2 INCOMPLETE SEQ light on.

Test more directly on what inputs cause DG2 INCOMPLETE SEQ, and what impact it has on the generator.

Changed the question to ask about inputs to the incomplete sequence and part 2 for what is expected with incomplete sequence light on.

Changed to memory fundamental.

Rev. 2 - Minor editorial changes based on CE comments during NRC Validation Week. (Used the light full description in the question to ensure understanding)

NRC - The stem of this question is in Verdana font which is different from the rest of the exam.

49 H

3 N

S No changes 50 H

3 B

S NRC 2021-06 No changes

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 51 H

3 N

S No changes 52 F

3 N

S No changes 53 H

4 X

X N

U S

Distractor A and C not plausible. A Demand = Actual, =

Demand on B, so the idea that somethings wrong on A Recirc Flow Controller not credible.

Created a new question. Raised LOD to 4.

Rev. 2-Made question (1) more direct and added minimum requirements to ensure only one correct answer per CE comments during NRC Validation Week.

NRC OK.

54 F

3 B

S No changes 55 H

4 N

S The explanation has Error! in procedure excerpt.

Fixed the Error in the procedure excerpt.

56 H

4 X?

N S

Confirm that part (2) is required RO knowledge per CNS LOIT program, especially safety limit.

This is enabling objects 11, 12, and 13 of INT0070501R27 to know the safety limits and the basis from memory. RO knowledge.

NRC OK.

57 H

4 N

S No changes 58 H

4 X

N E

S The answer to the part (2) question doesnt really make sense with the required operator action of tripping the main turbine only. If I understand the explanation correctly, the reason you only trip the turbine, and not the rx is that you DO have adequate BPV capability under these conditions, so MCPR is not at risk for being exceeded. The concept is sound but need to reframe part (2) question.

Changed the stem to 1st stage pressure rising causes 9-5-2/C-4 to clear, this changed the answer to C to trip the reactor and trip the turbine. Changed wording to ensure reason for protecting Core thermal limit is apparent.

Rev.2 - Minor editorial changes on sentence structure for question 2 per CE comments during NRC Validation Week.

NRC OK.

59 H

3 N

E S

Part (1) - not plausible that B Feed Pump would trip but not A Feed Pump when the stem explicitly states that A RFP suction pressure is 10 psig lower than B RFP. Test on the difference in trip time delay between A and B RFPs instead.

Changed first question to test on the operators knowledge of the Time delay for the RFP trips on low suction pressure.

NRC OK.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 60 F

3 N

S This is Fundamental knowledge, not higher cognitive level.

LOD = 3 not 4.

Changed the cognitive level to Memory/Fundamental.

61 H

3 N

S No changes 62 F

3 X

N E

S ADD only to A and B part 1. Ad trip(s) to stem for grammatical flexibility.

This is Fundamental systems knowledge, not higher cognitive level, because there is no synthesis of information required to arrive at correct answer.

Changed the cognitive level to Memory/Fundamental and made suggested changes.

NRC OK.

63 H

4 X

N E

S Why is it necessary to say July? Better to include some quantitative value like river temperature or something.

By using the phrasing, the entry conditions for X is met, it implies that 2 circ water pumps tripping is a direct entry condition for 2.4VAC, which is not technically the case.. the resultant lowering of condenser vacuum is the entry condition.

More precise to phrase it, The resultant plant response will first satisfy entry conditions for _____.

Added a river temperature that requires 4 CW pumps to maintain vacuum at 100% power and made editorial changes to question.

NRC OK.

64 H

3 X

X N

E S

Part 1 distractor is not plausible, unless there is some other semi-annual watchstanding requirement to maintain quals at CNS that could be plausibly justified.

For part 2, put per procedure 2.0.7 in stem.

Added [procedure 2.0.7 to the second part of the question.

Changed the first part of the question to test knowledge of positions and number of watches to maintain his license.

Changed stem to be more direct and less confusing on hours prior to the 3rd quarter based on validation comments.

Rev. 2 - Fixed typo in third paragraph. (s) to (es) per CE comments during NRC Validation Week.

NRC OK.

65 F

3 X

B S

Reference the relevant procedure granting the authority in the stem.

Added procedure 0.31 Equipment Status Control to the stem.

NRC OK.

66 F

3 X

B S

C should say worker,to match B.

In B, should IAC be I&C?

Reference the relevant procedure in the stem.

Added procedures to the stem and made editorial changes.

NRC OK.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 67 F

3 X

B E

S NRC 2021-06 a procedure change is required is not necessary to state in stem.

The question is asking about lowest classification of procedure change. Are procedure changes listed in a certain numerical hierarchy? Want to ensure lowest is actually definable. Lowest level of approval?

The excerpts from Procedure 0,4, is the first part explaining a pen and ink change? I dont see that specifically identified.

Include the references for the distractors in the excerpts at the end.

Added a statement in the stem that how classification levels are determined and list that answers are listed from highest level to lowest level of classification. Added a table showing the reviews required for each classification in the explanation section.

NRC: I do not like the level of explanation being provided to the applicant in the new stem paragraph: Per Procedure 0.4, Procedure Change Process, the levels of classification for procedure changes is based on the number of reviews needed to make the change. (Answers are listed in Highest classification to the Lowest classification).We should not be explaining to an applicant how the station determines the hierarchy of its procedure changes. Modify stem question to:

Per Procedure 0.4., Procedure Change Process, what is the lowest classification of procedure change permitted to implement this change (fewest reviews required)?

68 F

3 B

S NRC 2015-12 No changes 69 F

3 X

B E

S Reference 5.1ASD in the stem.

Added 5.1ASD to the stem.

Changed distractor C to venting the CRD scram air header due to C being able to be challenged. Change made based on validation comments.

NRC OK 70 H

3 N

S Pre-review.

No changes 71 F

3 B

S No changes 72 F

3 B

S Add, At Cooper Nuclear Station to start of stem.

Added Cooper Nuclear Station to the beginning of the stem.

73 F

2 M

S Delete that occurs from stem.

Add, At a BWR like Cooper Nuclear Station to start of stem.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Added Cooper Nuclear Station to the beginning of the stem and removed the words that occurs from the stem.

74 F

3 x

X B

U S

Isnt A partially correct, in that if LHGR is not maintained within limts, then APLHGR may not be within limits?

Same with C, if MCPR is not maintained within limits, may not PCT exceed 2200F in a DBA?

This Isnt really defining APLHGR, its testing on the impact of APLHGR. Rewrite to test on how APLHGR is calculated.

Recognized the K/A used from the exam bank didnt match the sample plan K/A. Used a question from the NRC exam bank that matched the K/A, on the definition of APLHGR.

Rev. 2 - added the words At CNS at the beginning of the stem to imply that these conditions can be met at CNS and not generically any reactor per CE comments during NRC Validation Week.

NRC OK 75 F

3 X

B E

S This question would be better written for Cooper Nuclear Station if you listed several operating limits / safety limits, and asked which one reduces the likelihood of brittle fracture failure. You could also achieve the same by asking questions about different portions of the P-T limit curve.

Developed a CNS specific question on administrative limits on brittle fracture and what condition minimizes the consequences heatup vs. cooldown.

Rev. 2 - Changed distractor from 50F/HR to the Technical Specification limit of 100F/HR based on CE comments during the NRC Validation Week.

NRC OK 76 H

3 N

S Is the CRS always responsible for reporting to the ASD?

What makes the question SRO-only is the temperature limit in part 2, which is referenced in the SRO-only justification.

Step 4.5 directs the CRS to report as the ASD Operator and assign other personnel to the other sections. So both parts are SRO only.

NRC OK 77 H

3 B

S NRC 2011-06 Reference provided No changes 78 H

3 N

S Reference provided Part 2 of the stem could be modified to shorten answer/distractors by including maintain drywell pressure below since all answer/distractors include such statement.

-Include excerpt of plotted curve, and basis for HCTL in explanation for examiner reference and verification.

Added plotted curve. Added HCTL Basis from the PSTGs NRC OK

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 79 H

3 N

S No changes 80 H

3 X

N U

S Water level EQUAL TO -2" meets the T.S. acceptance criteria so no choice is correct.

Changed the question to As Suppression Pool water level lowers, _____(1)_____ on PC-LI-12, TORUS LEVEL is the lowest level which does not require entry into TS LCO 3.6.2.2, Suppression Pool Water Level. Only one correct answer.

Rev. 2 - Reworded first question based on CE comments during NRC Validation Week.

As Suppression Pool water level lowers, when level drops below _____(1)_____ on PC-LI-12, TORUS LEVEL entry into TS LCO 3.6.2.2, Suppression Pool Water Level is required.

NRC - The answers are all in Verdana font which is different from the rest of the exam.

81 H

3 X

X M

E S

Pre-review.

-Modified question - original not attached or changes not noted

-Justification for plausibility of Part 2 distractor is not credible if an applicant didnt think that an update to the NRC for an upgrade to SAE was required, then why was a second update to the NRC made at 1929?

-The more discriminatory way of testing this knowledge is to ask directly what time the report is required to be made, not a yes/no question.

Provided the question to the justification and changed the question to just bank. No part of the answers was changed just the stem so it is still a bank question.

The reason that the 2nd part is asked the way it is the only plausible distractor would be 15 minutes to the state and locals and that is tested in another question. The part on if they dont think the SAE was required is confusing. I am not making them determine that status they are given it?

Rev.2 - Changed the question to ask a direct time and changed the stem to initial EAL to an ALERT based upon discussion with the CE during the NRC Validation Week.

NRC: The timing in the stem is not what we discussed during validation week. We discussed that the First communication to the NRC for the ALERT was made at 1815, and then at 1819 the SAE is declared. 1915 is still the correct answer due to 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> follow up.

Procedural change identified in validation - station has committed to submitting training needs request.

82 H

4 X

N E

S Answer B and distractor D appear to be missing Emergent -

should state EmergenT Risk Significant Window as distractors A and C do.

If the condition was entered due to a failure it would be an Emergent Risk Significant Window. If it is planned it is just a

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Risk Significant Window. They are being tested on this concept as well.

No change.

NRC OK 83 F

3 X

N E

S Correct formatting in question stem to avoid having the

__(1)__ extend onto a second line.

Get rid of to at start of all answers. done Why does FS/P-3 say lower to 940 psig, but PSTG explanation says 937 psig? EOPs picked a value close to the PSTGs where the BPVs would be full open but easier to remember.

This is not higher, it is fundamental memory of EOP steps and basis. Changed Added to answer for Part 2, to not only state the action but the effect to match the distractor. This is based on validation comments.

NRC OK 84 H

4 X

N U

S Question is not SRO-only. Despite the stem asking what of 2.4MC-RF says, the question can be answered solely using system level knowledge of how the recirc runback impacts steam flow and how steam flow & feed flow work to control RPV level.

Changed question what actions were needed to mitigate the transient.

Rev.2 - Changed question to what procedures(s) are required to be entered and what system is used to control reactor water level. This is based on CE comments during the NRC Validation Week.

NRC OK 85 F

3 x

X X

N E

S Pre-review.

-Restore some of the stem initial conditions that were in the pre-review version, to more clearly tie this question to inadvertent reactivity addition, which is the KA. Restored

-Replace 15 min with 1 hr for better plausibility. Changed

-Additionally, this is fundamental recall of information, not higher cognitive level. Changed Following validation removed reference to writing a Condition Report and asked directly what the Facility License Limit is.

NRC OK 86 H

3 N

S E

Alarm card 9-3-1/C-8 should be listed as a technical reference, as part 1 is answered using such reference. Procedure 2.2.9 is referenced for CS-MO-11A/12A positions, however, the question stem asks specifically the valve position IAW the alarm card. Ensure answer justification references alarm card actions, since this is what the question stem is asking.

-The SRO only justification is incorrect, SRO credit would not be given simply for knowing alarm card actions. The knowledge of the basis is what makes it SRO knowledge.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Added the alarm card to the technical references and in answer justification.

Minor stem changes to remove confusion of the Core Spray pump start causing the overpressurization of the piping based on validation comments.

NRC - Part 2 distractor, I dont really understand what this is supposed to mean, The reason this is done is to prevent directly causing a perturbation on the CS system causing RCS Boundary Leakage. What perturbation?

Part 2 revised.

87 H

3 X

N E

S Parts 2 of distractor A and C -- add subsystems after SLC in these distractors. Consider adding ONLY to answer B and distractor D, prior to B SLC subsystem.

The explanations talk about a constant discharge pressure, but the stem doesnt specify that the discharge pressure is constant. Also, if the relief valve is lifting, wouldnt pressure be cycling as blowdown occurs and the valve reseats? Or is the intent that it is stuck open? Either way, stem focus needs to be increased to clarify. Relief valve setpoint drifted and lifting preventing pressure getting higher. Can be lowered only if he opens the discharge valve to lower below the reset setpoint.

Positive displacement pump will maintain pressure constant through the valve until NLO changes conditions.

Fixed grammar. Added only to answer Band D part 2.

NRC OK 88 F

3 X

N E

S Stem says overpressure pressure protection Changed to overpressure protection 89 H

2 N

S Reference provided No change 90 H

3 x

N U

S Not SRO only. Question can be answered with systems knowledge of SGT response to high temp, and modes of operation of FP system.

Changed the question to ask what actions are required for a High temperature alarm on SGT train and the amount of time to correct a SC TS inoperable.

NRC OK 91 H

4 X

N E

S The sentence preceding (1) is a long run-on, difficult to understand. Simplify it.

Simplified as stated.

NRC: Change (1) to read closer to the LCO basis: The RBM is designed to prevent violation of the __(1)__ safety limit that may result from a single control rod withdrawal error event.

92 H

3 X

X X

B E

S Dist C - why is it plausible that with Drywell pressure increasing, Initiating Torus Spray is the correct procedural response? Also, the explanation for Dist A states that torus pressure will also rise along with drywell pressure, which

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level would make B a correct answer also. This question strikes me as knowledge an RO would be required to have.

The reason for Initiating Torus spray vice Drywell Sprays is because the action would be the initial action taken in EOP-3A for rising pressure Drywell pressure even though it is rising we would spray the torus first. It is wrong because it would not be what is entered to mitigate it. The SRO would need to understand the correct procedure to mitigate this event would be 2.4SRV and the first subsequent operator action would be to place the affected SRV to open. The second part is asking about which procedure is used to correct the issue by asking subsequent actions. And while Torus pressure rises it rises due to Drywell pressure rise and pressure being felt thorough the downcomers and the question is asking about the initial response.

No change.

Rev.1 - Made minor editorial changes to the second question to ensure that correct answer could not be challenged based on validation comments.

NRC OK 93 F

4 X

N E

S

-Add between 25.25 ft to stem.

-I dont like the stem format of testing on what a hypothetical CRS already did do (CRS has ordered ). Ask the direct question about what action is required.

Reworded question for EOP-5A to establish the level band.

-Question is Fundamental Memory, not higher cognitive level, as there is no synthesis of information required. Changed NRC OK 94 F

3 X

N U

S

-Distractors A and C both seem like functions the Refuel Floor SRO would be expected to perform. Is the Refuel SRO not placekeeping progress anywhere? Is the Refuel SRO (The senior licensed individual on the floor) not responsible to be a line of defense for human performance? I can see an argument made for both of these being correct answers, whether explicitly stated in 2.2.31 or not.

-Answer C is not phrased correctly for the procedure. Per the procedural excerpt provided in the question, the SRO is responsible for verifying the correct location and orientation of the PREVIOUSLY moved bundle, not the next transfer. The Answer as phrased implies he is verifying the next transfer to be made.

Changed the correct answer to show the words of ensure that the completed transfer was done correctly and changed distractor for the bridge operator to one of his other responsibilities to prevent any challenges to being a line of defense.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Rev.2 - Changed Answer A to read To maintain the official copy of the SNM Transfer Form during fuel moves and Answer D to read Verify the correct location and orientation of the fuel bundle that had just been moved, to ensure that the completed transfer was done correctly. based on CE comments during the NRC Validation Week.

NRC OK 95 F

3 X

N E

S

-Same comment as 93 re: filling in blank for what an Ops Manager has already done. Ask the direct question about the requirement.

- Add reference to 0-EN-OP-112 in stem to tie the question to the guiding procedure. This is a general comment.

Fixed question to be a direct question and added reference EN-OP-112 to stem.

Rev. 2 - Added 0-EN-OP-112 title to the procedure. Night and Standing Orders based on CE comments and validation comments.

NRC OK 96 F

3 X

N E

S

-Rephrase these as direct questions, not fill in the blanks.

Changed

-This is Fundamental recall of information, not Higher cognitive level. Changed NRC - The line spacing in part (1) is wider than part (2).

97 H

3 N

E S

-Remove by from Answer A, redundant. Changed

-Add space between 2.3.1GENERAL Changed

-Make the question a direct question, not a fill in the blank.

Doubly confusing since there is a blank higher up in the stem that is not part of the question. Made a direct Question NRC - There is a semi-colon after following instead of a colon.

Insert this screenshot of the referenced procedure step in the stem, the copy and paste isnt really translating well:

Same with completions steps:

98 H

3 B

S NRC 2020-08 No change

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/ M / N)

7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 99 H

3 N

S Pre-review.

Rev. 1 - Changed question based on CE comments.

Rev. 2 - Changed JIC to Operations Support Center based on Operations Representative comments and validation comments.

Operations Support Center is in Verdana font, different.

100 F

3 X

N E

S Rewrite the final sentence to exclude "shift manager" due to possible cueing to first part.

Removed the words from the Shift Manager from question 2.

Rev. 2 - changed sentence to All required emergency facilities have been manned and activated. to prevent cueing on question 99.

NRC - OK.

GENERIC Comments:

-Only 1 question on the SRO exam has answer A - highly unbalanced. Rearrange several questions answers from D and C to get more balance in A. Fixed. New numbers at bottom of sheet.

-17 questions on the SRO exam are tied to 43.5. Look through these and see if any can be plausibly tied to something else, in particular 43.1/.3/.6/.7. A question can usually be tied to various categories. Fixed

-When developing a questions pedigree, only credit a single category of 41.x or 43.x. When I go through I have to tally these up, and multiple documented credits confuses the situation. Do not list a 43.x credit for an RO question, and do not list a 41.x credit for an SRO question. The category the question is binned into should be determined by the question AS-WRITTEN, it is not obligated to be credited to what is in the NUREG K/A catalog. Fixed only one per question New numbers at bottom of sheet.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length).

This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.

41.1 4

43.1 5

41.2 5

43.2 5

Form 2.3-5 Written Examination Review Worksheet CNS 2022-06 41.3 1

43.3 0

41.4 8

43.4 3

41.5 7

43.5 9

41.6 7

43.6 1

41.7 21 43.7 2

41.8 2

41.9 0

41.10 15 41.11 1

41.12 1

41.13 0

41.14 3

Current SRO mix is SRO-A 5 SRO-B 6 SRO-C 7 SRO-D 7