ML22354A252

4_RBS-2022-12 Form 2.3-5 Draft Written Examination Comments (Final)

ML22354A252
Person / Time
Site:	River Bend
Issue date:	12/14/2022
From:	Heather Gepford NRC/RGN-IV/DORS/OB
To:	Entergy Operations
References
Download: ML22354A252 (1)

Text

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 1

M 3

N S

NRC - OK 2

C 3

x N

E NRC - Make the stem question more direct:

Based on the given indications, which one of the following valves automatically closed AND caused RHR A Pump to trip?

Updated stem to be more direct.

NRC OK 3

M C

3 X

N E

NRC - The question is technically sound, but the stem needs to be revised to more directly reference EOP-1 Adequate Core Cooling criteria and phrasing, rather than implicitly expecting them to reference EOP bases, which is usually considered SRO knowledge.

Rephrase stem and answers as follows:

Per EOP-1 Adequate Core Cooling Criteria, under these conditions adequate core cooling exists by which mechanism?

A.

Core Submergence B.

Steam cooling with Injection C.

Spray Cooling [correct]

D.

Steam Cooling without Injection Also, this is higher cognitive level, not memory.

Updated stem and distractors as requested. Changed the cognitive level to comprehension.

NRC - Add Criteria after Adequate Core Cooling in stem to be precisely correct with name of table.

Updated stem as requested.

NRC OK 4

C 3

X N

E Replace Part 2. It is too simplistic and questionably accurate: while erosion might be a concern over the long term, the immediate concern is vibration causing damage to pump internals like seals, bearings, shaft,

etc, Changed Part 2 as requested.

NRC - pull the word excessive out of the stem and put it in A and C part 2, as you wouldnt typically say excessive overheating so it cues the answer.

Updated stem and answers.

NRC OK 5

C 3

B S

NRC 2018 Q7 Good practice for questions with screenshots of indications like this, is to include a reference page showing lit vs unlit indications. Someone is going to ask if those lamps are lit or not.

A key will be attached with references.

NRC OK

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 6

C 2

N S

NRC - OK 7

M 3

X N

E Tier 4 Generic Put common indicators in the same order in each choice. Or alternately, put the choices in the stem and make the answer selections a series of numbers, i.e. 1, 2, and 4 Revised distractors as requested.

NRC OK 8

M C

3 X

N E

Left-justify part 1 answers.

Specify in the explanation what procedure directs bypassing IRM A, to satisfy KA.

Stem, how is it plausible that All IRM are on range 6 reading middle of scale with IRM A voltage almost failed low? Is this supposed to be a subsequent failure? Better to have voltage degrade to something like 60% of normal.

C AND D PART 2, read vs reading.

Left justified part 1 of answers Added procedural guidance (EN-OP-115) for bypassing NIs Changed IRM voltage failure to 60% of normal voltage Changed C and D answers part two from reading to read Changed to comprehensive analysis.

NRC OK 9

M C

3 X

N E

Part 2 too similar to part 2 of question 8. Revise one or the other.

Left justify part 2.

Changed part 2 of answers to IRM indications due to loss of power to the RPS Bus to differentiate from question 8.

Changed part 1 to a loss of normal / alternate power supply to RPS bus to go with part 2.

Left justified part 2 Changed to comprehensive analysis.

NRC - Add the following additional sentence before the picture: Subsequently, the following alarms are received: (or equivalent language).

Delete following the event from stem sentence, redundant.

Updated stem.

NRC OK

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 10 M

C 3

N S

Left justify part 1.

Left justified part 1 Changed to comprehensive analysis.

NRC OK 11 M

C 3

B M

S NRC 2021 Q13 This is a modified question. Included the original question.

Changed to comprehensive analysis.

NRC OK 12 C

3 N

E For distractors C and D, what does stabilize at the original level mean? Original speed? Something else?

Enhanced distractors C and D to stay speed NRC OK 13 M

2 N

S 14 M

3 X

N E

Cant Prevent a loss of coolant inventory be argued partially correct, since NSSS actuation prevents a loss of coolant from bypassing containment?

Updated distractors B and D to be more specific and less correct.

NRC OK.

15 M

2 X

N E

Line up the initial conditions with the rest of the stem.

Specifically state that the operability test is to be performed.

Clarify question that All operation of the Division 2 SRVs are performed at Made editorial revision to the stem as requested NRC OK.

16 M

2 N

S 17 M

3 N

S 18 C

3 X

N E

Line up initial conditions with rest of stem.

Modify stem to What electrical transient would cause the following response?

Made editorial revision to the stem as requested NRC OK.

19 M

3 X

N E

-The explanation states that under normal electric lineup, SWGR 1C is powered by 1A, but the power distribution alignment check excerpt provided in the highlighted references shows NNS-SWGR1C aligned to SWGR1B, not SWGR1A (breakers 24 and 28 closed).

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level

-I prefer to have systems named at least once in the stem, with the system abbreviations in parenthesis for clarity.

Added noun names to the components in the stem and revised explanation.

NRC OK 20 C

3 B

S NRC 2021 Q21 21 M

3 N

S Bold Div 2 in stem.

Made editorial change as requested.

22 C

2 X

X N

E

-Stem: dont tell the applicant why the DG output breaker failed to close, just tell them that the DG started and is running at 58 Hz and a value <4160V but within acceptable range, and the output breaker remains open.

-Include a procedure reference in the stem.

-Edit stem to What minimum actions must be taken

-Why is it plausible in distractor D that frequency would automatically increase after breaker was closed?

-I think incorporating an applicant decision point about whether Voltage or Freq need to be adjusted would make this a better question.

Modified the stem so the applicant must determine what prevented the output breaker from closing and changed the distractors to make the choices between voltage and frequency.

NRC - 4100VAC is not sufficiently discriminatory in stem for the distractor. Modify to 3900VAC.

There is technically no correct answer the way this is written because the breaker will automatically close when frequency is raised to 58.67 Hz, not 60 Hz.

Modify the choices as follows:

A.

Manually raise frequency to 60 Hz and then manually close the output breaker.

B.

Manually raise voltage to 4160 VAC and then manually close the output breaker.

C.

Manually raise frequency to at least 58.67 Hz and verify the output breaker closes automatically.

D.

Manually raise voltage to at least 4014 VAC and verify the output breaker closes automatically.

Updated stem and answers. NRC OK

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 23 M

2 X

N E

Stem: which bus?

Added to the bus noun name to the stem NRC OK.

24 C

2 X

X N

E Not keen on all IAS/SAS compressors as a distractor. I can see it being argued partly true.

Change it to directly ask if the Diesel Air Compressor is expected to start automatically or must be manual.

Changed Part 2 of the stem to ask about the starting on the diesel air compressor manually or verify an auto start.

NRC - The part 2 sentence in the stem is a statement, not a question.

Part 2, eliminate the reference to 107 psig, as the manual start is not tied to a specific pressure.

Rephrase to As IA header pressure continues to lower, the Diesel Air Compressor should be Updated stem.

NRC ok 25 C

3 N

S 26 M

3 N

S 27 C

M 3

X N

U Parts 1 and 2 language doesnt make any sense.

These indications are the result of CRD Flow Control Valve A.

The resulting effect of the above indications are due to a full closure of CRD high temperature alarm.

This should be higher cognitive level.

Changed level to COMPREHENSION NRC OK 28 C

3 X

N E

Confirm you would be allowed to continue a RX startup with CRD Pump B tagged out.

Remove of in all of the answer choices.

Should stem sentence be accumulator fault on uninserted control rod Add reference to the directive procedure in the stem of the question, e.g, per Confirmed NO restrictions to startup with only one CRD pump available Removed of in all four answer choices Changed of to on uninserted rod Added per ARP CRD PUMP A or B Auto Trip and T.S. 3.1.5 to stem of question

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC OK 29 C

3 N

S The explanation for part 1 on A talks about 4 notches of movement but the distractor is 8 notches.

The stem is based on position indication which is position 8 which is the same as 4 notches.

NRC OK 30 C

3 N

S 31 M

3 B

N S

NRC 2021 Q30 Technically adequate, but review against Question 2 for overlap of content.

Modified question to differentiate content from question 2 by changing stem to a spurious Div 2 Main Steam Tunnel Temperature Isolation signal NRC - OK - I consider this a new question with the mods.

Updated to new.

32 C

3 X

N E

Add The to A and C part 1.

Specify cavitation of which pump in part 2 distractors.

Added the part 1 answers A and C Added SFC-P2A as pump to cavitate NRC OK 33 C

3 X

X N

E Stem should read remain below a MAXIMUM of 25 psig if I understand the system correctly.

Added MAXIMUM to the stem.

NRC OK 34 C

3 X

N E

(1) AND (2) are swapped in the stem incorrectly.

Swapped (1) and (2) in the stem of the question NRC OK 35 C

3 X

N S

Replace resulted in with caused Replaced resulted in with caused NRC OK 36 C

3 X

N E

K/A Applicability field states that this question tests on surveillance procedures which is incorrect.

Distractor D explanation incorrectly talks about CRE envelope for part 1. No part of the explanation talks about why Emergency Release is wrong, or plausible.

Updated the K/A applicability explanation to better match the given KA.

Added explanation on why Emergency Release is wrong answer.

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Fixed explanation for answer D part 1 to correctly talk about Emergency vs Alternate release.

NRC OK 37 C

2 X

N E

Rephrase stem to Reactor Water Cleanup suction paths from the reactor are located at Change Stem to read Reactor Water Cleanup suction paths from the reactor vessel are located at _(1)_.

Changed LOD from 3 to 2 NRC OK 38 M

3 X

N U

Need to reference the procedural requirement directing the action in the stem. Per procedure..

Distractor A explanation is cluttered and difficult to understand.

Stem: The Shutdown Protection ?

I dont see any basis in the explanation for why part 2 is the correct choice.

prolong uncovering the core is the opposite of what is desired.

Added IAW AOP-51 in Part 1 of the question stem and rewrote the distractors so the applicant will have a better understanding of what is being asked of them.

NRC OK 39 M

3 X

X N

E Generic: need to cite the applicable procedure in the stem of question when testing on procedurally-required responses. This improves defensibility of question.

The SR 3.8.1.10 excerpt justifying the answer and distractors for part 2 is the bounding criteria for performing a loss of load surveillance but does not actually state what the load limit is. Provide a reference more directly stating the load limit.

Added AOP to the stem. Added references for the load limits for Div 1/2 Diesel Generators.

NRC OK 40 C

3 X

N E

THE GROUND IS ON THE (1) SIDE Made editorial change as requested NRC OK 41 C

3 N

S 42 C

3 N

S Spell out what RACS and RGDS are.

Made editorial revision to the stem as requested.

NRC OK 43 M

2 X

N E

Explicitly state the procedure the crew has entered to respond to the MCR fire. Someone is going to argue

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level that you could use H13-P601 to start RCIC initially before evacuating the CR so this needs to be ironclad.

Added Per AOP-31 to the stem.

NRC OK 44 C

M 3

N S

This is memory/fundamental systems knowledge.

Changed cognitive level from Comprehension to Memory 45 M

2 X

N U

Part 1 distractor is implausible that rods would drift out on a loss of IA and the justification that the applicant may confuse how the CRD system responds is inadequate.

Rewrote Part 1 of the question to ask how the SDV vent and drain valves fail.

NRC OK 46 M

C 3

X N

E State Per EOP-1D Adequate Core Cooling Criteria in stem.

Added IAW EOP-1D, DECAY HEAT REMOVAL, to the stem. Changed to comprehensive analysis.

NRC OK 47 M

2 N

S Pre-Review Initial conditions should mostly be complete sentences, not partials like Offloading and transferring Irradiated Fuel. Generic.

Replaced Offloading and transferring Irradiated with the movement of Irradiated Fuel is in progress in the initial conditions NRC OK 48 C

3 N

S List the choices in the stem with numbers assigned to them and put the numbers in the choices in order. As submitted the choices are all mixed up in different orders which is very confusing.

Realigned choices in a more consistent pattern.

NRC OK 49 C

2 N

S State Per step RP-2, RPV Pressure in stem.

Added Per step RP-2, RPV Pressure to stem.

NRC OK 50 M

C 3

X N

E Stem: The following indications are observed State title of EOP-2.

Replace IS NOT with IS NOT YET.

Confirm that Part 2 is RO knowledge per accredited ILT program.

Changed STEM to read The following indications are observed Added EOP-2 title to STEM replaced IS NOT with IS NOT YET for part 1 answer Verified Part 2 is RO knowledge and added AOP-35, SORV objective to objective section from ILT Changed to comprehensive analysis.

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC OK 51 C

M 2

X N

E Replace warranted with required in stem. Specify the EOP that requires the SCRAM in stem.

This is a lower cognitive level memorization question.

Rewrote question to better suit the K/A NRC OK 52 M

C 3

X N

U Part 2 distractors A and B are true. Adding water to the SP does maintain margin to HCTL because it increases the margin to the UNSAFE region of curve Change part 2 distractor from margin to HCTL to above SRV Tailpipe Level Limit.

Changed to comprehensive analysis.

NRC OK 53 C

3 X

N S

Is the applicant expected to specifically know by memory that on a loss of 1 steam flow input, RWL will stabilize ABOVE the Scram setpoint? Knowing that it lowers and stabilizes is one thing, knowing specifically WHERE it stabilizes is another.

This topic is covered in ILT lesson plan RLP-LO-STM-107 for loss of one steam flow signal vs total loss.

ILT class has been tested on loss of one feed flow signal on System 4 RO/SRO exam.

NRC OK 54 C

3 X

X N

E

-Its not clear to me if the white lamps are supposed to be lit or not - someone is going to ask the question.

Solution is to either write lit on the lamps, or include an index with all the lamps shown both lit and unlit for reference.

-Part 2 is questionably accurate. The question asks for the reason for manually scramming when a scram setpoint has been exceeded without automatic action occurring. But the answer and justification for part 2 is the purpose of the lift setpoints on the SRVs. These are separate concepts.

Index to be generated for entire exam.

Added justification to explanation for RPS Scram Instrumentation on High Reactor Pressure and included T.S. Bases 3.3.1.1 to highlighted references.

NRC OK 55 C

2 N

S Ensure picture is large enough to read.

River Bend response:

Enlarged EOP-3 DRMS screen NRC OK 56 M

2 X

N E

The pedigree does not state what the EPE/APE is for this question. (Plant Fire on Site).

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Is there a station fire protection procedure that can be cited for this requirement, other than a training lesson plan?

Added Plant Fire on Site to the pedigree A search was performed of all Fire Protection procedures (both site and Corporate) and no requirement other than the training material as to what is preferred.

NRC - Make LOD=2.

Changed to LOD 2 NRC OK 57 C

2 N

S 58 C

M 3

B S

2021 RBS [What Question?] [61]

-Pedigree does not state what question this was on the 2021 RBS exam.

-This is not higher comprehensive LOK, it is memory of EOP-2.

Added Question 61 to pedigree Changed LOK to Memory in pedigree NRC OK 59 C

2 N

S The pedigree does not state what the EPE/APE is for this question.

Include Per AOP-24, the ATC operator should to stem.

Added Inadvertent Reactivity Addition (EPE/APE) to K/A statement Added Per AOP-24 to stem Changed LOD from 3 to 2 NRC - make distractor B first letter lowercase like the other choices.

Fixed B.

NRC OK 60 C

3 N

S

-Its not clear to me if the white lamps are supposed to be lit or not - someone is going to ask the question.

Solution is to either write lit on the lamps, or include an index with all the lamps shown both lit and unlit for reference.

-EPE/APE is Inadvertent Containment Isolation.

Index to be created for lamps for entire exam to avoid confusion Changed font color for isolation to match inadvertent containment for EPE/APE.

NRC OK 61 C

3 X

N E

-Part 1 is not a complete sentence.

-Part 2 has received AN isolation Added required to end of part 1 stem Changed part 2 stem from and to an

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC OK 62 C

3 X

N S

Non-Embedded Reference Provided Improve the verbiage in this explanation: Although SR 3.6.4.1 is NOT met and require LCO entry for Secondary Containment, Auxiliary Building Pressure is NOT and EOP-3 entry requirement. Secondary Containment is in override SC-1 for EOP-3:

Im not seeing what the reference to SC-1 in EOP-3 means.

The reference to SC-1 states If STBY Gas cannot restore and maintain Secondary Containment (which AB is secondary containment) below 0.0 inches of water ANDthen it allows the use of enclosure to restart HVAC to maintain your secondary containment at negative pressure. This is what makes the distractor plausible because although AB pressure is a T.S. requirement AND listed with a specific value in EOP-3 it is NOT and entry requirement.

Added the verbiage to explanation as to what the reference in EOP-3 means to plausibility as to why EOP-3 entry is plausible distractor.

NRC OK 63 C

3 B

S NRC 2019 Q65 64 M

3 N

S Bold PRIOR in the stem.

Made editorial change as requested.

NRC OK 65 M

2 X

N E

Bold maintain and regain in stem.

Part 2 of stem should say a/an.

Made editorial change as requested.

NRC OK 66 M

2 X

N E

Reference Per PID-00-02A, Engineering P&I Diagram Drawing Symbols & General Notes in the stem of the question. Also provide this reference to me.

Made editorial change as requested.

NRC OK 67 M

2 X

N E

Stem uses CN but the reference provided uses EC primarily. Is there a reason to cite one or the other?

Corrected the references section NRC OK 68 M

2 X

B E

NRC 2016 Q73

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Change part 1 distractor to either 100 mrem/hr (HRA) or 500 rad/hr (VHRA), which are preferable to a non-existant 500 mrem/hr limit.

Change part 1 distractor to 100 mrem/hr (HRA) for 500 mrem/hr.

NRC OK 69 M

3 N

S 70 M

2 X

B?

E

-Is this New or Bank? The Pedigree and Written Exam Breakdown do not agree.

-Need GOP-1 Reference.

-The stem is inadequate without referencing stable period.

-Reframe stem to ask Per GOP-1, Plant Startup, when the reactor is declared critical, neutron count rate should be increasing at an (exponential vs linear) rate, without further rod movement. A steady, reasonable period post-criticality should be (15 to 50 VS 50 to 150) seconds.

Rewrote the stem to reference a stable period.

NRC OK 71 M

3 X

B E

-Change A nuclear power plant to Riverbend Station.

-The explanations all state that it has been 2 months since last power operation but this is incorrect. The stem states it has been 2 weeks since shutdown.

Revise explanations with this information. Also provide some backup info from your GFE training materials.

Changed stem to say River Bend Station and added references.

NRC - Explanations still incorrectly state it has been 2 months since shutdown - see highlighted above.

Fixed explanation.

NRC OK.

72 M

2 B

S Simply pasting GFE bank questions without any supporting references is not adequate. Provide RBS-specific references to justify the answer and distractors.

Added references NRC - If an applicant chooses evenly burn up the control rods, and subsequently argues that there is nothing in the supporting documentation that differentiates either fuel or rod burnup as the primary

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level purpose of control rod program changes, how will the station respond to that argument?

Changed distractor and rearranged answers shortest to longest.

NRC OK 73 M

2 B

S 74 M

3 X

B E

The explanation doesnt talk about MAPLHGR, it talks about the basis for the 2200F cladding temp limit in a LOCA. Provide site-specific references to explain MAPLHGRs specific relationship to it, and how it doesnt have a relationship to the distractors.

Added references as requested.

NRC OK 75 M

3 X

B E

Cite the reference you pulled the explanation from.

-Distractor B is going to be rough. While I get that the total thermal expansion is the idea that makes it wrong, its going to be hard not to bite on it when Thermal expansion is generally a core reason for warmup limits. I recommend a different distractor.

Replaced distractor B NRC - The explanation for new distractor B was not updated to reflect edit.

Updated explanation for B.

Nrc ok 76 C

3 N

S Pre-Review First part of distractor D analysis is incomplete. Only says, Part 1 of the question is Edited/completed distractor D analysis NRC OK 77 C

3 N

B S

Pre-Review Imbedded references provided. This question is not new. It is essentially question 76 from the 2010 NRC exam, with imbedded references added.

Modify stem to, which of the following actions should first be directed by the CRS?

Changed stem to which of the following actions should first be directed by the CRS?

Changed from NEW to BANK question in pedigree Question was changed at request previously.

NRC OK 78 C

3 N

S Modify part 1 to something similar to, which of the following procedures are required to be entered in response to this event?

Part 55 content says 43.11 which doesnt exist.

Should be 43.7.

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Made the editorial changes requested and changed the Part 55 from 43.11 to 43.7 NRC OK 79 C

3 X

N E

Pre-Review Non-embedded Reference Provided [Should be]

1)

Ensure part 1 of A and C match (rejecting vs reject). [Repeat comment from free review - not corrected]

2)

Distractors B and D part 1 need a verb -

establishing

3)

Stem: Vales should be Valves

4)

SPL-2 requires that the SP inventory can be retained in radwaste, but nothing in the stem provides that information.

5)

This should be categorized Reference Provided and the 2 graphs should be included in an attachment. Embedding them in the body of the question is really awkward.

Added Reference Provided to the stem. Changed rejecting to reject to match part 1 A and C.

References added to list for applicants. Added establishing to distractors B and D part 1. Changed vales to valves in stem of question. Added Radwaste available to accept any rejected water inventory Removed graphs from question and placed with reference material as handouts with other reference material and marked with Reference Provided NRC OK 80 C

3 N

S 81 C

2 X

N E

I dont see how it is plausible that an STA can be responsible for assuming Emergency Director role and classifying emergencies unless it is a requirement that they be licensed SROs at RBS. Explanation doesnt provide valid plausibility for this distractor.

EIP-2-001, CLASSIFICATION OF EMERGENCIES is repeated twice in stem.

Rewrote Part 2 of the question for better plausibility.

NRC OK 82 C

2 X

X N

E Pre-Review Stem part 1: Is it a manual scram or an automatic RPS-initiated scram?

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level Im confused about this question. Correct answer part 2 states RPS only, but the explanation says both parts of Encl 12 (RPS and ARI) are needed to reset the scram. The justification for why ARI is not correct is from the bases stating ARI initiation at RPV Level 2 is desired to initiate a reactor scram But Level 2 has already been exceeded and ARI presumably actuated, so this point seems irrelevant because the question is asking what is required to Reset the Scram. D appears to be the correct answer by this logic.

The justification for why 18 is a plausible distractor is inadequate. if the applicant gets confused is not an acceptable justification. What is significant about 18?

Also, were not asking about how high SDV level gets after a scram, were asking about the scram setpoint.

This language is a holdover from the original free look question and it has not been revised for the edited question.

Rewrote the explanation statement to justify the correct answers.

For part 1, not necessary to couch this in terms of a manual scram, implying that an automatic scram failed to occur. If were just testing on recognition of the automatic scram setpoint, ask that question directly: An automatic scram is expected when SDV level exceeds the setpoint of ___1___ to ensure Updated stem.

NRC -OK 83 C

3 N

S 84 C

2 X

N E

Pre-Review Non-embedded Reference Provided Is the stem requiring the applicant to implicitly determine that the declaration is a GENERAL EMERGENCY, or is this something that should be provided?

Label the 3 options (EVAC, PAR, ED) 1, 2, 3 in stem.

Then make the choices, 1 only, 1 and 2 only, etc.

Revise stem question to Which of the following actions must be taken There is no 55.43 reference provided in the pedigree.

Rewrote the stem with the answer choices numerically listed and added a timeline of release rates so the applicant knows what is being asked exclusively for that level of classification.

NRC OK

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 85 C

3 X

N E

Pre-Review Should probably ask IAW EOP-3 Bases instead of IAW EOP-3 since this information is not contained in EOP-3.

Changed stem to read IAW EOP-3 Bases NRC OK 86 M

2 X

N E

Pre-Review Should probably ask IAW with the tech spec bases, because that is where you find the information about manually starting.

Changed part 1 of stem to read IAW LCO 3.5.2 Bases NRC OK 87 M

2 X

X N

E Stem should state initial suppression pool level is something under 20 ft 0 in, otherwise part 1 A and B not really plausible.

Explanation doesnt provide justification for why 200 is credible distractor.

Stem already states initial level at 19 11 Added distractor information in part 1 A and B as to why 20 0 is plausible distractor.

NRC OK 88 C

3 X

X X

N U

The question has (1) twice instead of (1) and (2) in stem.

If APRM B is also inoperable, the question should be revised to state something like in addition to APRM B

)

Im confused as to why B, APRM D, is the correct answer. The explanation states that APRMs B and F are co-located in Panel P670. So, why is APRM F not the correct answer?

Not SRO-only knowledge. Part 1 is systems knowledge of APRM arrangements, and for part 2 an RO (as well as any maintenance worker) is expected to know who is authorized to approve tagouts.

Rewrote the question to meet the SRO only requirement.

NRC OK 89 C

3 X

N E

Im not clear about why the control room is being evacuated. The fire is in EJS-SWG2B, and Distractor D explanation specifically says a fire did occur in a safety related area outside the control room. Why is the control room being evacuated for a fire outside the control room? Clarify.

Changed the stem of the question to read a plant in the aux/control bldg. has resulted in the following.

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC - stem: has resulted should be have resulted.

automatics should be automatic actions Corrected stem.

NRC OK 90 C

2 X

N U

S I dont see how this is SRO-only. Interpreting board indications and taking manual action to start a pump that failed to start automatically is a fundamental RO competency.

Changed question to require applicant to assess event (1.68 DW psid) and indications (total loss of CCP) and direct appropriate subsequent actions and time requirement to meet SRO level knowledge.

NRC - stem: add comma after indications.

Part 1: add to avoid pump damage to end of sentence.

Part 2: Rephrase to IAW AOP-11, Loss of CCP, the CRS is required to direct that both Reactor Recirc Pumps be [tripped, only / both tripped and isolated.]

Updated stem and answers.

NRC OK 91 C

3 x

N E

instead of SAPs, I like Alternate Level Control better.

More plausible than going straight to Severe Accident Procedures.

The spacing is excessive between the two parts of the answer.

Changed Part 2 distractor from SAPs to Alternate Level / Pressure Control NRC - OK 92 C

3 X

N E

Need to add a (1) to the first stem question.

Consider if 6 feet 9 inches is a better distractor than 8 feet 2 inches, since it is also in the TR surveillance requirement.

Changed distractor from 8 ft 2 in to 6 ft 9 in. Both values are in TSR 3.9.12 NRC - OK 93 M

3 X

N U

S Good tier 3 generic question but has nothing to do with Radwaste.

Use this question as-written for Question 95.

Choose a different generic K/A for this question, and write it to the RADWASTE system.

Changed question to K/A G2.3.14 for applicant to have knowledge of EOP-3 entry requirements as well as EAL rad release requirements for Radwaste Rad Monitor elevated levels for Radwaste spill

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC - Part 1, add entry to stem statement.

Updated stem. NRC OK 94 M

2 N

S Pre-Review 95 M

3 X

B N

U 2021 NRC Q84 This is not a Tier 3 generic question. It was a Tier 1 question on the 2021 exam, and remains so.

Replace this K/A (G2.1.7) with G2.4.5 and use the question you submitted for 93, as written.

Replaced question with question 93 and exchanged K/A from G2.1.7 to G2.4.5 NRC - OK 96 M

3 N

S State name of EN-WM-104 in stem.

Added On Line Risk Assessment Title to EN-WM-104 in the STEM of the question.

NRC - OK 97 M

3 N

S 98 M

3 X

X N

E Is Part 1 of A and D supposed to mean the same thing? They are worded somewhat differently - D should have the same language as A.

Im confused by the documented actions in ADM-0054 and how the answers and distractors are constructed.

ADM-0054 has 3 requirements listed:

1.

Two independent samples obtained and analyzed - [OK, this is part of the correct answer.]

2.

Two technically-qualified members of the chemistry staff independently verify the release rate calculations. -[This is NOT listed as part of the correct answer, D.

Therefore, none of the choices list the MINIMUM requirements to perform a release]

3.

Two technical qualified members of the Radwaste staff independently verify the discharge valve lineup. [Answer D says 2 OPERATORS perform this - this seems to be more stringent than the instruction, which only requires Radwaste staff conduct it, unless RBS requires all members of the radwaste staff be operators. Even then,

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level not all operators are qualified Waste Control Operator.]

The explanations further discuss 1 SRO review and authorization required for release, but this is not included in the correct answer. A lot of work required to make the language of this question PRECISELY CORRECT, with 1 fully correct answer.

Changed part 1 of A and D wording to match exactly Removed any reference to SRO authorization as this is not listed in Admin 54 of TRM.

Added all three of required actions to the distractors and correct answer with varying the number required for the distractors.

Added Radwaste to Operator to indicate operator must be Radwaste qualified.

NRC - having the numbers in parentheses on each line is confusing since we use that for multi-part questions. Replace (1) with ONE and (2) with TWO.

-Begin stem with In addition to SRO authorization,

-End stem with per ADM-54, Radioactive Liquid Effluent Batch Discharge Updated stem and answers.

NRC OK 99 M

3 X

N U

S The question would be fine for E/APE specific K/A, but it is not Tier 3 generic.

Changed question to more generic Tier 3 question about procedure hierarchy for AOPs and EOPs and Entry into SAPs requiring an EAL declaration NRC - Part 2 stem, replace emergency with General Emergency.

Updated stem.

NRC - Stem was incorrectly edited.

Part 1 stem should still say During emergency conditions once Severe Accident Procedures Part 2 stem should say require declaration of General Emergency 100 M

2 X

N E

Pre-Review Shorten last sentence in stem to simply, Per EIP 002, Classification Actions, the FIRST notification MUST be transmitted no later than ___.

Changed last sentence in stem to Per EIP-2-002, Classification Actions, the FIRST notification MUST be transmitted no later than ___.

Changed LOD to 3 from 2

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level NRC - OK

Form 2.3-5 Written Examination Review Worksheet Generic Comments:

Last 2 NRC exams: 2021 and 2019 5 Questions on the RO exam are Bank repeats from the last 2 exams, which exceeds the limit of 4 required by Form 2.3-4 Written Exam Quality checklist Item 4. Modify exam to satisfy this limit. Question 11 was modified and included the bank question. NRC -OK There are 39 lower and 36 higher cognitive level questions on the RO exam as submitted, based on the Written exam breakdown spreadsheet. Needs to be 50-60%

higher cognitive level. Form 2.3-4 Written Exam Quality checklist Item 7 incorrectly documents 38 lower and 37 higher cognitive, which still would not satisfy this requirement. Some of the questions are not actually at the documented level, so re-evaluate, edit spreadsheet, and modify exam to get within limits with margin so that someone cannot question the cognitive level of 1 or 2 questions and claim the RO exam did not actually meet requirements. Added 7 comprehensive analysis designations.

NRC -OK Form 2.3-4 Written Exam Quality checklist now requires Y/N entries in the columns, not initials. Edit form to include this, along with updated quantitative attributes as discussed above.

Several questions do not include the name of the system or EPE/APE in the K/A statement.

Where is Level of Difficulty documented for each question? Needs to be a field in the pedigree. Level of difficulty is located at the beginning of the question in the first box.

NRC - See it now.

Form 2.3-5 Written Examination Review Worksheet

[NRC Note: All questions are assessed to have been modified to Satisfactory Status. Original status left for historical review.]

Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.