Text

File 2A3 Robinson 2021-301 Exam Administrative Items Delayed Release.Pdf
	ML22011A173
Person / Time
Site:	Robinson
Issue date:	01/11/2022
From:	; NRC/RGN-II
To:	;
References
	Download: ML22011A173 (95)
	v • d • e

ML22011A173

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

JPM Comments Generic Comments:

The second part of RO Admin JPM A3 is LOD=1 and will require modification. See comments below.

Excessive overlap between SRO Admin JPM A3 (liquid waste release) and SRO Q98.

One of these test items will need to be replaced. See comments below.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments RO CONDUCT OF OPS (A1a): RO Perform a Rod Position Channel Check For the provided data sheet: do the Robinson Group Counters include a decimal position? If so, please add either a.0 or.5, as appropriate, to the given data.

Also need to add a decimal place to the M-8 IRPI to ensure consistency between the given parameters/information.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments RO CONDUCT OF OPS (A1b): RO Short Term Relief No substantial comments identified. JPM appears S at this time.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments RO EQUIPMENT CONTROL (A2): RCS Leakage Calculation Should there be bounds on the values of Leakoff Collection Tank, RCDT level, and PRT level if an applicant chooses to use the curve book graphs instead of the provided tabular information? Please add and input these bounds into the key information.

For the Task Standard - add the actual values plus or minus the above bounding data for the key parameters. As an example, consider the following: The operator will complete and 2 of OST-051 IAW the attached KEY, and determine total RCS leakage to be 2.12 gpm; total identified RCS leakage to be 0.56 gpm; total non-RCPB leakage to be 0.15 gpm; and total unidentified RCS leakage to be 1.41 gpm.

Based on this determination, the operator will identify that the unidentified RCS leakrate exceeds that allowed by TS LCO 3.4.13.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments RO RAD CONTROL (A3): RO Determine Time to 200 F in the SFP/Documenting a Radiation Monitor Setpoint Change Too much teaching in the cue in the first bullet; just state Predict the Time-To-200 F in the SFP in accordance with AOP-036.

Based on the Curve 7.23, it appears that the time to boil should be 13.5 plus or minus 0.25 - the triangle data point is not near the 13.0 or 14.0 lines. Electronic references can zoom in as much as needed for accurate determinations.

The second part of the JPM is LOD=1 because the applicant is simply copying information that is part of the given information onto an administrative form. One suggestion might be to actually do this JPM in the simulator and have the applicant change the RM-5 setpoint and then correctly document. You dont have to use this recommendation, just a thought; in any case, second part JPM will require some adjustment. JPMs that are simple one-step JPMs are not allowed.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments SRO CONDUCT OF OPS (A1a): SRO Determine Reportability and Narrative Log Entry Requirements We need to delete the 0625 cue that the SM believes the event is reportable, teaching in the JPM.

Another reference for this condition that seems to be appropriate is procedure AD-SY-ALL-0150. Please include appropriate steps from this procedure as potential correct answers.

Recommend splitting up the third bullet of the initiating cue into two separate bullets: (1)

Make a Narrative Log Entry for the Event (if required). (2) Make a Narrative Log Entry for the Event Notification (if a notification is required). Non-critical part of the second log entry would be to record the assigned EN number set by the NRC.

Another potential answer would be to write a Condition Report. Please include as part of the answer key. Would this be a critical step?

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments SRO CONDUCT OF OPS (A1b): SRO Evaluate Overtime Eligibility We need to add something like the following to the initiating cue and JPM guide: For any operator who CANNOT be held over for two hours, state the reason why (which work-hour limit would be violated). Making the correct call for the correct reason should also be critical steps in the JPM.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments SRO EQUIPMENT CONTROL (A2): SRO Determine Compensatory Measures for Inoperable Fire Detection Equipment No substantial comments needed. JPM as submitted evaluated as S at this time.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments SRO Radiation Control (A4-S): Liquid Release Permit Review There is too much overlap between this JPM and SRO Q 98. Can we change one of these into a waste gas tank release? One of the other of these test items will need to be replaced.

On p. 4 of 9 - can delete list of available rescuers? seems to be from another JPM.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

ADMIN JPM Comments SRO EMERGENCY PLAN (A4): SRO Determine Emergency Classification The 301-1 form for SRO admin JPMS lists this JPM as also Recommended Protective Actions; however with a SAE there are no PARs.

Need to say Do NOT use Emergency Coordinator Judgement as a basis for your classification or similar statement somewhere in the initiating cue.

On the JPM cue sheet that is handed out to the applicants, recommend adding lines for the applicant to write down the EAL classification, maybe something like the following:

EAL Classification: ________________________ (to be completed by applicant)

Time of Classification: ______________________ (to be completed by examiner)

Time of Notification: ________________________ (to be completed by examiner)

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM A: Respond to a Tref Failure and Continuous Rod Motion Failure IAW AOP-001 Cueing in the initial cue: respond to plant alarms means that there will be alarms; also, we dont want an applicant to incorrectly think we want them to reference an ARP for this event (in other words, I dont want to lead anyone astray by having them focus on alarms). Better for the initial cue to state: The CRS has directed you to respond to plant events, if any.

Alternate path is weak for this JPM - for a no tell JPM like this, the initial failure is basically an initiating event. We need a subsequent issue to really drive alternate path performance. JPM performance step 3 is not critical as written because it does not work. Recommend the following change to the JPM sequence: When the operator places rod control in manual, it stops the initial rod movement. Then, in (for example) 20-30 seconds, rods start to move in again due to a different postulated continuous rod insertion failure - now its genuinely alternate path and JPM step 3 is critical.

JPM performance step 5: recommended bounding criteria here is to insert a manual Reactor Trip before Rod Insertion Lo-Lo limits are reached, which places the unit outside of design basis analysis/FSAR 15 analysis.

Validation time of 2 minutes somewhat optimistic, recommend something closer to 7 minutes? We can check/update during prep week as necessary.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM B: Isolate the SI Accumulators During Degraded Core Cooling Need to issue a procedural correction CR for FR-C.2 which has duplicated steps (14.c RNO c.2.b and c.2.c) after the license exam is complete. No impact on exam. JPM as submitted evaluated as S at this time.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM C: Place LTOP in service What is the bounding criteria for the critical step to close the block valve? For example, is it required that the operator perform all steps from memory, or if an operator takes the time the pull out the book and correctly performs the steps, is it appropriate performance? What plant parameter can we use to tie to incorrect performance of the principal critical step? (maybe pzr level going solid? PRT rupture disk?) -needs to be reachable given the conditions of the JPM.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM D: Establish RCS Bleed and Feed Understand there is a generic TCA for this action for the station; however, for this JPM, what is the adverse effect on the plant if an applicant does not get all high point vents open by 8 minutes exactly? We need to have this information to make the JPM time critical for the purposes of NUREG-1021 exams (generic TCAs are fine for requal but each time critical JPM has to be validated against the performance criteria established by that JPM).

Is it truly critical to get all head vents open? What if an applicant gets all-but-one head vent open, is that satisfactory performance? Ive seen this happen before, we need to check it against the adverse plant condition determined in the above step.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM E: Respond to Flooding in Containment - Isolate HVH-3 Slight training in the initiating cue - recommend just stating The CRS has directed you to perform FRP-J.2. no other/additional information is required.

JPM performance step 3 is not critical because there is no action required. The critical steps are JPM performance step 4 and step 5.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM F: Restore N-44 to Service No comments noted. JPM as submitted evaluated as S at this time.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM G: Swap CCW Pumps/Refill CCW Surge Tank On p. 2 of 22, JPM No. states that this JPM is 2021 Systems Audit. Was this JPM used as an audit JPM, or during the training program for the applicants?

Please discuss.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Simulator JPMs JPM H: (RO only) Transfer of Auxiliary Electrical Load From the UAT to SUTs No significant comments noted. JPM evaluated as S at this time.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

In-Plant JPMs JPM I: Isolated Leakage in the RHR Pump Pit IAW EPP-24 We need to have photos of the control power fuses/inside A RHR pump breaker for JPM step 2 to be correctly simulated. (maybe two or three photos showing different perspectives inside the breaker for accuracy) What are the specific electrical safety requirements for pulling these fuses?

What are the consequences of incorrectly closing SI-915? Would this be creating and missing a critical step? Lets discuss during prep week

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

In-Plant JPMs JPM J: Locally Establish AFW Flow From the SDAFWP and Control S/G Levels and Pressures Is it possible to obtain photos of IA-297, so that when the applicant locates it in the plant the applicant and examiner would not need to climb ladder?

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

In-Plant JPMs JPM K: Align Charging Pump Emergency Cooling IAW AOP-014 Based on experience with this type of in-plant JPM, it is necessary to allow the applicant to open the emergency equipment box in order to accurately describe how to make the connections, etc. I would not intend to actually run hoses, but being able to have connectors in hand is vital in order to being able to accurately describe how to make the connections. Photographs of the inside job box are normally not sufficient. Still do not believe this will make the JPM time-critical.

Please update task standard to include the detailed critical steps in completing the JPM.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Scenario Comments Generic Comments:

For any planned up-or down-power event, we need to provide the applicants with a valid reactivity plan to include, at a minimum: desired rate of power change, total amount of boration/dilution required, target rod height at the end of the power change.

For all scenarios, please format the TS pages so that the LCO information includes functional units (if applicable) and if possible that the LCO stuff fits together on one page (example p. 33 of 57 for Scenario 1).

Recommend adding AD-OP-RNP-1001 step 5.1.1 as a first page examiner note to all scenarios to remind the NRC about guidance to operate systems when under failure to perform or respond correctly. (e.g. Scenario 1 p. 25) This guidance is good reminder to examiners and is applicable to all scenarios.

Please add the specific operator actions whenever the procedure directs them to Delete CALO processing

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Scenario 1: Faulted S/G to FR-H.1 On p. 3 under Scenario Objectives no. 7 is the faulted S/G the A or the C?

On p. 30/57 is there a standard set of expected S/G levels for manual Rx trip (e.g., Attachment 1 of AD-OP-RNP-1001), please add as an examiner note to help us jog our memory.

Event 4 - how long does it take to reach the manual Rx trip limit if no action is taken? Please add as an examiner note.

On p. 33/57 please add detailed steps for this specific instrument when procedure says Delete CALO processing On p. 33/57 potential typo? After several bullets there is an a.?

On p. 33/57 please format TS LCO title on the same page with the additional information.

On p. 34/57 which functional units are affected? Please add.

On p. 38/57 please add additional steps in AOP-018.

On p. 42 of 57: for the TCA to be valid for this scenario, all conditions that are involved in the TCA calculation must be met in this scenario. Please verify with the TCA information that this is so; otherwise may not be truly TCA for an ILO exam.

On p. 51 of 57: how long does it take to get to feed/bleed if there is no effective operator action to restore secondary feed flow? Please add as an examiner note in case a team zigs when we think they will zag.

Turnover: green risk with a TDAFWP under clearance? Please verify correct.

P. 56/57 please add more steps in FR-H.1 in case we need to go longer P. 2 of the ES-D-1 major event description: are the applicants going to be able to terminate SI in E-2?

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Scenario 2: ATWS to LOOP Turnover - recommend just state that the next 3.8.1.1 is due in 7 hours8.101852e-5 days 0.00194 hours 1.157407e-5 weeks 2.6635e-6 months .

P. 3 objectives possible typo no. 3 is transmitter desig -474 or -494?

For Event 1, add trip limits and time to reach them as examiner note.

On p. 14/57 CALO processing detailed steps please On p. 29/57, add steps in AOP-006 (says while continuing)

On p. 30 for boration, add steps for flushing the line if the applicants choose to do so On p. 48 please move procedure title to top of p. 49 and change note on p. 45/57 Are there verifiable actions for event 8?

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Scenario 3: SBLOCA w/RCP Trip Criteria For event 3: says the indication is Tref failing low; does this cause inward rod motion? ES-D-2 is unclear and just mentions taking all FRV to manual (p. 16/57)

Could event 3 cause an unnecessary plant trip? If so, how long does it take to reach trip parameters with no effective operator action?

Please re-format the TS pages (e.g. 20-22/57) so that one full TS is at least on one full page.

On p. 21/57, please list the interlock lights that are going to be verified on or off, as applicable, to verify REQUIRED ACTION T.1 (interlock is in required state for existing unit conditions)

For Event 4/Dropped Control Rod, what QPTR was observed in validation?

Should be the same for all scenarios. Please include as an examiner note so we can evaluate this part of the TS call for this event.

RCP Trip criteria - how long would it take for the RCS to get superheated, if the operators did not trip the RCPs and they tripped due to event 7? We need to ensure it is reasonable to credit this as a critical task for this scenarios particular size of SBLOCA.

Robinson 2021 Draft Exam Comments (Attachment to ES-301-7)

Scenario 4: Low Power SGTR Why is the Service Water leak not a TS call, at least until isolated? Lets discuss during prep week Move bottom of p. 14 to top of p.15?

Add additional critical task to terminate SI flow/stop the primary to secondary leak before S/G overfills water into the steam lines (W EOP based)

Lets add a few more steps in E-3, terminate scenario after step 27 verifies no more SI needed.

ROBINSON 2021-301 Written Examination Review ES-401 Written Examination Review Worksheet Form ES-401-9 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only All General/Generic Comments:

1. For all the Modified questions, NRC CE needs to see the source Q to ensure the NUREG requirements for Modified are met; otherwise, there is no way to evaluate this aspect.

2. Overlap with operating test: Several written exam Qs were evaluated as E but had potential excessive overlap with op. test items, please check the following Qs for a discussion:

Q8, Q19, Q43 1

F 2

X X

Y N

B E

K/A 007EK1.02

1. Q=K/A, Q=RO level.

2. Given Tave of 547 and stable lessens the plausibility of emergency boration being required. Recommend Tave 545 and lowering, or other value and lowering. Applicant could have mis-belief that boration may be required due to excessive cooldown.

3. To be technically precise, we need to change first part distractors to Iodine-135 and Xenon-135.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 2

H 2

X Y

N N

E K/A 009EA1.04

1. Q=K/A, Q=RO level.

2. Second part Q statement is somewhat indeterminate (what should be done for the above conditions). Recommend modification similar to the following:

RCPs ___(2)___ [are/are NOT] required to be secured for the above conditions.

3. Need to add a which ONE of the following statement to this Q.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 3

H 3

X X

Y Y

B U

K/A 011EK3.10

1. K/A match is weak but acceptable, see below recommendation that may be closer K/A match for the keyword reasons for.

2. Second part Q is essentially requiring the applicants to evaluate plant conditions and determine an EOP transition beyond AOP/EOP entry conditions; therefore SRO-only territory.

3. Distractors C(1) and D(1) non plausible because they are YELLOW path numbers, not required to be known from memory by RO applicants.

Recommend substituting 295 degrees for these distractors (i.e. ORANGE path and RED path numbers that are required to be known from memory).

4. Last given bullet recommend stating Total RHR flow to the RCS.

Q as submitted evaluated as U due to license-level mismatch (requiring RO applicant to assess/evaluate plant conditions and determine an EOP transition, which is an SRO-only job function).

Consider the following recommendation for the second part Q statement and distractors:

The reason step 2 of FR-P.1 checks for LBLOCA conditions is ___(2)___. [to route the operators to the appropriate section in FR-P.1 that mitigates PTS for a LBLOCA/to direct the operators to transition out of FR-P.1 because FR-P.1 does not mitigate for a LBLOCA]

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 4

F 3

X Y

N N

E K/A 015AA2.11

1. Q=K/A, Q=RO level.

2. Delete following a Large Break LOCA after the first bullet, if we have a LBLCOA in FR-C.1 starting RCPs will not be effective because the primary and secondary are uncoupled. More plausible that we have a SBLOCA, but we can just leave out that information; were in FR-C.1 for whatever reasons.

3. Need to add extra information in the bullets: (1) No RCPs are running; (2) All S/G NR levels are 30-35%. This information needed to ensure conditions are established for RCP start in the associated loop.

4. Recommend adding the word setpoint in the first part Q statement, if CET temperatures are above a MINIMUM setpoint of ___(1)___.

5. Consider the following tweak to second part Q statement to more closely parallel the FR-C.1 language (?): Minimum seal injection flow ___(2)___ [is/is NOT] required to be established before starting the associated RCP.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 5

H 2

Y N

B S

K/A 022AA2.03

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

6 H

2 Y

N B

S K/A 022AA2.03

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 7

H 2

X X

N N

N U

K/A 026AG2.4.9

1. Q does not match K/A. Conditions in the Q are given at 100% power; the K/A requires knowledge of low power or shutdown conditions in an accident.

The correct answer can be reached without any reference to low power conditions that may or may not be different from those at 100% power. For example, the plant is in MODE 4 or 5 on some kind of RHR cooling, then you have a loss or failure of CCW - how do you mitigate that accident, vs. mitigation of accident initiated at full power.

2. For the submitted Q, note that there are subset issues with the distractor choices - logically, if A is correct then B is also correct; if C is correct then D is also correct.

Q as submitted evaluated as U due to not meeting the supplied K/A statement.

Facility Licensee provided a completely new Q for this K/A statement. The new Q was evaluated as meeting all NUREG-1021 requirements. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 8

F 3

Y N

N E

K/A 027AK2.03

1. Q=K/A, Q=RO level.

2. The submitted Q may be ok. However, we need to evaluate the overlap between this Q and Scenario 4 Event 5 (PC-444J control band shift). If the indications are similar, the applicants will have just seen this failure in the simulator scenario, and we may need to replace either this Q or the event.

Lets discuss.

Q evaluated as E at this time due to potential overlap with op. test event.

After discussion with facility licensee concerning the overlap issue between this Q and the operating test, NRC agrees with facility licensee that there is not an unacceptable amount of overlap between this Q and the op test. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 9

H 3

X X

Y N

N E

K/A 038EK1.04

1. Q=K/A, Q=RO level.

2. Plausibility of distractors B(2) and C(2) is weak. If the intact S/Gs are fully depressurized, this is an extremely desperate action when core cooling is in extremis. Once a S/G is fully depressurized, what could an operator do to continue to promote reflux boiling? In other words, A(2) and D(2) are continuous actions that are under operator control; B(2) and C(2) are a plant condition/end state. Recommend the following easy tweak to resolve the issue for B(2) and C(2): the intact S/Gs are depressurized at maximum rate.

Q as submitted evaluated as E at this time.

Upon further discussion with facility licensee during validations, it became evident that asking for actions to promote reflux boiling (2nd part Q statement) was an unacceptable Q because the facility licensee procedures did not directly support operators taking actions to promote reflux boiling. Facility licensee further modified the Q to elicit knowledge of the appropriate flow configuration for reflux boiling. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 10 H

2 Y

N B

S K/A 040AK2.02

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 11 H

3 X

X Y

N B

U K/A 055EK3.02

1. Q=K/A, Q=RO level.

2. Cueing/plausibility: the phrasing of the B(2)/C(2) distractors lead to their being non-plausible. Specifically, the language of ensuring that no reactor head voiding will occur is too strong and contains obvious cues that would lead an applicant to reject them as being true - it is impossible to ensure no reactor head voiding will occur in a loss-of-all-AC-power situation. Moreover, the logical combinations of distractor choices do not work for distractor B, because when pressurizer level is 0% (or in other words, offscale low) how would an operator know that no head voiding will occur when there is no additional information provided (CET temps, RVLIS, etc)

3. Recommendation to increase the plausibility of these distractors: consider something like prevent excessive reactor head voiding.

Q as submitted evaluated as U due to multiple non plausible distractors.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 12 H

3 X

X Y

N N

E K/A 057AA2.17

1. Q=K/A, Q=RO level.

2. Strong cueing/teaching in the Q with the phrase indicating that IB-3 power is lost. Need to delete this phrase from the first part Q statement. As submitted, one could question the plausibility of channel 3 lights lit when channel 3/instrument bus 3 power is lost.

3. For grammatical parallelism with controller(s), all second part distractors need to include verb are for example is/are or is/are NOT Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 13 H

3 X

Y N

B E

K/A 058AA1.02

1. Q=K/A, Q=RO level.

2. Believe this Q is at higher cognitive level, requires analysis of plant conditions beyond one-step recall of information.

3. We dont know what action an operator will take; first part Q needs to ask what action(s) are first required by procedure.

4. Consider asking second part Q this way: Both Instrument Bus 2 and 7

___(2)___ [will/will NOT] de-energize.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 14 H

3 X

Y N

N E

K/A 062AK3.04

1. Q=K/A, Q=RO level.

2. First part Q statement: add in accordance with OP-and change is throttled to allow to is required to be throttled to allow Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 15 H

2 X

Y N

B E

K/A 065AA1.05

1. Q=K/A, Q=RO level.

2. Think we need to add additional bullet in given information: all FRVs are in AUTO or similar statement.

3. Distractor C - for increased parallelism with distractor D, lets say trip the turbine and go to AOP-007 Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 16 F

2 Y

N N

S K/A 077AK2.06

1. Q=K/A, Q=RO level.

2. Consider adding the word currently in Q statement?

Q as submitted appears S at this time.

17 H

2 X

X Y

N N

U K/A W/E04EG2.4.47

1. Q=K/A, Q=RO level.

2. In Q statement, would it be better to ask the leak was FIRST isolated between

3. Based on the given table, it doesnt contain any parameter that would cause an applicant to choose the incorrect distractor (-30 to -40 minute). In that time, Tavg is constant, RCS pressure is going up but FIRST began to go up in the previous time frame, PZR level is going down as it has been in every time frame, and RWST level has continued to go down as it has been in every time frame. Recommend at the T-40 minute column, set pressurizer level to 22%;

that way, if an applicant has an incorrect belief that ECA-1.2 says that the leak is isolated when level starts to go up they could plausibly choose the incorrect distractor.

Q as submitted evaluated as U at this time due to multiple non plausible distractors.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 18 H

3 X

X X

Y N

N E

K/A W/E11EK1.3

1. Q=K/A, Q=RO level.

2. Need to add a which ONE of the following statement.

3. Need to add additional parameters in the given information: recommend stating Containment Pressure is 16 psig and lowering slowly; and 2 CV Recirc Fans are running.

4. Distractors A and B are logical subsets of each other.

5. One way to fix the subset issue: ask whether BOTH Containment Spray pumps should be stopped or stop ONE Containment Spray pump; ask whether BOTH SI pumps should be stopped or stop ONE SI pump. Dont have to use this recommendation, just need to fix the logical subset issue.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 19 F

2 Y

N N

E K/A 001AK2.08

1. Q=K/A, Q=RO level.

2. Q as submitted psychometrically acceptable at this time. However, there is too much overlap between the second part Q and sim JPM Awhether rods are continuously moving in or continuously moving out, the AOP actions are the same. Either the second part of this Q or sim JPM A will need to be changed.

Q as submitted evaluated as E at this time due to the overlap issue identified with the operating test simulator JPM A.

Facility Licensee Modified Q as requested by NRC. The proposed second part Q statement was changed to another knowledge item related to the K/A, but that did not include any information that was potentially an overlap with AOP information as required in sim JPM A. Minor subset correction in first part Q statement, add the word (minimum) just after the ___(1)__ ? (or equivalent, whatever you think is the best way to express the minimal concept).

New second part Q is acceptable and Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 20 H

3 Y

N B

S K/A 028AA2.11

1. Q=K/A, Q=RO level.

Q as submitted evaluated as S at this time.

21 F

2 X

Y N

N E

K/A 051AK3.01

1. Q=K/A, Q=RO level.

2. Minor tweak: recommend adding the word setpoint to the Q statement, consider: UNLESS condenser vacuum degrades below a MINIMUM setpoint of ___(2)___.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 22 H

3 X

Y N

B E

K/A 060AK3.02

1. Q=K/A, Q=RO level.

2. Three minor tweaks recommended to Q statement, consider the following:

IAW AOP-009, the D WGDT is required to be placed on cover and pumped to a lower pressure WGDT until the leaking tank has reached a pressure LESS THAN a maximum setpoint of ___(2)___.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 23 H

2 Y

N B

S K/A 068AG2.4.34

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

24 H

2 X

X Y

N N

E K/A 074EA1.12

1. Q=K/A, Q=RO level.

2. Last bulleted parameter: 3 S/Gs vs. 4 S/Gs?

3. WOOTF statement needs to be rewritten as is required to implement not will implement

4. Distractor B is a YELLOW path procedure and therefore never required to be entered; also it would only be entered on SRO discretion (potential license-level mismatch). Therefore, distractor B is non-plausible.

Q as submitted evaluated as E at this time due to one non-plausible distractor.

Facility Licensee Modified Q as requested by NRC, including modification of Q statement to eliminate the non plausibility of proposed distractor B. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 25 F

3 X

Y N

B E

K/A W/E08EK1.2

1. Q=K/A, Q=RO level.

2. Plausibility of second-part distractors is weak, recommend strengthening by changing second part Q statement as follows: Based on the given conditions, once RCS temperature is stable, the crew is required to perform a soak for a MINIMUM of ___(2)___.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 26 F

3 X

Y N

B E

K/A W/E10EK2.2

1. Q=K/A, Q=RO level.

2. Minor editorial enhancements - (1) recommend adding in RCS cold legs in first part Q statement after allowed. (2) recommend adding setpoint after MINIMUM pressurizer level Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 27 F

2 X

X Y

N M

E K/A W/E15EA2.1

1. Q=K/A, Q=RO level.

2. In order to fully evaluate whether or not FRP-J.2 entry conditions are met, we need to add additional parameter of CV pressure. Recommend additional bullet stating CV Pressure is 7 psig and slowly lowering or equivalent.

3. Distractors A(2) and B(2) are technically non plausible when compared with the correct choices. Giving some benefit-of-the-doubt here due to the difficulty in constructing plausible distractors for such an infrequently performed procedure. Discussion is as follows: title of the ORANGE-path procedure is CONTAINMENT FLOODING. The C(2) and D(2) choices are directly related to concerns (direct cause and effect relationship) that would arise with too much water in the containment building. A(2) and B(2) essentially say that the containment building gas volume could contract like a pressurizer and lead to high containment pressure. But if that was the concern, we know we have RED path procedure to mitigate high containment pressure; it is not a direct concern of a FLOODING procedure. Consider the below recommendation for A(2)/B(2):

containment building structural integrity and seismic qualification could be challenged if beyond-design-basis flooding occurred.

Q as submitted evaluated as E at this time due to non plausible distractors.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 28 H

3 X

Y N

M E

K/A 003K6.14

1. Q=K/A, Q=RO level.

2. Minor editorial tweaks: recommend adding setpoint after MINIMUM in first part Q statement.

3. Recommend adding IAW OP-101, [procedure title], RCS pressure

___(2)___. To second part Q statement.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 29 H

3 X

Y N

B E

K/A 004K5.50

1. Q=K/A, Q=RO level.

2. Believe we need to add a few extra system parameters to tighten-up the plant status. Add No adjustments have been made to turbine load, maybe after the TCV-144 bullet? Also add Control rods are in MAN due to Reactor Engineering recommendation or equivalent.

3. Recommend adding with no operator actions, which ONE Of the following

4. Second part Q statement - add automatically after TCV-143, VCT/DEMIN DIVERSION, will Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 30 H

2 X

X Y

N B

E K/A 004K6.31

1. Q=K/A, Q=RO level.

2. Technically there is an internal subset issue here in that the numbers for the distractor (7-18) are bounded on both sides by the numbers in the correct answer (6-20). Recommend to following editorial change to the second part Q statement to deal with this issue: APP-001-B4 states that the expanded seal water injection flow control band MINIMUM and MAXIMUM limits are

___(2)___.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 31 F

3 Y

N N

S K/A 005G2.2.39

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 32 H

3 X

Y N

B E

K/A 006K5.07

1. Q=K/A, Q=RO level.

2. First part Q statement: when is the MINIMUM subcooling value reached?

E.g., at end of cooldown or at end of depressurization? Please add correct parameter to end of first part Q statement for additional technical accuracy.

3. Please provide simulator data to support the second part Q statement. Its probably better to ask a steady-state to steady-state trend here rather than a blanket during RCS cooldown to prevent any fluctuations from changing the answer. Consider asking During RCS cooldown, (as submitted) vs. When operators stop the RCS C/D at the correct target temperature, (recommendation)

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 33 F

3 Y

N M

S K/A 007G2.1.20

1. Q=K/A, Q=RO level.

Q as submitted evaluated as S at this time.

34 H

3 X

X Y

N N

E K/A 008K2.02

1. Q=K/A, Q=RO level.

2. Technically there is a subset issue between distractor A and distractors B and D. Giving benefit-of-doubt here due to ease of correction: recommend editorial tweaks to WOOTF statement as follows: with no operator actions, which ONE of the following identifies all currently operating Component Cooling Water Pump(s) and the/their current source of electrical power ONE MINUTE after the SUT loss?

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 35 H

2 Y

N N

S K/A 008K4.09

1. Q=K/A, Q=RO level.

Q appears S at this time.

36 H

2 Y

N B

E K/A 010A2.01

1. Q=K/A, Q=RO level.

2. Better to ask insufficient Pzr Heater Capacity instead of Operability, technically an operability evaluation is SRO only level.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 37 H

3 X

Y N

M E

K/A 012K3.01

1. Q=K/A, Q=RO level.

2. Recommend the following slight tweaks to the second part Q statement to increase clarity: OTT rod stop will disable Control Rod withdrawal at a MINIMUM setpoint of ___(2)___ BELOW the OTT trip setpoint.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 38 H

2 X

X X

Y N

B E

K/A 012K4.04

1. Q=K/A, Q=RO level.

2. Distractors are confusing/readability issues. We need to include the full breaker status for distractors B and D in the same order and format as the other answer choices, and deal with subset issues with the reasons for the breakers opening.

3. Recommend in the initial given information, we provide the initial status of all Rx Trip and Rx Trip Bypass breakers in the following order, then keep this same order for all distractors:

-Reactor Trip Breaker A (RTA) is currently closed

-Reactor Trip Bypass Breaker A (BYA) has been racked in and closed

-Reactor Trip Breaker B (RTB) is currently closed

-Reactor Trip Bypass Breaker B (BYB) is racked out

4. Then, for all distractors, list various combinations of breaker status in the same sequence using the above abbreviations for readability:

A. RTA will remain CLOSED.

BYA will OPEN due to the shunt coil energizing ONLY.

RTB will OPEN due to the shunt coil energizing and the undervoltage coil de-energizing B. RTA will remain CLOSED.

BYA will OPEN due to the shunt coil energizing ONLY.

RTB will OPEN due to the shunt coil energizing ONLY.

Etc Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 39 F

2 X

Y N

N E

K/A 013K1.19

1. Q=K/A, Q=RO level.

2. Recommend editorial enhancements to the Q statement as follows: Which ONE of the following correctly identifies ALL the below listed valves that receive an automatic isolation signal on a Safety Injection (SI) actuation?

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 40 F

3 X

X Y

N B

E K/A 022A4.01

1. Q=K/A, Q=RO level.

2. Lots of actionable subsets here we need to resolve: distractors A and C are logical subsets; distractors B and D are logical subsets. Consider the following re-organization and re-write of the distractors:

A. With no operator action required, the HVH units will automatically start on LOOP sequencer [Robinson insert plant-specific terminology here?]

B. Operators are required to push the vibration reset pushbutton(s), this will allow the HVH units to automatically start.

C. No action is needed with regards to control power fuses; however, the operators are required to manually start the HVH units with the control switch(es).

D. Operators are required to remove and then reinstall control power fuses (otherwise the control switch will not function), then manually start the HVH units with the control switch(es).

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 41 F

2 Y

N B

S K/A 026A1.01

1. Q=K/A, Q=RO level.

2. Consider adding setpoint after MAXIMUM in second part Q statement?

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 42 H

3 X

Y N

B E

K/A 039K3.06

1. Q=K/A, Q=RO level.

2. Recommend we add one additional open bullet as follows: Pressurizer pressure is lowering rapidly.

3. Better to add a with no operator actions clause (before WOOTF?)

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 43 H

2 X

Y N

B E

K/A 059K4.08

1. Q=K/A, Q=RO level.

2. Minor stuff: all caps ONE in WOOTF statement, add with no operator action before second part Q statement.

3. Bigger issue is overlap with scenario 1, event 4 - both deal with a PT-475 failure at relatively high power/on main FRVs. One of these exam items will need to be modified. A few thoughts: can we tweak this Q for a low power situation (on FRV bypass values in MAN or AUTO, for example) then ask the same thing (PT-475 fails HIGH) and plant response or required operator action(s)? I would consider these plant conditions different enough that we would not have excessive overlap.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC, including a different instrument failure occurring at a much lower power level. Overlap concern resolved. Modified Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 44 F

3 X

Y N

B E

K/A 061K5.01

1. Q=K/A, Q=RO level.

2. Recommend the following tweaks to the Q statement: Based on the given conditions, IAW FRP-S.1, AFW flow must be greater than a MINIMUM setpoint value of ______ to establish a Secondary Heat Sink.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 45 H

2 Y

N B

S K/A 062A3.05

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

46 F

2 Y

N B

S K/A 062K2.01

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

47 F

3 Y

N N

S K/A 063K2.01

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

48 F

3 X

Y N

N E

K/A 064A2.02

1. Q=K/A, Q=RO level.

2. Slight cueing in distractors C(2)/D(2) - does the nomenclature for this panel include all caps AND? better lower case, we usually reserve all caps for a logical connection, which I dont think we have here. If its the way its labeled in the plant, keep it all caps.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC, changing C(2)/D(2) distractors to all caps. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 49 F

2 X

Y N

N E

K/A 064K1.03

1. Q=K/A, Q=RO level.

2. Slight tweak - define DFOST acronym in the first part Q then use the acronym in second part Q.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 50 H

3 X

X X

X Y

?

B E

K/A 073A1.01

1. Q=K/A, Q=RO level.

2. Is it required RO level knowledge at Robinson to know the PSAL limits from memory? Most places that CE is familiar with, this would be either an SRO job function (or really Chemistrys terminology), or at the minimum something you determine with a reference. NRC CE will accept providing the PSAL definitions in the Q statement stem as given information as per the below recommendation as long as the total RCS leakage given puts you into a different PSAL. Not a direct lookup because you have to correctly convert units (gpm to gpd) and recognize which leakage value to use in the PSAL (total unidentified RCS leakage vs. S/G, primary-to-secondary leakage.). Would this change-of-direction be more RO level knowledge flavor? Maybe or maybe not. If this is required RO level knowledge at Robinson, could you please provide a learning objective to that effect? We can discuss, see the below Q sketch for what Im thinking might work.

3. In any event, we need to fix subset issue with second part Q statement -

maybe something like when less than a MAXIMUM of ___(2)___ power.

Q as submitted evaluated as E at this time.

Recommendation to address my comment in number 2 above:

Given the following:

-The crew has entered AOP-035, S/G TUBE LEAK, based on multiple indications of A S/G tube leakage.

-The crew is at step 11 of AOP-035, CHECK Leak Rate Determination -

COMPLETE.

-OST-051, Reactor Coolant System Leakage Evaluation, was performed and determined total unidentified RCS leak rate to be 0.06 gpm.

-Chemistry has performed isotopic analysis of S/G samples and confirmed the primary-to-secondary leak rate to be 0.04 gpm.

Which ONE of the following correctly completes the statements below?

IAW AOP-035, a ___(1)___ [PSAL-2/PSAL-1] event has occurred.

PSAL-1 defined as leak rate GREATER THAN OR EQUAL TO 30 gpd.

PSAL-2 defined as leak rate GREATER THAN OR EQUAL TO 75 gpd AND LESS THAN 100 gpd.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 51 H

2 Y

N B

S K/A 073A4.02

1. Q=K/A, Q=RO level.

2. Need to add a WOOTF statement to this Q?

Q as submitted appears S at this time.

52 H

3 Y

N B

S K/A 076K1.09

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

53 F

3 X

Y N

B E

K/A 078K3.03

1. Q=K/A, Q=RO level.

2. What procedure specifies the reasons why IA cant be used for Breathing Air (second part Q statement)? Could not determine from the provided distractor analysis information. We need to add that procedure to the beginning of the second part Q statement, e.g., IAW OP-XYZ, the reason(s) IA will no longer be used. Is Breathing Air only used inside Containment? If so, do we need to set this Q at a lower power MODE with workers inside? (given condition is 100% power)

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 54 H

3 X

X Y

N N

E K/A 061 A3.01

1. Q=K/A, Q=RO level.

2. So CE is thinking we need to throw in a with no operator actions somewhere and make the first part Q distractors did/did NOT automatically start; however, then were unbalancing the A(2) and C(2) distractors because then some operator action (starting the SDAFWP) is theoretically occurring (albeit incorrectly because its already started). But to make the second part of this Q work you need it, and Im also worried that we need some further statement about no operator adjustment to AFW FRVs. All in all that would make the entire set of distractors cumbersome. What do you smart guys think?

Would an applicant say that given a manual start of the SDAFWP that it is not operationally valid to not throttle AFW flows to whatever they need them to be?

Not sure the best way to handle the above random thoughts.

Q as submitted evaluated as E at this time.

Further NRC Comments on modified Q:

-for the third time bullet, would it be better/more clear to write it this way: ALL S/G narrow range (NR) levels have trended from their initial levels to 18% NR, and have stabilized. The first time the CE read this bullet it was confusing, potentially SGWL NRs was somehow wide range not narrow range.

-first part Q statement, recommend including time of evaluation (12:12:00)

-second part Q statement, recommend stating: the SDAFW pump delivers a total of ___(2)___

Facility Licensee Modified Q as requested by NRC, making changes to the provided distractors to more closely tie the information to discrete times as given in the timeline. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 55 H

4 Y

N B

S K/A 103A3.01

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

56 H

3 Y

N N

S K/A 001K4.20

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 57 H

2 X

X Y

N B

E K/A 002K5.12

1. Q=K/A, Q=RO level.

2. Ugh. As submitted, the third bullet information does not provide any definite information. Does the RO see an offscale high Tcold instrument or not? I recognize its a bank question, but we have to provide clear indications for a valid ILO level question. Much better to state third bullet information the same way as second bullet information: Loop 3 Tcold instrument indicates offscale high.

3. Need WOOTF statement for consistency with rest of exam?

4. First part Q statement: we need to get the indeterminate nature out of the question - think of it this way, how am I, as an applicant, going to know what an arbitrary RO expects to see? I could answer, anything, be correct, and challenge this and win because of the way the Q is worded. Better to state:

Based on the given conditions, the loop 3 delta-temperature gauge will be

___(1)___.

5. Giving major big benefit-of-doubt on plausibility of the A(2)/B(2) distractor set. It is a logical true given by the other choice that there are three Thots per piping loop, my choice is: Tavg calculated using all three installed Thots, or some arbitrary one Thot? (if an applicant momentarily forgot how many Thots there were the C(2)/D(2) distractor choice will jog the memory). Doesnt make sense for anyone to pick an arbitrary one Thot, otherwise why would the plant design need to include 3 Thots and only one Tcold? Recommendation to fix this problem:

A(2)/B(2): ONLY the highest one of three C(2)/D(2): an average of all three Q as submitted evaluated as E due to non plausible distractors and major big benefit-of-the-doubt.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 58 F

2 Y

N B

S K/A 011K2.02

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 59 F

2 X

Y N

B E

K/A 014A4.04

1. Q=K/A, Q=RO level.

2. Teaching/cueing in the Q statement: we dont need to know all the other functions the Rod Control startup button does (should be known by the applicants anyway), we just need to ask the relevant part of the Q statement.

Consider the following recommendation: When the OATC depresses the Rod Control Startup pushbutton, among other functions it RESETS ______.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 60 H

2 Y

N B

S K/A 015A2.04

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

61 F

2 Y

N N

S K/A 017K1.01

1. Q=K/A, Q=RO level.

2. CE believes this Q is more Fundamental LOK - simple recall information, no comprehension/analysis needed (or extremely limited).

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 62 F

3 X

X X

Y N

B E

K/A 029A1.03

1. Q=K/A, Q=RO level.

2. We have another internal subset issue here; distractors A(1)/C(1) are bounded above and below by the numbers is distractors B(1)/D(1). Therefore, logically an applicant could choose distractors A(1)/C(1) and still be within the limits as asked by the question (if an operator ENSURES pressures stay within

-0.4 to 0.9 psig then we have also ensured pressures are within the larger (and correct) bounds of -0.8 to 1.0 psig).

3. Second part Q statement is awkward because it provides plant status/conditions in the Q statement. Better to put all plant status information in the bulleted list of given the following. Consider the following recommendation:

[Given first three bullets the same, then]

Current Conditions:

-A Containment Pressure Relief and CV Purge are both in progress

-CV Pressure has lowered to 0.2 PSIG Which ONE of the following correctly completes the statements below?

OST-020 states that operators are required to Ensure CV Pressure is

___(1)___.

Based on the current conditions, IAW OP-921, the Containment Pressure Relief

___(2)___.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 63 F

3 Y

N B

S K/A 033K3.02

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

64 F

3 Y

N N

S K/A 035G2.4.1

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 65 F

2 X

Y N

B E

K/A 068K6.10

1. Q=K/A, Q=RO level.

2. Plausibility of distractor B is slightly weakened in comparison to the additional information given in distractor D in a similar situation. Recommend adding something extra to distractor B to improve the psychometric balance/parallelism with distractor D. Consider the following recommendation:

B. NOT automatically close. The release may continue IAW APP-036-E7 Annunciator Procedure, if operators establish the required compensatory measures.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021 66 F

2 X

X Y

N M

U K/A G 2.1.37

1. Q=K/A, Q=RO level.

2. First part distractors non plausible/LOD=1 (second part LOD is acceptable).

Applicants will have performed many AOP power reductions in the simulator, during training, the audit, and the NRC op. test. During none of these power reductions will any procedure specify that a dedicated Reactivity Manager be stationed. Furthermore, consider an abnormal situation that happens to occur during a Saturday midnight shift. Are the operators required to wait on performing a down power to call in an extra SRO to serve as Reactivity Manager? It just doesnt pass the common sense test, specific knowledge at the RO level is not needed to elicit the correct answer.

3. Second part Q statement - need to change indefinite future tense of shall inform to are required to inform by procedure Q as submitted evaluated as U due to multiple non plausible distractors.

Facility Licensee provided completely new Q to meet the same K/A statement as before. Newly proposed Q tests different applicant knowledge of AD-OP-ALL-0203 REACTIVITY MANAGEMENT procedure. Minor tweak, recommend underline and/or bold font the word required in the second part Q statement.

Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 67 F

2 X

Y N

N S

K/A G 2.1.42

1. Q=K/A, Q=RO level.

2. Consider the following recommendation for the second part Q statement (increased parallelism with LCO statement format): LCO 3.7.13, FUEL STORAGE POOL BORON CONCENTRATION, states that the fuel storage pool boron concentration shall be ___(2)___.

Note: the NRC CE is o.k. with the submitted second part Q statement if it is a preferred format to the above.

Q as submitted appears S at this time.

68 F

2 Y

N B

S K/A G 2.2.13

1. Q=K/A, Q=RO level.

Q as submitted appears S at this time.

69 F

2 X

X Y

N B

E K/A G 2.2.35

1. Q=K/A, Q=RO level.

2. Technical accuracy/partial: the experienced applicant with TS may know that we could have shifted MODES from 4 to 3 under a LCO 3.0.4.c exception (e.g.,

authorized engineering risk analysis) which would make 1 a correct answer.

3. Think we need to add further information to the given stuff to ensure we fully explain plant status, consider the following:

Given the following:

-SRNIs are 14 cps and stable.

-All Reactor Trip and Reactor Trip Bypass breakers are OPEN.

-RCS Tavg is 380 °F and stable.

-RCS Tcolds are as follows: 380 °F / 381 °F / 378 °F and stable.

-No testing is in progress WOOTF.

The plant is in MODE ___(1)___.

In order for the applicable TS LCO(s) to be met without exceptions, at least

___(2)___ Safety Injection Pump(s) is/are required to be OPERABLE.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 70 F

2 X

Y N

B E

K/A G 2.2.43

1. Q=K/A, Q=RO level.

2. First part Q distractors non plausible, but giving some benefit of the doubt because of hopeful ease of correction. Argument is as follows - an alarm that is broken is also DISABLED. How is it plausible that it could be guaranteed to be restored prior to the end of the shift? What if it broke 30 minutes before shift turnover? Also, if I were an applicant taking the exam on game day, I would ask whether or not the second part question was a temporarily disabled annunciator, or a fully disabled annunciator. We would need to answer such a question to clarify the expectation, which would basically give away the correct answer. Recommend just asking about a temporarily disabled annunciator, or moving in a different direction for the second part Q statement.

Q as submitted evaluated as E at this time, giving benefit-of-doubt.

Facility Licensee Modified Q as requested by NRC, and is now asking knowledge of a temporarily disabled annunciator in the second part Q statement. Q now appears ready for administration on the NRC written exam.

MKM 08/31/2021 71 F

3 X

X Y

N B

E K/A G 2.3.13

1. Q=K/A, Q=RO level.

2. Believe that technically the first part Q statement reference is AD-RP-ALL-004, RADIOLOGICAL POSTING AND LABELING.

3. Second part Q statement implies that there is only one requirement for entry, when in reality there are several. See recommendation below:

[same first sentence]

WOOTF.

IAW AD-RP-ALL-0004,, the correct radiation posting for this area is a

___(1)___.

IAW AD-RP-ALL-2017,, among other requirements, the applicable checklist for entry into this area ___(2)___ [does/does NOT] verify continuous RP coverage is assigned as required by the RWP.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 72 F

2 X

?

Y Y?

B E

K/A G 2.3.4

1. Q=K/A.

2. Is this Q on the RO level at Robinson? Normally, Emergency dose limits and authorization for Emergency dose limits is an SRO-only topic for written exam Qs and administrative JPMs. Were asking an RO applicant to have direct recall of all Emergency dose limits without a reference. Is there an associated training objective at Robinson for this? Im worried that we may be on SRO territory here and want to ensure we are logically consistent among sites and exams. Lets discuss.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 Following Branch Chief review, additionally added TEDE after first part Q statement blank for further clarification. MKM 09/10/2021 73 H

2 N

N B

U K/A G 2.4.11

1. Q does not meet K/A requirements for Tier 3 generics. This Q as submitted deals with specific knowledge of one particular AOP and does not cover general plantwide, admin procedure type knowledge of AOPs. To meet this K/A, we need to focus on information/knowledge that is likely found in documents like an AOP users guidefor example, do AOP steps have to be completed before moving on in the procedure? Do AOP CAUTIONS and NOTES have to be read to all operators? Are transition briefs required to be held on transitioning from one AOP to another? How do you handle AOP if at any time or continuous action type steps? This is an example of the type of generic information that I would expect to see on the Tier 3 level for this K/A.

Q as submitted evaluated as U due to not meeting the specified K/A statement on the Tier 3 level.

Facility Licensee submitted entirely new Q after NRC comments. Newly proposed Q meets assigned K/A statement and elicits applicant information concerning concurrent use AOPs that is appropriate for a Tier 3 Generic K/A.

Facility licensee certified that knowledge of concurrent use AOPs was RO-level knowledge, not SRO-only level. Minor typographical comment, recommend:

correctly identifies all CONCURRENT AOPs from the following.

Q now appears ready for administration on the NRC written exam. MKM 09/10/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 74 F

3 X

Y N

M E

K/A G 2.4.12

1. Q=K/A, Q=RO level.

2. Recommend expanding the scope of the second part Q statement to make it more of a generic type Q rather than specific to a given situation. Consider the following recommendation: IAW OP-RNP-1001,, EOP foldout criteria(s)

___(2)___ [do/do NOT] continue to apply after a transition is made from an EOP E-series procedure to an FRP-series procedure. Similar to the comment on Q73, the intent is to make the Q more generic rather than specific to a particular given situation, because this is a Tier 3 K/A.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 75 F

2 X

Y N

B U

K/A G 2.4.6

1. Q=K/A but weak on the associated Tier 3/generic requirements; maybe better to ask for knowledge of more than one specific EOP procedure rather than eliciting information about only one EOP procedure (see recommendation below). Q=RO level.

2. Distractors A and B are not plausible. Distractor A involves essentially overfilling the ruptured S/G which is a negative/bad condition for this ruptured S/G when considering releases, etc. Consider that E-3 provides for a special maximum rate cooldown because youre in a race to terminate SI and stop the primary-to-secondary leakage filling the ruptured S/G, why would a subsequent procedure then have you over-feed the secondary side of a ruptured S/G especially given a WR level criteria? Also note that distractors C and D specifically provide for mitigation strategies to lower S/G levels by draining the ruptured S/G, not filling. Distractor B is not plausible because it specifies only use of the steam line PORV, which would constitute an unmonitored radiological release directly to atmosphere. No additional reason is given as to why operators would decide to initiate this release (no plant condition that would force this decision); furthermore, this is the only choice that would theoretically be a direct release path. Similar to choice A, hopefully the operator applicants would recognize that distractor B is a bad condition choice, not what the procedure would direct, at least in the absence of in extremis conditions that were not given in the Q stem information.

Q as submitted evaluated as U due to multiple non plausible distractors.

Consider the following recommendation using similar knowledge material:

Which ONE of the following correctly completes the statements below?

IAW the applicable EOP BASIS DOCUMENT, procedure ___(1)___ [EOP-ES-3.1/EOP-ES-3.3] is designed to minimize both the spread of contamination on the secondary side and release of radioactive effluent to the environment during cooldown to cold shutdown.

IAW the applicable EOP BASIS DOCUMENT, procedure ___(2)__ [EOP-ES-3.2/EOP-ES-3.3] provides the most rapid means of depressurizing the RCS and ruptured steam generator to cold shutdown conditions.

EOP-ES-3.1, POST-SGTR COOLDOWN USING BACKFILL.

EOP-ES-3.2, POST-SGTR COOLDOWN USING BLOWDOWN.

EOP-ES-3.3, POST-SGTR COOLDOWN USING STEAM DUMP.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/31/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only

- - SRO-ONLY EXAM QUESTIONS BELOW***

76 H

3 X

X Y

Y B

E K/A 007EG2.2.37

1. Q=K/A, Q=SRO-only.

2. Units: Please provide the appropriate page(s) of the Station Curves that relate the CST level in % (given in Q stem) with the CST level in gallons (listed in Tech Specs), also include REFERENCE PROVIDED statement.

3. Add additional statement that the SDAFWP was not/is not running for this event.

4. Need additional information regarding the alternate suction source: never in service? Ref. OP-402 P&L.

5. Change Q stem term Subsequently: to something like Current Conditions to improve parallelism with the capital-letter Q statement terminology.

6. Need to add references to Q statement for clarity, maybe something like this:

In accordance with TS 3.7.5, Condensate Storage Tank (CST), the CURRENT status of the CST is ___(1)___; and In accordance with OP-402, Auxiliary Feedwater System, the CURRENT status of the SDAFW pump is ___(2)___.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 77 H

2 X

X X

Y Y

N E

K/A 008EG2.2.44

1. Q=K/A, Q=SRO-only.

2. The given plant conditions are very confusing to me. In the initial set of conditions, we are presented with a case of a failed-open PORV and PORV block valve. But if current RCS pressure is still 1900 psig and lowering, why would the operator have manually actuated SI? Does not seem to be supported by E-0 step 4 RNO column. (were not given a high Containment pressure or high steam flow values).

3. Second bullet - recommend being more specific that just the valve, recommend PCV-455C for clarity because we then immediately talk about another valve.

4. Also Recommend re-sequencing the given information bullets to read more like the flow of actions in the control room. For example, after placing the RC-536 switch in STOP, what does the OATC see, then say that the Rx automatically tripped, etc. As written, it is confusing as to when these conditions occurred.

5. Cue/Plausibility - in order to make a transition to ES-1.1, much more information regarding SI Termination parameters needs to be provided in the stem to make this a plausible distractor; we need to add enough plant information status iaw E-0 step 17 that an SRO applicant can make a full assessment of all SI Termination criteria. To mention just one parameter as an example, because this is a vapor-space accident, maybe give pressurizer level high and rising.

6. Need to change the indefinite future construct of what the CRS will transition to, better to ask what the CRS is required to transition to.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review 78 H

2 X

X X

Y Y

N E

K/A 029EA2.01

1. Q=K/A, Q=SRO only.

2. What has cooled the plant down so much? Temps of 490 F would seem to be way low for an ATWS that has seemingly continued for a while - if we are still at 4% power and lowering slowly that is not an indication of a tripped Rx.

3. At a minimum distractor C is non-plausible because there is not a direct transition from FRP-S.1 to ES-1.1 anywhere in the FRP-S.1 procedure.

Furthermore, similar comment to above question - the applicant is not provided with all SI Termination parameters, therefore it is a cue that this distractor is not correct/non-plausible.

4. Recommend the following modifications to the answer choices(?):

(1) The CRS is required to transition to E-0 ___(1)___. [step 1 to check for Reactor Trip/step 17 to check for SI Termination criteria]

(2) The CRS ___(2)___ [is/is NOT] required to complete performance of FRP-S.1 Attachment 4 before a transition to E-0.

Q as submitted evaluated as E at this time due to one non-plausible distractor.

Facility Licensee Modified Q as requested by NRC. However, the recommended modifications introduced additional complications upon validation. CE is now not sure that the above recommendation is technically correct. Specifically, the absolutely correct answer to the procedure transition Q is return to procedure and step in effect. However, we do not provide that information in the Q stem (which already has a lot of information in it). Heres another stab at a recommended fix: all the Given and Current conditions the same, then:

WOOTF statement The CRS ___(1)___ [is/is NOT] required to complete performance of FRP-S.1 before making a transition out of FRP-S.1 Independent of the status of Attachment 4, based only on the current conditions, FRP-S.1 ___(2)___ [does/does NOT] direct a transition out of FRP-S.1.

Maybe this new recommendation takes care of the problem that FRP-S.1 does not require a transition to either E-0 step 1 or E-0 step 17; it requires a transition to procedure and step in effect. Lets discuss during exam review.

Facility licensee proposed new first and second part Q statements and distractors with timeline relatively intact that better capture a specific SRO-only level decision related to procedure selection. With these additional changes, Q now appears ready for administration on the NRC written exam. MKM 09/10/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 79 H

3 X

X Y

Y N

E K/A 054AA2.04

1. Q=K/A, Q=SRO only.

2. What is the impact of the loss of MCC-10? Its not stated in the distractor analysis. I am inferring that it causes a loss of control power to some valve(s) in an AFW flow alignment/affecting LCO 3.7.4?

3. As written, technically all second part answers are correct, because the station is in a REQUIRED ACTION of CONDITION A of LCO 3.7.4 to restore the SDAFW pump in 7 days. Giving you some benefit-of-the-doubt here.

4. Recommend we change second bullet to say SDAFW Pump was tagged out yesterday at 0700 for oil replacement and is expected to return to service in three shifts. Then for the second part Q statement ask something like the following:

If the status of the AFW system components does not change from the current conditions at time 1301:15, then IAW ITS 3.7.4, AUXILIARY FEEDWATER (AFW) SYSTEM, the LATEST time the Plant is required to be in MODE 3 is

___(2)___ [today at 2001:00/seven days from yesterday at 0700 plus six hours] (-determine what these times would be)

Q as submitted evaluated as E at this time due to multiple correct answers to the second-part Q statement as currently worded.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 80 F

2 X

Y Y

B E

K/A 056AG2.2.25

1. Q=K/A, Q=SRO-only.

2. Need to add IAW TS 3.8.1 BASES statement to first part Q statement.

3. Need to add automatically to the second part Q statement, otherwise an applicant could mis-construe that we are asking if the EDG is capable of manual start and load.

4. NOTE: Reviewer does not believe that this Q has been significantly modified enough to classify as a MODIFIED question. Recommend classification as BANK.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 81 H

2 X

X Y

Y M

U K/A W/E05EA2.2

1. Q=K/A, Q=SRO-only.

2. Not sure what to make of this Q. First part asks me how to realign a backup source to AFW because we dont have a CST anymore. However, choosing OP-903 as a distractor is non-plausible due to the title of the given Attachment 1 to FRP-H.1. If operators are in FRP-H.1 and FRP-H.1 has an Attachment to align alternate water supply to AFW pumps, how would that attachment not be the appropriate procedure to use? If Attachment 1 were to direct the operators to use OP-903, then it would still be a correct choice.

3. Then, Q says that action has been taken to address the CST concerns, which implies that alignment of service water to the AFW pumps was successful. Therefore, it would seem that the actual correct answer to the second part question would be that the next major action would be to reset/restore AFW, not Main Feed. Also, if the given wide range SG levels were setpoints for feed-and-bleed, we would immediately initiate feed and bleed cooling and not worry about re-aligning service water to the AFW pumps? This makes the distractor initiate RCS Bleed and Feed potentially non-plausible.

Consider the following: the given conditions state that the trip was initiated by a loss of both Main Feed Pumps, then I am given other indications that all AFW pumps also tripped. Not knowing additional information concerning these pumps status, it would not be incorrect to go straight to alignment of Condensate pumps.

Q as submitted evaluated as U as this time due to multiple non-plausible distractors/potential no correct answer.

Further NRC Comments on modified/proposed Q:

-First part distractors of following completion of Attachment 1 are not plausible; this is an indeterminate time when compared to a step for checking for total AFW flow. In other words, we dont know when the hypothetical operator completes Attachment 1 vs. when the hypothetical CRS reads the step to check for total AFW flow. Recommend a re-write of the first part Q as below:

The earliest transition to FRP-H.1 in EOP-E-0 is ___(1)___ [before/after]

checking the major accident diagnostic steps (faulted S/G, ruptured S/G, LOCA inside containment) and SI Termination criteria

-also please add iaw FRP-H.1 to second part Q statement.

Facility Licensee Modified Q as requested by NRC, including modifications to the first part Q statement and changing the second part Q statement to involve information concerning the preferred sequence of alternate sources of water for the AFW pumps. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 82 F

2 X

X Y

Y N

E K/A 003AG2.1.23

1. Q=K/A, Q=SRO-only.

2. Distractor D is partially correct if an applicant believes that dilution is appropriate to restore Tave. Q statement of after 30 minutes is a big cue that were looking at TS actions (tied to timing of conditions) because AOP steps will not be stated as after 30 minutes.

3. Further problem is that there is a subset issue with distractors A and B because in order to do A operators would need to trip the turbine.

4. Maybe another possible distractor to use? borate to ensure SDM using OP-301-1.

Q evaluated as E at this time due to at least one non-plausible distractor.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 83 H

2 X

Y N

M U

K/A 032AA2.04

1. Q=K/A

2. Q does not meet SRO-only criteria. The bases information in the second part Q statement is also answered with less than one-hour TS action statements in LCO 3.3.1; therefore, the second part Q can be answered with RO-level knowledge only. Is there information in the TS bases that is not specifically tied to immediate-timed TS actions?

3. First part Q statement does not need to say CRS determines, just has or has NOT the overlap been met satisfactorily? Also add iaw GPP-003 and procedural title to the first part Q statement.

Q evaluated as U at this time due to not meeting SRO-only criteria.

Facility Licensee Modified Q as requested by NRC; now the second part Q statement requires SRO applicants to determine if the LCO 3.3.1 statement may be exited (exit criteria vs. previous question on actions which was not SRO-only). Also added REFERENCE PROVIDED to this Q due to the picture. Q now appears ready for administration on the NRC written exam.

MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 84 F

3 X

Y Y

B E

K/A 036AA2.02

1. Q=K/A, Q=SRO-only.

2. Recommend greater parallelism with the language of the ITS Bases for the second part Q statement. Consider the following: With regards to the postulated Fuel Handling Accident analysis, the BASES for LCO 3.9.7, Containment Purge Filter System, states that recently irradiated fuel is fuel that has occupied part of a critical reactor core within the previous __(2)___. [116 hours/125 hours]

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 85 F

3 X

Y Y

B E

K/A 037AG2.4.18

1. Q=K/A, Q=SRO-only.

2. Recommend adding the following to the second part Q statement: In accordance with the background document to ECA-3.1, if S/G overfill becomes a concern.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 86 H

2 X

X X

Y N

B U

K/A 003A2.03

1. Q=K/A

2. Q does not meet SRO only requirements. The second part question is not related to procedure selection beyond entry conditions (which is normally SRO-only). Instead, the first part Q statement says that the RCP is tripped because it exceeded a bearing temperature limit - not a seal problem. Therefore, the difference between closing the spray valve and closing a seal return isolation valve can be determined with systems-level, RO-level knowledge.

Potential fix? Make the submitted first part Q a second part Q, use the following for a first part Q:

The CRS is required to enter AOP-018 section ___(1)___. [Section B for High Reactor Coolant Pump Vibration/Section C for Loss of RCP Seal Cooling]

As submitted, the Q is evaluated as U at this time due to not meeting SRO-only criteria; the question can be answered solely using systems knowledge and the conditions as given in the Q stem.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 87 H

2 X

X Y

Y N

E K/A 006A2.12

1. Q=K/A, Q=SRO-only.

2. Need to rewrite the Q statement indeterminate future tense will direct to CRS is required to

3. Recommend tweaking the A distractor for improved parallelism with the other distractor choices; maybe something like continue performing steps in EOP-ES-1.1; a procedural transition is not required at this time.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 88 H

3 X

Y Y

N E

K/A 007A2.04

1. Q=K/A, Q=SRO-only.

2. Recommend slight tweak to the second part Q statement as follows:

Specific procedural steps to lower PRT pressure are contained in ___(2)___.

(what the CRS will do rears its ugly head again)

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 89 H

2 X

Y Y

N E

K/A 013G2.2.22

1. Q=K/A, Q=SRO-only.

2. The A(2) and D(2) distractors are worded awkwardly which minimizes plausibility: how would a containment isolation signal cause containment flooding? Recommend re-write to talk about isolating [what is true at Robinson? CCW, or CCW and SW?] into containment to prevent containment flooding beyond design basis, or words to that effect.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 90 H

2 X

X Y

Y B

E K/A G2.2.38

1. Q=K/A, Q=SRO-only.

2. Do we need to provide a reference to the applicants for this Q? Lets discuss.

3. Subset issue: distractor B is logically equivalent to distractor C. Not sure how to fix without throwing off psychometric balance. Due to the subset issue, one of these distractors is non plausible.

Q evaluated as E at this time due to one non-plausible distractor.

Facility Licensee Modified Q as requested by NRC. Distractors B, C, and D required modification in order to resolve the single non-plausible distractor issue. Based upon discussion with facility licensee, agree that references not required for this Q. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 91 H

3 X

Y Y

M E

K/A 016G2.1.7

1. Q=K/A, Q=SRO-only.

2. Need to include a, in correct answer B.

3. In Q statement specify current required action(s) due to the given conditions?

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 92 F

2 X

Y Y

N E

K/A 043A2.03

1. Q=K/A, Q=SRO-only.

2. Recommend modification of first part Q statement for improved parallelism with procedure FMP-019 as follows: IAW FMP-019, if a fuel assembly must be placed in a Temporary Fuel Storage locations, it is required to obtain approval from ___(1)___.

Q as submitted evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 93 H

2 X

X X

Y Y

B E

K/A 086A2.01

1. Q=K/A, Q=SRO-only.

2. First bullet can delete mode 1, just say at 100% power.

3. Minutia potential: lots of potential for minutia here based on asking action statements greater than 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months actions without a reference. Recommend adding reference paged from FP-012 as required to analyze the situation (or just provide entire procedure?)

4. Recommend rewrite of the Q statement to prevent potential for multiple correct answers as follows:

Based on the given conditions, if the status of the fire protection system remains unchanged which ONE of the following identified the most restrictive action(s) required IAW?

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 94 F

2 X

X X

Y Y

B E

K/A G 2.1.15

1. Q=K/A, Q=SRO-only

2. Subset issue: it is technically correct that if the CRS or above can approve and sign [a] revision, then the SM or above can approve as well. That makes CRS non-plausible as a distractor. Giving some benefit-of-the doubt here due to ease of correction. Recommend revision of the second part Q statement dealing with revisions to more closely parallel the procedure step.

Consider the following:

IAW AD-OP-ALL-0111, if a revision to a Standing Instruction is issued, the CRS ___(2)___ [is/is NOT] the lowest level authorized to review and sign the revision.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 95 F

1 X

X Y

Y B

U K/A G 2.1.35

1. Q=K/A, Q=SRO-only.

2. Q is LOD=1/multiple non plausible distractors. The Q can be answered simply by recognizing that distractors A, B, and D are functions performed in the control room, not on the refueling bridge. Distractor B further highlights this/provides a cue by overtly stating stationed in the Control Room. It is common knowledge for any operator at a nuclear plant that the refueling SRO is required to be in containment directly supervising the core alterations process.

Q as submitted evaluated as U due to LOD=1 and multiple non-plausible distractors.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 96 F

2 X

Y Y

B S

K/A G 2.2.18

1. Q=K/A, Q=SRO-only.

2. Recommend slight tweak to first part Q statement: must be implemented during the shutdown is when ___(1)___ is entered.

Q as submitted evaluated as S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 97 H

3 X

Y Y

B E

K/A G 2.2.240

1. Q=K/A, Q=SRO-only.

2. For consistency between all Qs, recommend using consistent format for the TS terms MODE 4 vs. Mode 4 - some Qs have had all caps, and some have not. Recommend using all caps when referencing TS MODEs.

3. Second part Q statement minor tweak recommend Based on the given timeline, the LATEST rather than Based on this Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 98 F

2 X

Y Y

B E

K/A G 2.3.6

1. Q=K/A, Q=SRO-only.

2. Subset issue of authority with first part Q. If the SM is required to sign (which is a given logical true in the Q statement) then if the CRS is required to sign (lower than SM) the EC Supervisor (additional level of authorization) may also be required to sign. In other words, the CRS signature would be required for any permit, the EC Supervisor would only be required for a permit that is more complicated or higher rad release levels, which is the condition given in the stem. Recommendation: take out in addition to the Shift Manager as this is essentially teaching in the Q. Consider the following modification: The EC Supervisor ___(1)___ [is/is NOT] required to sign this Release Permit.

Q evaluated as E at this time.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021 99 H

2 Y

Y B

S K/A G 2.4.30

1. Q=K/A, Q=SRO-only.

Q as submitted appears S at this time.

ROBINSON 2021-301 Written Examination Review Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 100 H

2 X

X Y

Y B

E K/A G 2.4.40

1. Q=K/A, Q=SRO-only.

2. Fifteen minutes following EAL declaration to get all accountability performed is not plausible; its barely enough time to get the forms filled out and the state and local governments called. Recommend using 30 minutes/60 minutes, see suggestion below. Giving some benefit-of-the-doubt here due to ease of correction.

3. Big cueing in the Q statement when we ask if site evacuation is required or not, and then in the very next statement we say that site evaluation was initiated.

4. Recommend just straightforward ask the requirements without any given information or timeline, consider the following:

IAW AD-EP-ALL-0111, CONTROL ROOM ACTIVATION OF THE ERO, which ONE of the following correctly completes the statements below?

Site Evacuation is required for EAL declarations of ___(1)___ [ALERT/SITE AREA EMERGENCY] or higher.

When accountability is required, the LATEST time at which all personnel in the Protected Area must be accounted for is ___(2)___ [30 minutes/60 minutes]

following EAL declaration.

Facility Licensee Modified Q as requested by NRC. Q now appears ready for administration on the NRC written exam. MKM 08/30/2021

ROBINSON 2021-301 Written Examination Review Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6.

Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7.

Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8.

At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

ML22011A173

Text

Navigation menu

Search