ML23115A340

4-CGS-2023-02 Form 2.3-5 Written Exam Comments Final

ML23115A340
Person / Time
Site:	Columbia
Issue date:	03/09/2023
From:	Heather Gepford NRC/RGN-IV/DORS/OB
To:	Energy Northwest
References
Download: ML23115A340 (1)

Text

Form 2.3-5 Written Examination Review Worksheet 1

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only General comments: 1. for Last NRC Exam mark N/A as appropriate. 2. Correct answer choice distribution should be approximately equal among the four choices.

1 H

2 N

E NRC:

1. Should the word automatically be underlined or emphasized to make clear that answer choice D could not be argued as also correct (since it would accomplish the same thing)?

2. ABN-RRC-LOSS contains the entire answer; therefore suggest stem include IAW ABN-RRC-LOSS, what is the basis for..

3. Noun names should be included for the following:

RRC-P-1A ABN-RRC-LOSS RRC-CB-RPT3A RRC-CB-RPT4A Assume 3A is main breaker and 4A is field breaker-another reason for noun names.

4. Explanation Section:

Typo in B and D - replace is either with if either

1. Underlined automatically.

2. Modified as suggested.

3. Added noun names.

4. Corrected typos.

Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 2

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 2

F 2

X N

E NRC:

1. Distractors are subsets of each other. Need to add only to distractors A, B, C. Potential for more than one correct answer.

Minimum wording in stem provides coverage, but ONLY ensures a single correct answer.

2. ELAP requirements are procedure entry condition.

Recommend adding IAW PPM 5.6.1

3. Use of numbers to replace DG may add more confusion. Validation comments?

1. & 2. Revised as recommended

3. No comments during validation.

Revised as recommended for clarity. Need to revise A. Distractor A revised as discussed Accepted by NRC 3

H 3

X N

E NRC:

1. Noun names should be included for procedure, battery, and battery chargers

2. Procedure caution provides answer, recommend stem include IAW ABN-ELEC-125VDC

1. Included noun names.

2. Added ABN-ELEC-125VDC to stem per suggestion.

Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 3

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 4

H 3

X N

S NRC:

1. Use Procedure noun name

2. Revise to Expect an automatic trip of the main turbine Revised Accepted by NRC 5

H 4

X N

S NRC:

1. Distractor C is a potential second correct answer. Injection of SLC is BEFORE wetwell reaches 207F.

SLC injection would not be expected but allowable by procedure. Revise stem to make it clear that the correct answer choice is required vice allowed or should.

Second correct answer (C) is reason for U rating. Possible distractors 1 only, 2 only, 3 only, 1,2,and 3.

CGS - Revised to Which of the following actions are required to be performed? Revised selections as recommended:

a. (1) ONLY

b. (2) ONLY

c. (3) ONLY

d. (1) and (2) and (3)

Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 4

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 6

H 3

X N

S NRC:

1. Noun names for pump, procedure, and valves.

2. Remove second person reference - YOU - replace with the RO is directed to start SW pump.

3. Stem indicates ALL of the immediate and subsequent actions are completed - however steps remain in Attachments?

4. In B(2) and D(2) change SW-V-2B to the valve

5. This question tests the same concept as Q47.

1. Noun names added

2. Revised as recommended

3. Removed subsequent actions from statement

4. Revised as recommended

5. Replaced Q47 Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 5

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 7

H 4

X N

S NRC:

1. Typing error on page 1 in K/A Match statement - change if to of.

2. Noun names for procedures and equipment.

3. Distractor A different than other distractors with reference to IAW ABN.

4. Explanation indicates immediate actions for ABN-ELEC-SM3/SM8 are completed; however, this ABN was never acknowledged in the stem? If the RO performs Immediate Actions, wouldnt the CRS enter this ABN also to verify those actions?

5. Double jeopardy to Q47. Since same knowledge is required for operation of RCC-V-6, this question is rated unsat. Only one of the two questions would be Unsat.

CGS

1. Corrected

2. Noun names added. Recommend removing them for clarity

3. Revised to remove ABN reference

4. Wrong procedure listed. Revised to ABN-ELEC-SM1/SM7

5. Replaced Q47 Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 6

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 8

F 2

X N

S NRC:

1. Is Distractor D, the correct answer as the stem is written? Step 5.6.2 directs VERIFY at 20, this seems to indicate CIA is at 20?

Distractor D is verifying the CIA program is at 20 - seems like the correct answer. Next step to manually reset to 20 is only performed if REENERGIZING CIA programmers. What indications within the stem indicate CIA programmers need to be reenergized?

ABN-CIA is used for all CIA issues.

If the bottles are placed in service (as indicated in the stem), the programmer will not be in 20 and must be manually realigned.

Accepted by NRC 9

H 3

X N

S NRC:

1. Second correct answer -

distractor D is correct based on answer explanation: While the entry conditions are technically met for entry into ABN-ADHR Nothing in the stem indicates RPV level can be restored, which would allow SDC loop restoration, making distractor D correct.

Form 2.3-5 Written Examination Review Worksheet 7

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only Second correct answer is reason for U rating Corrected Distractor D answer explanation. Based on the stem, RHR B is in SDC, therefore, an Alternate Shutdown Cooling method is NOT in use. The entry conditions for ABN-ADHR have not been met which makes entry into ABN-ADHR an incorrect answer.

Accepted by NRC 10 H

3 X

N E

NRC:

1. L.O.D. is missing. Note: for Last NRC Exam, change N/S to N/A

2. Stem should specify actions IAW PPM 6.3.2

3. RO - REQUEST bridge crew stop - should be DIRECTS insertion to stop - procedure problem not question problem.

1. LOD has been added and N/S corrected to N/A

2. Stem modified as suggested.

3. Added to file containing items to be fixed after ILC-25.

Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 8

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 11 F

2 X

N E

NRC:

1. Remove second person reference

- third person - the RO is directed to spray the DW.

2. Part 1 should reference FSAR design requirements.

3. Part 2 IAW PPM xxx chugging is prevented by

4. Potential double jeopardy with question 23. Lets discuss.

1. Reworded stem to remove second person reference.

2. Referenced FSAR in stem.

3. Referenced PPM 5.0.10A in stem.

Accepted by NRC 12 H

3 N

E NRC:

1. Noun names for procedures.

2. Alarm in stem is a single alarm for Hi/Low - applicant needs to determine a High condition exists?

3. PPM 5.5.23 procedure name is different from attached reference.

4. Correct answer is based on simulator testing? Are the applicants trained on this testing?

5. The learning objective appears to be incomplete.

1. Added noun names.

Form 2.3-5 Written Examination Review Worksheet 9

Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only

2. Applicant can determine that a high condition exists because pressure control is with SRVs.

3. Corrected name.

4. Reworded simulator testing to simulator plant response.

5. Completed learning objective.

Accepted by NRC 13 H

4 N

S NRC:

1. In the question stem, first sentence, add the before Wetwell.

2. Condition 1 is an entry condition for Primary Containment Control, which is memory (L) level.

3. Provides assistance in answering Q59

4. Q24 Double Jeopardy - Same wetwell entry condition is contained in both 13 and 24, this is the reason for U rating.

1. Corrected.

2. The question has two parts. It requires knowledge of entry conditions for PPM 5.2.1 and knowledge of when a scram is required by PPM 5.2.1

3. Q59 revised.

4. Q24 revised to remove DJ.

Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 10 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 14 F

2 X

N S

NRC:

1. LOD 2, Question is simply a DW temperature entry condition for Primary Containment Control -

similar to question 13. Not a Higher order question.

2. K/A issue, window dressing? -

stem is not required to answer the question. Annunciators not required to answer question.

Changed K/A and question. LOK -

F Accepted by NRC 15 H

2 N

S 16 H

3 X

N S

NRC:

1. Distractors C and D are not plausible -50 inches. RPV level of -

50 is so far below a normal level or scram/EOP entry level, it is not plausible. This is the reason for U rating.

2. Consider one RPV water level instrument fails low - half scram, full scram. Is there a low level alarm before the RPS trip when out of the normal level band?

Changed Distractors C and D from -50 to +31.5, the setpoint for the low reactor water level alert alarm, as suggested.

Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 11 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 17 H

3 N

S 18 F

3 X

N E

NRC:

1.Only answer choices B and C are underlined in the Explanation section. For the entire test make the underlining consistent in the Explanation section.

2. Distractor C is not plausible. All the other distractors involve air flow operations, this is a change to cooling water operation - unrelated.

Another CR HVAC mode would be a plausible distractor - PURGE, Isolate - verify automatically shifted to RECIRC

1. Removed underlining here and throughout document.

2. Made Distractor C plausible by changing it to a type of CR Ventilation mode.

Accepted by NRC 19 H

3 N

S NRC:

1. Distractors B and D explanations for Appendix R actions is incomplete. Supporting material provides the explanation but should be captured in the explanation section.

2. Could choice B be argued as also correct since the Note says if youre

Form 2.3-5 Written Examination Review Worksheet 12 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only unsure if the fire is in an Appendix R area, to treat the fire as if it is?

1. Revised explanations.

2. Appendix R fire locations are defined in ABN-Fire. Main Xfmr deluge is not in this list. This is expected knowledge based on Appendix R fire definition.

Accepted by NRC 20 H

4 N

S NRC:

1. 55.41 content is incomplete.

1. Completed 55.41 content.

2. Note K/A wrong and correct it (70000 to 700000).

Accepted by NRC 21 H

4 N

S NRC:

1. +13 to +54 inch level band in stem provides a cue that +13 inches is NORMAL, which provides a cue for answering Q16 - RPV Control entry on RPV level. Consider replacing with level restored to normal post scram level band.

Revised question to specify normal post scram level band vice +13 to

+54 inches. Revised explanation of

Form 2.3-5 Written Examination Review Worksheet 13 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only A(2) to include normal band (+13 to

+54).

Accepted by NRC 22 F

2 N

S NRC:

1.The answer explanation for D(1) should say See B(2) above, not B(1). The answer explanation for D(2) should say See C(1) not C(2).

Corrected explanations for D:

D(1) Correct - see C(1) above D(2) Correct - see B(2) above Accepted by NRC 23 H

2 X

N S

NRC:

1. Double jeopardy to Q11 on DW temperature and chugging for distractors B and D. Consider replacing B2 and D2 with another component or reason - DW hardened vent, DW vent/purge valves.

Revised B(2) and D(2) with another reason to remove double jeopardy.

Revised LOK to H Accepted by NRC 24 H

2 X

X N

S NRC:

1. Another EOP entry condition question - Primary Containment Control. Double Jeopardy to Q13 -

Form 2.3-5 Written Examination Review Worksheet 14 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only a near match. Same wetwell entry condition is contained in both 13 and 24, this is the reason for U rating. One of the two questions would be Unsat for repetition.

Consider a question on BIIT may be appropriate to fit K/A and eliminate DJ.

Rewrote question to remove double jeopardy while still meeting the K/A.

The question now concerns making an operational call on when to perform boron injection based on high suppression pool temperature.

Accepted by NRC 25 H

4 X

N S

NRC:

1. Repeat question on loss of FW heating - Q60.

Since same loss of FW heating impact on reactivity is tested, this question was rated U. Only one of the two question would be Unsat for repetition.

Replace either Q60 or Q25 -

resample if necessary.

Form 2.3-5 Written Examination Review Worksheet 15 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only CGS - 1/16/23 - Replaced question 60 Accepted by NRC 26 H

3 X

B E

NRC:

1. Earliest time? If so wouldnt Distractors A and B also be correct since they are earlier and obviously still meet all of the T.S.

requirements? If true more than one correct answer - U Fix is to change to LATEST time -

maybe.

Changed Earliest to latest in 2 places Accepted by NRC 27 H

4 X

N E

NRC:

1. Answer explanations are not correct - wrong answer is marked as correct answer and explanations for the four distractors do not apply to the question.

2. K/A match justification is for a different K/A.

1. Answer explanations have been corrected and correct answer is marked properly.

2. K/A match now aligns with K/A.

Accepted by NRC

Form 2.3-5 Written Examination Review Worksheet 16 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 28 H

4 X

N S

NRC:

1. Noun name for HX bypass valve

2. RV-53B is referenced for Distractor D explanation; however, it is never referenced in the question.

3. Only the HX bypass valve receives the LPCI signal - no other component - plausible?

1. Revised by adding the noun name to RHR-V-8, RHR-V-9,and RHR-V-48B.

2. Revised Distractor D to remove RHR-V-53B and add RHR-V-48.

3. Revised event to be a Div 2 LPCI initiation with RHR-C OOS.

This ensures RHR-V-48B receives plausible signals.

Accepted by NRC 29 F

3 N

S 30 F

2 B

S 31 F

2 N

S 32 F

2 N

S 33 F

2 N

S 34 F

2 N

S NRC:

1. Is number 2 a credible distractor?

If not, then that cues which answer

Form 2.3-5 Written Examination Review Worksheet 17 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only is correct as only one answer choice does not have (2) in it.

Revised by deleting the questionable credible distractor and making 1 of the 3 correct responses incorrect (#2). Revised selections as:

a. (1) ONLY

b. (2) ONLY

c. (1) and (3) ONLY

d. (2) and (3) ONLY Accepted by NRC 35 F

3 N

S For answer choices C and D, is the explanation for part 1 correct?

Clarified explanations for C and D.

Accepted by NRC 36 H

4 N

S 37 F

2 X

B E

NRC:

1. Distractor D is similar to early ADS question - Q29. Not a credible distractor.

Consider resample - ADS Logic oversample?

1. Changed distractor D to remove

Form 2.3-5 Written Examination Review Worksheet 18 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only jeopardy with Q29.

Accepted by NRC 38 F

3 X

B E

NRC:

1. Delete the first question, How is the

2. L.O.D. is missing.

3. Should the MSIV switch position be addressed since it will NOT reset the INBOARD if C/S is not in CLOSE position.

1. Question deleted.

2. Added L.O.D.

3. Added to stem that all MSIV control switches are in CLOSE.

Updated Explanation and Technical References.

Accepted by NRC 39 F

2 N

S 40 H

3 N

S 41 F

2 N

S 42 H

3 N

S 43 H

3 X

X M

S NRC:

1. Change conditions to condition(s) in the question.

2. Potential cueing; look at three option questions for pattern - it seems to NEVER be just a single

Form 2.3-5 Written Examination Review Worksheet 19 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only answer which eliminates A and B -

How many three item questions?

3. DJ with Q52.

4. Since in-synch check/synch light knowledge is tested for UPS inverter in both questions, this question was rated unsat. Only one of the two questions are unsat for repetition.

Resample if necessary.

Replaced Question. Revised to Modified.

Accepted by NRC 44 H

2 X

N E

NRC:

1. Distractor A is not plausible - the loss of one MC causes trip of ALL supplying battery chargers?

Recommend using explanation plausibility for another distractor -

DC bus supplied with alternate charger - an auto swap - which does not occur according to the explanation.

Updated distractors to make them all plausible. Included noun names and EPNs. Updated two of the distractors to include auto swap to alternate charger as suggested.

Updated reference.

Form 2.3-5 Written Examination Review Worksheet 20 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only Accepted by NRC 45 H

3 B

S 46 H

2 X

B E

NRC:

1. Distractor C is NOT incorrect - it will turn off after 30 minutes -

without a timeframe specified -

could be a second correct answer.

1. Changed Distractor C and explanation to make it plausible but incorrect.

Accepted by NRC 47 H

2 X

M S

NRC:

1. Q7 reviewed the same V-6 isolation as a result of less than 2 RCC pump in operation Since same knowledge is required for operation of V-6, this question is rated unsat. Only one of the two question would be Unsat. Same concept - Double Jeopardy.

2. Potential K/A mismatch. I am not seeing a strong link between in the question to CCW. Replace question or resample if necessary.

Form 2.3-5 Written Examination Review Worksheet 21 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only CGS Replaced question. Question revised to Modified.

Accepted by NRC 48 H

3 X

N E

NRC:

1. If valve is INADVERTLY closed it must have been OPEN. Change to RO reports V-68 is closed - which the applicant must determine if the valve is normally open or closed -

making no change to SW flow plausible.

Changed stem as suggested to BOP RO reports RHR-V-68B (RHR-HX-1B Service Water outlet valve) is closed.

Accepted by NRC 49 H

2 X

N E

NRC:

1. Same valve logic as Q58.

Double jeopardy?

Changed Q58 to remove double jeopardy Accepted by NRC 50 H

3 B

S 51 H

4 N

S 52 H

4 X

N S

NRC:

1. Q43 involves in-synch check for inverters - double jeopardy.

Form 2.3-5 Written Examination Review Worksheet 22 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only Since in-synch check/synch light knowledge is tested for UPS inverter in both questions, this question was rated unsat. Only one of the two questions are unsat for repetition.

Replace either Q52 or 43.

Replaced Q43 Accepted by NRC 53 F

3 N

S 54 H

3 B

S 55 H

3 N

S 56 F

2 N

S NRC:

1. Should ABN-RPIS should be attached as a reference to verify correct answer action?

Revised by adding ABN-RPIS as a reference to the answer explanations.

Accepted by NRC 57 F

2 N

S

Form 2.3-5 Written Examination Review Worksheet 23 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 58 H

3 N

E NRC:

1. Same valve logic as Q49.

Double jeopardy?

1. Removed double jeopardy by linking question to Suppression Pool Cooling with Discharge to Rad Waste to lower wetwell level. Changed valve from RHR-V-42B to RHR-V-49.

Accepted by NRC 59 H

3 X

X B

S NRC:

1. Is there a reference that states to start the pump vice letting level get to the TAF and then starting it?

Could answer choice B be argued as a subset of answer choice A?

Add the word Immediately to A(2) and C(2).

2. Part 2 of the question does not match the part 2 answers. Part 2 asks what is the effect, but the answers are actions.

3. Provides assistance in answering question 13 to assess Wetwell temperature limit of 225F and/or vice versa.

1. Reference to PPM 5.0.10A added. Immediately added as recommended.

Form 2.3-5 Written Examination Review Worksheet 24 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only

2. Revised to say how should RHR pump C be operated?

3. Revised WW temp to 195F. This question focuses on WW level limits while Q13 focuses on WW temp limits. Additionally, different ECCS pumps have different limits.

Accepted by NRC 60 H

2 X

B S

NRC:

1. Repeat question on loss of FW heating - Q25 Since same loss of FW heating impact on reactivity is tested, this question was rated U. Only one of the two question would be Unsat.

Replace either Q60 or Q25 -

resample if necessary.

CGS - Replaced question. LOK changed to H Accepted by NRC 61 F

3 N

S 62 F

2 B

S 63 F

2 B

S NRC:

1. Three part question construction. Is there a way to

Form 2.3-5 Written Examination Review Worksheet 25 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only make this a 1x4 or regular 2x2?

1. Revised construction to a regular 2x2 as suggested.

Accepted by NRC 64 F

2 B

S 65 F

2 B

S NRC Exam 2017 NRC:

1. Incomplete answer - action requires operator to initial the step.

Revised to include initialing step.

Accepted by NRC 66 F

2 B

S NRC Exam 2021 67 H

2 B

S NRC Exam 2019 68 H

3 B

S 69 H

2 X

B S

NRC:

1. Question uses EAL Notifier -

the procedure calls the individual the Offsite Notifier - align question with procedure.

Revised as recommended Accepted by NRC 70 F

2 N

E NRC:

1. Distractors B and D explanations are incorrect - when a CR is

Form 2.3-5 Written Examination Review Worksheet 26 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only withdrawn on the approach to criticality SRM counts RISE, as a result of subcritical multiplication, period decays to infinity. According to this explanation, a startup would never occur.

2. LOD-1?

1. Question has been changed:

The rod movement occurs during a startup and not at power. All distractors are now plausible and incorrect.

2. Changed LOD to 2.

Accepted by NRC 71 F

2 B

S NRC GFE B558 NRC:

1. Exam Outline Cross-Reference is marked as 70, change to 71.

2. No operational validity established Xenon, Sm, Gadolinium, Boron changes during power operations or core lifetime could be used to cover K/A

1. Revised

2. Revised to provide operational validity and address KA.

Accepted by NRC 72 F

2 B

E NRC GFES Bank B1065

Form 2.3-5 Written Examination Review Worksheet 27 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only NRC:

1. Question should be listed as a Modified Question since its source is the NRC GFE Bank. NRC question would not contain CGS or CGS procedure reference.

2. Add original GFE question to references.

3.Three part question with single answer. Is there another way to write this questiona 1x4?

1. Revised

2. Added

3. Unsure of 4th plausible reactivity addition method.

Change to bank configuration Revised as recommended Accepted by NRC 73 H

2 M

E NRC GFE Bank B286 NRC:

1. No operational validity established. Use a CGS shell and tube heat exchanger where flow is raised to create a CGS tie.

Revised as recommended.

Revised to TSW HX

Form 2.3-5 Written Examination Review Worksheet 28 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only Revised to TG H2 Cooler, which uses TSW as the cooling water.

Accepted by NRC 74 F

3 B

S NRC GFE bank #B1789 NRC:

1. Question should be listed as a Modified Question since its source is the NRC GFE Bank. NRC question would not contain CGS or CGS procedure reference.

2. Add original GFE question to references.

3. Not operational validity, RR flow changes should allow a tie to CGS operations.

1, 2, & 3 Revised as recommended Accepted by NRC 75 F

2 B

E NRC GFE bank #B995 NRC:

1. Question should be listed as a Modified Question since its source is the NRC GFE Bank. NRC question would not contain CGS or CGS procedure reference.

2. Add original GFE question to references.

3. Not operational validity.

1. Revised as recommended

Form 2.3-5 Written Examination Review Worksheet 29 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only

3. At CGS, reactor pressure is adjusted using DEH. Not sure why this is not operationally valid.

Additionally KA asks for effects on CPR due to reactor pressure changes Look at potential questions from the NRC bank This is the best bank question. No other changes required.

Accepted by NRC 76 H

2 N

E Imbedded reference.

NRC:

1. I dont think entry conditions for ABN-LEVEL are met, but if they are, it is a slippery slope on whether B would also be correct, because there is no trend information available on level.

2. Add noun names for the procedures

1. Improved explanation on why entry into ABN-LEVEL is plausible. Removed the level indication and stated the level and trend in the stem.

2. Added noun names as suggested.

Accepted by NRC 77 H

2 N

S

Form 2.3-5 Written Examination Review Worksheet 30 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only 78 F

2 N

S NRC:

1. Not a comprehensive level of knowledge. Knowing a setpoint in a procedure.

79 F

3 X

N S

NRC:

1. A and C not plausible. The stem of the question states, due to a loss of forced circulation which mean no RRC pumps are in operation.

2. Add noun name of the procedure.

1. Added ABN-RHR-SDC-LOSS and removed due to loss of forced core circulation flow in the question making A and C plausible.

2. Added noun names of procedures.

Accepted by NRC 80 H

3 X

N S

Imbedded reference.

NRC:

1. Cueing the correct answer. The correct answer includes the words SRV tail pipe which is also the title of the graph. Also, would not call this a good K/A match, as you are not determining, or interpreting, reactor pressure based on SP level.

Rewrote question to match K/A and remove cueing.

Form 2.3-5 Written Examination Review Worksheet 31 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only Accepted by NRC 81 H

3 N

E NRC:

1. I struggle with this as an SRO question because I think it is RO knowledge on what level requires ED.

Rewrote question to which action should the CRS direct, without PPM 5.1.1 reference. Strengthened distractor analysis and added ABN-RPS, OI-15, and HPCS Pump Curve as references.

Accepted by NRC 82 H

3 M

S Reference Provided.

NRC:

1. I would call this modified bank of question 89 on the 2019 NRC exam.

Revised as recommended Accepted by NRC 83 F

4 N

E NRC:

1. Rather than state offgas flow is higher than normal, provide an actual parameter. I think by stating it is higher than normal cues that you need to check excessive offgas air purge flow.

Revised offgas flow to 90 SCFM, up slow - Normal flow is approximately 40 SCFM

Form 2.3-5 Written Examination Review Worksheet 32 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only Accepted by NRC 84 H

3 N

S 85 H

2 N

S Reference Provided.

86 H

3 N

E NRC:

1. Stem says A is the correct answer, but distractor analysis says C is the correct answer.

A is the correct answer. Revised distractor analysis Accepted by NRC 87 F

2 N

S 88 H

3 N

S Reference Provided.

89 H

2 N

E NRC:

1. The distractor analyses does not match the distractors, plus they state that C is the correct answer, but D is listed as the correct answer.

D is the correct answer. Corrected distractor analysis.

Accepted by NRC 90 H

2 N

E NRC:

1. Might want to clarify in the stem that RCC-P-1B is TAGGED out of service for maintenance to make it perfectly clear condition of the pump breaker. Is it

Form 2.3-5 Written Examination Review Worksheet 33 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only necessary to provide noun names for the valves?

2. Duplicates questions 7 and 47.

1. Clarified stem as suggested.

Added noun names.

2. Questions 7 and 47 were modified to remove double jeopardy.

Accepted by NRC 91 H

3 N

E NRC:

1. This might be some overlap with Scenario 2. Though that rod is drifting out, it is overlap enough to eliminate A as a distractor. I really struggle with A as a distractor, as you are asking to scram with one control rod not moving, and when it did move, it moved in the conservative direction. 2. In the stem change the first in to is.

1. Revised distractor A.

2. Revised as suggested.

Accepted by NRC 92 F

3 X

N U

Reference Provided.

NRC:

1. All of the actions contained in 3.3.2.1 C are immediate, thus must be known from memory.

By providing the tech spec, the correct answer is the only

Form 2.3-5 Written Examination Review Worksheet 34 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only distractor that contains words from the tech spec.

Revise to add C. to question stem and change C. to no additional actions.

Revise as recommended. Removed reference.

Accepted by NRC 93 H

4 N

S 94 H

2 B

S Prior 2 NRC exams.

95 H

3 B

E Prior 2 NRC exams. Reference provided.

NRC:

1. Like the question, but I would like to see the control rods be different than the bank question. Would not make it modified, but at least it would be different.

Revised as recommended Accepted by NRC During post examination analysis licensee staff identified a typographical error that resulted in two correct answers. Answer choice A was intended to contain control rod 46-43 but mistakenly contained control rod 46-47 (an

Form 2.3-5 Written Examination Review Worksheet 35 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only adjacent control rod). Therefore, distractor A is also correct.

96 F

2 B

E NRC:

1. I think this question works better if you eliminate distractor D (not wild about adjusting the setpoint as a distractor if you dont know what the annunciator is). I would then list the other distractors in a table, and have the question be like a) 1 only, b) 1 and 3 only, etc. (like question 79). NRC Exam 2017 Revised as recommended Accepted by NRC 97 H

4 N

E NRC:

1. Dont think it is plausible that a something classified as high risk would not require a risk mitigation plan (distractor D).

High risk, emergent work does not require a mitigation plan. Plausibility statements have been modified to reflect this.

Accepted by NRC 98 H

3 N

E NRC:

1. Would personally prefer to see two additional dose limits as choices and go with a 1x4 question. Seems to me

Form 2.3-5 Written Examination Review Worksheet 36 Q

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6. Source (B/ M / N)

7. Status (U /E /S)

8. Explanation Stem Focus Cues T/F Cred.

Dist Partial Job-Link Minutia

1. /

Units Back ward Q -

K/A SRO Only nothing could be mandatory when it comes to personal radiation safety.

Could go 15 rem (which is limit to eye) and 50 rem (which is limit to skin).

Different question selected Accepted by NRC 99 F

2 B

E NRC:

1. Would distractor D be better if it stated only after immediate operator actions of the reactor scram procedure.

I question why this is SRO level, since the NRC exam from the bank said it was an RO question, and was answered correctly by all 10 applicants in 2017.

Although it was on the RO exam in 2017, the question meets the requirements of an SRO question. It appears that the ROs took an SRO question in 2017.

Accepted by NRC 100 F

2 N

S NRC:

1. Not a comprehensive level of knowledge. It is knowing a specific responsibility,

1. Changed to F.

Form 2.3-5 Written Examination Review Worksheet 37 Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

• Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

• Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

• T/F: All of the answer choices are a collection of unrelated true/false statements.

This is an (U)nsatisfactory question.

• Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

• Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

Form 2.3-5 Written Examination Review Worksheet 38

4. Check the appropriate box if a job content flaw is identified:

• Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

• Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

• #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

• Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.