ML22251A023

Form 2.3-5 Written Examination Review Worksheet DC-Cook 2022

ML22251A023
Person / Time
Site:	Cook
Issue date:	08/10/2022
From:	Ted Wingfield NRC/RGN-III/DRS/OLB
To:	Indiana Michigan Power Co
Wingfield T
Shared Package
ML21263A041	List: ML22250A720 ML22251A016 ML22251A019 ML22251A020 ML22251A021 ML22251A023 ML22251A027
References
Download: ML22251A023 (15)

Text

Page 1 of 15 Form 2.3-5 Written Examination Review Worksheet Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 1

F 3

X B

U E

S FREE SAMPLE NRC:

GENERAL COMMENT

FOR ALL ITEMS - The question pedigree should include the following items (included in NUREG 1021, Rev. 12 Form 4.2-1, Written Examination Question Worksheet):

Which Tier & Group (i.e. T1/G1 for this Q)

Provided References (i.e. None for this Q)

Question Source (Bank/Modified/New B/M/N i.e.

B for this Q)

Related 10CFR55 Content (i.e. 55.41(b)(5) for this Q)

GENERAL COMMENT

FOR ALL ITEMS - The references provided are not complete enough to adequately assess the accuracy of the answers and distractors.

The question is not a KA match. Specifically, the KA asks: Why is ECCS flow reduced after a Reactor Trip? while the question asks: What happens to subcooling when ECCS flow is reduced during a SBLOCA?

Response: Question changed to ask about how and why ECCS flow is reduced when in ES 1.1.

NRC: The introductory statement is not necessary. Move Safety Injection Termination to between ES-1.1 and directs.

This question tests memory of procedure steps and their bases and therefore is testing fundamental knowledge (not higher cognitive knowledge).

Response: Moved ES-1.1 Safety Injection Termination, removed introductory statement, and designated as fundamental knowledge.

2 H

3 N

E S

NRC: Add additional between With no and operator action.

If the thumb rule is 0.5%/min, why is 1.0%/min plausible?

Response: Added additional as requested. 0.5%/min is not thumb rule but is from calculating 5% change in 10 minutes.1%/min is plausible as potential math error.

Page 2 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 3

F 3

B E

S 2004 NRC EXAM Q16 NRC: This question test memory of the bases for procedure steps and therefore is testing fundamental knowledge (not higher cognitive knowledge).

Response: Designated as fundamental knowledge.

4 F

3 X

N U

S NRC: This is a K/A mismatch. The K/A is to know WHY the level band is what it is. The question asks WHAT the level band is.

Change the question answer options so they do not overlap (ref.

ES-4.2 D.12).

Make sure the answer options use the same style wording -

narrow range SG between vs. narrow range SG levels between (ref. ES-4.2 D.15.i).

Somehow SG level should be clarified to NOT BE <13%. What is initial S/G level?

Response: Reworded stem to ask how SG inventory is maintained and why? Did not need to address initial SG level.

5 H

4 N

S 6

H 2

N S

7 H

2 X

N U

E S

FREE SAMPLE NRC: Distractors B and C are not plausible: When, if ever, are the RCPs tripped BEFORE the Reactor? Isnt it fundamental/basic knowledge that anything less than all RCPs running = Reactor Trip for Westinghouse?

Response: Question stem modified and implausible distractors were replaced.

NRC: Include a clearer transition between initial condition statement and the fault conditions in which the question is based.

Perhaps after the first IC statement preview the fault conditions with the word, Subsequently Response: Combined initial and fault conditions into one bulleted list of current conditions.

8 F

2 B

S

Page 3 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 9

H 4

N E

S FREE SAMPLE NRC: The answer explanations seem to be in the wrong order:

The explanation currently in A should be in C. The explanation currently in C should be in D. The explanation currently in D should be in A.

Response: Distractor analysis has been corrected.

10 F

3 B

S 11 H

3 N

E S

NRC: Make sure the answer options use the same style wording

- right now, 3 of the options refer to throttling the ESW cooling valve and one refers to throttling the CCW flow control valve (ref.

ES-4.2 D.15.i).

Add open or closed to the throttle statements to indicate which way the valve is being throttled.

Response: Standardized wording of valve description and added open to throttling direction.

12 H

4 B

E S

2008 NRC EXAM Q80 NRC: This is a bank question from the 2008 NRC exam. Update pedigree page to indicate bank.

Response: Identified question source as Bank.

13 H

3 B

S Prev 2: 2020 NRC EXAM Q12 14 F

2 N

S 15 H

3 B

E S

NRC: Include the applicable E/APE with the Generic K/A.

Response: Added APE to KA designator.

16 H

4 B

S 2012 NRC EXAM Q60

Page 4 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 17 F

2 X

N U

S NRC: Why is Steam Line D/P Safety Injection a plausible distractor during a normal plant cooldown?

How is what is in the Westinghouse material relevant to D.C.

Cook? Are the operators trained on what is NOT APPLICABLE to D.C. Cook?

Response: Changed stem to The crew is commencing the plant cooldown using Steam Generator Power Operated Relief Valves (PORVs). This makes Steam Line DP plausible.

Statement for Westinghouse material was added as DC Cook has different P designators. P-11 is associated with Pressurizer Pressure low where P-12 is associated with the block of steam Line Pressure Safety Injection.

18 H

4 B

E S

2008 NRC EXAM Q4 NRC: Include the applicable E/APE with the Generic K/A.

Response: Added EPE to KA designator.

19 H

3 M

S 20 F

3 N

S 21 H

3 N

E S

NRC: Change to make all answer options use actual values instead of by half and doubled to maintain the same style (ref.

ES-4.2 D.15.i).

When changing option D to comply with the above comment, dont change it to a value of air flow that makes the result the obvious choice.

Include the applicable E/APE with the Generic K/A.

Response: Changed answers A and B to include numerical values. Added APE to KA designator.

22 F

3 B

S 23 F

3 B

S 24 H

4 M

S

Page 5 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 25 H

4 N

E S

NRC: Please explain why the SG PORV would not be able to provide over-pressure protection in the case of SG overfill.

Distractors A, B, and D are all conditions where rising SG pressure is mitigated by the SG PORV why is that not the case for answer C?

Response: The SG PORV would normally provide overpressure protection from a SG overfill event but to meet entry conditions for 4023-FR-H-2, SG pressure is above the highest SG safety valve setpoint. SG overfill is the only potential cause this procedure lists of the answers choices given.

26 H

3 N

E S

NRC: Please explain why it is plausible to say Tcold indication is not susceptible to single loop effects.

Response: Distractor is plausible as part 2 describes the correct basis for temperature indication selection.

27 H

3 M

S 2008 NRC EXAM Q6 28 H

3 B

S 29 F

4 N

S 30 H

3 X

B U

S FREE SAMPLE NRC: The question is not a KA match. Specifically, there is not a valid operational lineup where the containment spray system is in a piggyback lineup from RH. (I believe this is a Combustion Engineering lineup). Recommendation is to change the KA to K4.08 which tests for interlocks associated with a piggyback lineup with high pressure injection.

The question pedigree should specifically indicate if this is a bank question and if it was previously used on an NRC exam.

Reference provided (excerpt of ES-1.3) does not provide specific evidence of the technical information to assess the accuracy of the answer and distractors. A lesson plan/training notes would have been more appropriate to determine what system interlocks are being tested.

Response: New bank question submitted that is associated with K4.08.

31 H

3 B

S 32 H

3 B

S

Page 6 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 33 H

3 B

S Prev 2: 2020 NRC EXAM Q34 34 H

3 B

S 35 H

3 N

E S

NRC: The containment pressure control system is not the associated system selected for the K/A. Is it possible to revise (or add to) the question to impact the environment of the pressurizer pressure control system?

Include the applicable system with the Generic K/A.

Response: Added System designator to KA. Replaced question based on new KA provided.

36 F

3 B

S 37 H

4 N

S 38 H

3 M

S 39 F

2 N

S 40 H

3 N

S 41 H

3 B

S 2008 NRC EXAM Q30 42 H

4 B

S 43 H

4 X

N U

S NRC: Distractor A - raising speed = lower discharge-P is not plausible (fundamental knowledge). Distractors B and D - raising power = lowering RFP speed is not plausible.

Response: Replaced KA and replaced question to match new KA 44 H

3 M

S 45 F

2 B

S NRC: Include the applicable system with the Generic K/A.

Response: Added System designator to KA 46 F

2 B

S 47 F

3 N

S

Page 7 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 48 F

3 X

N E

S NRC: Distractor D appears that it should state to separate the electrolyte into hydrogen and sulfate and not hydrogen and sulfuric acid. Since the distractor analysis indicates this is the process which occurs during battery discharge, it should be the sulfuric acid becoming hydrogen and sulfate per BE101 Slide 110.

Response: Changed distractor D to use sulfate vice sulfuric Acid 49 H

2 B

S 50 H

2 N

S 51 H

3 N

E S

NRC: Technical reference referred to in the question pedigree should be 2-OHP-4024-204, not 205.

Response: Corrected Technical reference designator 52 F

4 N

S 53 F

3 B

S 54 F

2 B

S 55 F

3 B

S 56 F

3 N

S NRC: This question test memory of which ventilation systems contain charcoal filter beds and therefore is testing fundamental knowledge (not higher cognitive knowledge).

Response: Designated as fundamental knowledge.

57 H

3 B

S 58 H

3 B

E S

NRC: Distractor B is missing the unit number (2) in front DRV-350 valve number.

Response: Distractor B contains Unit designator 2 for DRV-350 59 H

3 B

S 60 H

3 X

N E

S NRC: Grammatical error in question stem. Missing the word, be between will and restored.

Response: Stem corrected by adding the word be in correct location per above

Page 8 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 61 H

2 N

S 62 H

3 B

S 63 F

2 N

S 64 F

4 X

N E

S NRC: Since the answer is a negative response which is acceptable it would seem to be more appropriate to select a choice which is specifically not appropriate as an ODM as listed by procedure step 3.3.1 to avoid applicant contentions since the answer is not specifically addressed in the procedure.

Response: Changed answer to be an item listed in PMP-4010-ODM-001 section 3.3.1 65 F

3 B

E S

NRC: Spell out IPTE in the question stem. Also recommend adding IAW procedure PMI-4090 to the stem.

Response: Spelled out IPTE and added IAW procedure to stem 66 F

3 M

S Prev 2: 2020 NRC EXAM Q69 67 F

4 X

N E

S NRC: Distractor B incorrectly defines bi-weekly as twice < every seven days. This would be semi-weekly. Bi-weekly is over a two week period. Correct the definition to semi-weekly or adjust the time to two weeks if bi-weekly is more plausible based on the frequency of surveillances at Cook.

Response: Left distractor B as bi-weekly but changed it to once every 14 days.

68 H

4 X

B E

S 2008 NRC EXAM Q97 NRC: Distractor B is a subset of Distractor C. If C was the correct answer, then B would also be correct which means that there would be 2 correct answers. Therefore, an attentive applicant could eliminate these two distractors.

Response: Changed Distractor B to discuss Integrated (total)

Dose and remove potential subset of Distractor C 69 F

2 B

S 2012 NRC EXAM Q74

Page 9 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 70 H

2 X

X B

E S

FREE SAMPLE NRC: Tier 4 GFE questions should be operationally valid to the facility. The question stem presently says, A reactor is this should be, Unit 1 is operating at.

To more closely test the KA to explain the methods AND reasons perhaps just focus on control rod movement for part 1 and part 2 to test the reason for having to insert control rods to maintain RCS temperature, be to counter iodine decay and Xenon neutron absorption as the answer options.

For example:

over the next two hours, the operator must _______ control rods in order to counter the effects of ____________.

A.

Insert; Xe neutron absorption B.

Insert; Iodine radioactive decay C.

Withdraw; Xe neutron absorption D.

Withdraw; Iodine radioactive decay Response: Changes recommended above were made to the question stem and answer choices.

NRC: Answer and Distractor analysis incorrectly address the answer choices for question 71.

Response: Replaced Answer and Distractor analysis to align with this question 71 F

4 B

S 72 H

3 B

S 73 H

2 B

S 74 H

3 X

B E

S FREE SAMPLE NRC: Tier 4 GFE questions should be operationally valid to the facility. The question stem presently says, A plant is this should be, Unit 1 is operating at.

Response: Adjusted question stem using Unit 1 nomenclature.

75 F

3 B

S FREE SAMPLE

Page 10 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 76 H

3 M

E S

FREE SAMPLE BYRON 2010 NRC EXAM Q91 NRC: The reference provided does not provide sufficient detail to corroborate the answer/distractor analysis. Provide reference material, including lesson plans, which talk about what operators would expect to see from a reference leg leak and how they would be expected to make their way to the abnormal operating procedure for Excessive RCS Leakage. The reference procedure provided does not direct any action which can be used to indicate how the operator is supposed to act if the abnormal level instrument indication is a result of a leak on the level instrument.

Reference leg leak scenario is sufficient to meet the KA for a vapor space accident and the selection of a procedure pathway meets the requirements for 55.43(b)(5) for the SRO exam.

Response: Reference procedures provided and system description for pressurizer system and Sensors and Detectors.

77 F

F 1

2 X

X M

N U

E S

FREE SAMPLE NRC: Distractors C and D are not plausible: What are the indications of pump runout? Is there a lesson plan identifying what the indications of pump runout are?

Identifying cavitation is fundamental/basic knowledge and identifying which procedure to enter is not significant enough to warrant an SRO-only designation.

Response: Removed Distractors associated with runout and introduced a cycling RHR relief valve. Also added required actions in addition to procedure route determination to make SRO only.

NRC: Capitalize Pressure in the question stem ( Pressure Indicator ) as it is a part of plant equipment name.

This question tests memory of pump cavitation indications and procedure selection and therefore is testing fundamental knowledge (not higher cognitive knowledge). Revise the question pedigree to indicate fundamental knowledge.

Response: Capitalized Pressure and designated as fundamental knowledge.

Page 11 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 78 H

2 X

X N

U S

NRC: The applicant is provided the verbiage of E-1, Step 9 in the stem which includes the direction to assess RCS PRESSURE-STABLE OR LOWERING. We also tell the applicant the RCS pressure is 1500 psig and rising. Therefore, they would have to select the ONLY answer which indicates perform the RNO step for step 9. Distractor D is not plausible unless it included a plausible RNO action or assessment which was not able to be performed or was completed prior to moving on to step 10.

Response: Replaced Distractor D with a different procedure transition that is plausible based on plant conditions (RCS Pressure rising). Removed words perform Response Not Obtained from correct answer 79 H

4 N

S 80 H

3 N

S 81 H

4 N

E S

NRC: In the question pedigree section it implies that the difficulty level for this question is a 5. That would not be permissible by NUREG 1021 Rev 12. This question has been assessed a 4.

Questions may only have an LOD of 2-4. Update pedigree page.

Response: Changed LOD to 4 82 H

3 B

S REFERENCES PROVIDED: Attached to exam are Figure 13.1: AFD and ARM, TS 3.1.4. and TS 3.1.7.

83 H

4 N

E S

NRC: In the question pedigree section it implies that the difficulty level for this question is a 5. That would not be permissible by NUREG 1021 Rev 12. This question has been assessed a 4.

Questions may only have an LOD of 2-4. Update pedigree page.

Response: Changed LOD to 4 84 H

2 M

S

Page 12 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 85 H

2 X

N U

S NRC: The note in procedure 2-OHP-4022-CRE-001 specifically states Unit 2 Reactor and Turbine should be TRIPPED before evacuation. But also states that Steps 3 through 7 should be completed before control room evacuation IF conditions allow.

The stem by emphasizing immediately would imply conditions do not allow. Step 3 states - CHECK Main Turbine TRIPPED. The correct answer A specifically includes check turbine is tripped which would be step 3. The procedure note appears that it needs to be corrected to address these potentially conflicting statements and as such leaves this question highly vulnerable to being challenged/contested.

Response: Replaced question 86 H

2 N

S REFRENCES PROVIDED: PMP-2080-EPP-101 Rev 28 Figures 1 and 2 87 H

3 N

S 88 H

4 N

E S

NRC: In the question pedigree section it implies that the difficulty level for this question is a 5. That would not be permissible by NUREG 1021 Rev 12. This question has been assessed a 4.

Questions may only have an LOD of 2-4. Update pedigree page.

Response: Changed LOD to 4 89 H

3 X

N E

S U

NRC: To more adequately address section b) of the KA (what procedures would be utilized to mitigate), the questions should ask what LCO(s) if any are required to be entered and then the answer choices can be listed as LCO 3.8.4, Condition D Train N DC electrical power subsystem inoperable and LCO 3.7.5, Condition B One AFW train inoperable in MODE 1, 2, 3 Response: Modified question to ask Tech Specs to be entered.

Made question Open reference with TDAFP and N Train Tech Specs to be provided (TS 3.7.5 & 3.8.4)

NRC: After exam administration, this question was evaluated as having no correct answer. Therefore, the question is considered UNSAT 90 H

2 M

S 2018 NRC EXAM Q15

Page 13 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 91 H

4 X

N U

S NRC: Did the East FW pump trip on low suction pressure due to the trip of the running condensate booster pumps? Or was it an unrelated coincident event? If unrelated, this seems confusing for the applicant. If the FW pump tripped on low suction pressure, it would require pressure to be < 180 psig for > 5 seconds. Since this would occur at a lower pressure than the start of the standby condensate booster pump and the FW heater bypass valve opening, were those mitigating measures ineffective? Would they not be expected to prevent a trip of the FW pump in this situation?

Please confirm that there are no water-tight barriers between the condensate booster pumps and the condenser pit. At previous plants I have worked at a leak at the discharge of the condensate booster pumps would be prevented from causing a high condenser pit flood level alarm by water-tight doors.

Response: MFP tripping occurred due to pressure switch setpoint being low. Confirmed there are no water tight barriers.

Condenser pit is open to the elevations above.

92 H

4 X

N E

S REFERENCES PROVIDED: 12-OHP-4021-006-004, 12-OHP-4021-006-004 Data Sheet 1 NRC: It is not clear what value the provided reference serves.

The question at its heart is asking how as a supervisor do you handle permitting a release when conditions have changed which impact what has already been approved by other authorities. Perhaps where the release rates are determined could be also provided. This question in its present form has the hallmarks of a Tier 3 question. Please provide additional explanation of what role the provided reference serves.

Response: Required flowrates are listed in a table in step 4.8.1 of attachment 3. Candidate must recognize flow with the Circulating Water pump trip is below the minimum dilution flowrate of the permit.

93 F

3 B

S 94 H

3 B

S REFERENCES PROVIDED: OHI-4000 Att. 22 Shift Staffing 95 F

3 N

E S

NRC: In the question pedigree section it implies that the difficulty level for this question is a 5. That would not be permissible by NUREG 1021 Rev 12. This question has been assessed a 3.

Questions may only have an LOD of 2-4. Update pedigree page.

Response: Changed LOD to 4

Page 14 of 15 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist. Partial Job Link Minutia #/Units Logic Q-K/A License Level 96 F

4 N

E S

REFERENCES PROVIDED: 12-OHP-2110-CPS-001, Tagging Methods and Special Considerations NRC: In the question pedigree section it implies that the difficulty level for this question is a 5. That would not be permissible by NUREG 1021 Rev 12. This question has been assessed a 4.

Questions may only have an LOD of 2-4. Update pedigree page.

This question is a fundamental knowledge question as it only requires the applicant to identify information in the reference provided with minimal analysis.

Response: Designated as fundamental knowledge with LOD of 4 97 F

2 B

E S

REFERENCES PROVIDED: PMP-2291-TRS-001, Troubleshooting - Pages 17-19 NRC: This question is a fundamental knowledge question as it only requires the applicant to identify information in the reference provided with minimal analysis.

Response: Designated as fundamental knowledge 98 F

3 X

N E

S REFERENCES PROVIDED: PMP-4043-VLU-001 Valve Lineups and Position Control pages 11-14 NRC: This question is a fundamental knowledge question as it only requires the applicant to identify information in the reference provided with minimal analysis.

Distractor C is not plausible. There is no mention of the ALARA Committee in the reference provided and more importantly it is not plausible that Operations would go to RP to get approval for an operational decision (when it is appropriate to not check the condition of a valve when performing a valve lineup).

Response: Designated as fundamental knowledge. Replaced distractor C with plausible option in the procedure VLU-001.

99 H

3 N

S 100 H

4 B

E S

FREE SAMPLE NRC: In the question pedigree section it implies that the difficulty level for this question is a 5. That would not be permissible by NUREG 1021 Rev 12. This question has been assessed a 4.

Questions may only have an LOD of 2-4. Update pedigree page.

Response: Changed LOD to 4

Page 15 of 15 Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1) Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2) Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3) Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4) Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5) Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6) Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7) Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8) Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9) Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.