ML23201A134

Form 2.3-5 Written Examination Review Worksheet

ML23201A134
Person / Time
Site:	Seabrook
Issue date:	07/18/2023
From:	Thomas Setzer NRC/RGN-I/DORS/OB
To:	NextEra Energy Seabrook
References
EPID L-2023-OLL-0043 2023302
Download: ML23201A134 (1)

Text

Form 2.3-5 Written Examination Review Worksheet

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 1

F 2

N S

2 H

2 B

E Added third bullet, No operator actions are taken.

3 F

3 N

S 4

H 3

B S

5 H

3 N

S 6*

F 3

N S

Overlap with Q16 - edits addressed in Q16 7

H 2

N E

Added question stem to increase operational validity.

8 F

2 B

S 9

H 3

N E

Changed is to are in A and B.

10 H

3 N

S 11 F

3 N

E Added per OS1000.07, Approach to Criticality 12 H

3 N

S 13*

F 3

N E

Rewrite as 2x2. Is the procedure selection (first part)

RO level?

Distractor A - does Fuel Handling Mode place them in operation? Why is this wrong? Not required by procedure.

Distractor C & D: why is entering OS1252.03 wrong?

Second part is wrong.

Reworded distractors to have AND and edited stem to include procedurally required actions.

14 H

2 B

E Added under all conditions to distractor D.

15 H

3 B

S 16 F

2 N

E Changed part (2) of question to avoid overlap with question 6.

Added from 100% power to first bullet.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 17 H

3 N

S Overlap with Q49 - edits addressed in Q49 18 F

2 B

S 19 H

3 B

S 20*

F 2

N E

Tier 4 - this may be too specific for Tier 4. Question doesnt really doesnt explain what local peaking factor is.

Added Seabrooks to first sentence in the question.

21*

F 2

X N

U Tier 4 - C is correct also. Do the commas in the distractors mark 2 separate parts in the answer where each parts corresponds with each other directly?

Rewrote question, needs edits - Added As seen on RE-1, Critical Boron Concentration vs Burnup to the beginning of the question stem.

22 F

2 B

S 23 F

2 N

S 24*

H 3

X B

U K/A mismatch - how does this question determine the source of the water in the sump? Redrew K/A 25 F

3 B

S 26 H

3 B

E Added fourth bullet, No operator actions are taken.

Added note stating: Simulator run validated that the Main Feed Pumps will not trip on over speed in response to FW-PT-508 failing low before the reactor trips on high SG level.

27 H

3 X

B U

B is subset of D. Added only to B. B does not contain transfer SEPS supply from Bus E6 to E5 which is contained in D.

Correct answer C is also a subset of D.

Added only to C. C does not contain manually start the SEPS diesels as the SEPS will automatically start. This statement is contained in D.

A, B and C use only to eliminate subset issues.

Statement to verify auto start of SEPS was not added to A and C because this action is not contained in ECA-0.0.

28 H

3 B

E Changed Which component(s) were running, if any?

to Which component(s) were running when the FBL returned, if any? to better specify what time is in question.

29 H

3 N

S 30 H

3 B

S 31*

F 2

X N

U D may be correct also. Why is D wrong where the crew will verify that bleed and feed is not required? D is correct also. Question will be reworded to ask what actions are required vice why the procedure requires actions to be done. Rewritten question is SAT 32 H

3 B

S

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 33 H

3 B

S 34 H

3 B

S 35 H

3 B

S 36*

H 1

X X

N U

Backwards logic for an A2 K/A. Test what actions (secure both charging pumps) instead of providing that information.

Isnt D correct? You wont lift a PORV? Question will be rewritten. Question does not meet K/A and all part 2s are correct.

37 F

3 B

S 38 H

3 B

S 39 H

3 B

S 40 H

3 B

E Added to 800 MWe to incorrect distractor A to better distinguish it from incorrect distractor D.

41 H

3 B

E Added if any to make the question stem read, Which of the following describes the effect if any, 42 F

3 N

E Changed Per the VPRO to In accordance with the VPRO.

In C and D removed statements verify that the pump trips automatically with the following changes:

o C changed trip the reactor and transition to E-0, Reactor Trip or safety Injection, verify that the pump trips automatically due to high vibrations to, trip the reactor and transition to E-0, Reactor Trip or safety Injection once the pump trips automatically due to high vibrations.

o D changed verify that the pump trips automatically due to high vibrations, then transition to OS1231.03, Turbine Runback/Setback to, transition to OS1231.03, Turbine Runback/Setback once the pump trips automatically due to high vibrations.

43 H

3 B

S

44 H

3 B

E Corrected typo, RSC to RCS.

45 H

3 B

S 46 H

3 B

E Added noun name to EDE-PP-1D, Vital 120 VAC Power Panel 1D.

47 H

3 B

E Removed statement the crew has reset the Safety Injection signal on both trains in the fifth bullet and replaced with the crew has turned both A and B train SI reset switches to RESET.

48 F

3 N

S

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 49*

H 3

N E

Add gpm to stem 3 is choices and 1 is not choice is awkward.

Rewrite to a better 2x2 where they determine if there is a leak and how big.

Changed PSO has reduced charging to seals only to CS-HCV-182, RCP Seal Flow Control Valve has been closed to avoid overlap with question 17.

50 H

3 B

E Added 100% power as first bullet.

Added Due to a small break LOCA in the fourth bullet.

51 F

3 N

S 52 H

3 B

S 53 H

3 B

S 54 F

3 N

55*

F 3

X B

N U

S Rewrite, backwards logic - 3rd bullet in stem is what we should be testing. Use current distractors as a part 2.

New question SAT 56 F

3 B

S 57 F

3 B

E Added reference to OX1460.01 Weekly Air Lock Door Seal Air Flow Rate to question stem. Reworded question for better readability.

58 H

3 B

S 59 H

3 N

S 60 F

3 N

E Added note to question pedigree documenting site specific OE where IA was lost to CS-HCV-182. *OE driven question.

61 H

3 N

S 62 F

3 B

E Added In accordance with the background document for ES-3.1 to beginning of question stem.

63 F

3 N

S

64 H

3 N

E Added element in plant conditions for loss of offsite power to eliminate direct look up concern and changed distractors accordingly.

Form 2.3-5 Written Examination Review Worksheet

• These questions were submitted as part of the presubmittal sample for preliminary review and comment. As such, they do not count towards the total of unsatisfactory questions since changes have been incorporated.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 65 F

2 N

S 66 H

3 B

S 67*

F 3

X B

U K/A mismatch - doesnt test electrical power supplies.

Reselected K/A.

Revised question SAT 68 H

3 B

S 69 F

2 N

S 70 H

3 B

S 71 F

2 B

S 72 H

2 B

E Added K/A justification.

73 H

3 B

S 74 F

3 N

S 75 H

3 B

S

Form 2.3-5 Written Examination Review Worksheet Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.

marked by an (S) on this form.