ML22019A146

11 Draft Written Examination Comments

ML22019A146
Person / Time
Site:	Wolf Creek
Issue date:	12/07/2021
From:	Heather Gepford Operations Branch IV
To:	Wolf Creek
References
Download: ML22019A146 (8)

Text

ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

a. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

b. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

c. The answer choices are a collection of unrelated true/false statements.

d. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

e. One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

a. The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

b. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

c. The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

d. The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

Place the answer letter here Place the 55.41 or 55.43 item here

8. At a minimum, explain any Unsat ratings (e.g., how the Appendix B psychometric attributes are not being met).

1. LOK 2. LOD 6 73. Psychometric Flaws 4. Job Content Flaws 5. OtherAns CFR 8 Letter Q# (F/H) (1-5)stem cueing T/F cred partial job link minutia# / back-K/A SRO-B/M/N U/E/S A/B/C/ 55.41x55 Explanation focus dist units war d only D.43y

480 v power supply is not a credible distractor. I recommend changing the answers to a 2x2 1 F 2 X N U A 41.6and asking what bus the breaker is on and the normal supply for DC control power. Changed distractors and now SAT 2 H 3 N S C 41.2 3 F 2 N S C 41.6 Closing an inlet valve to a heat 4 F 2 X N U B 41.5exchanger to lower temperature of RHR is not credible.

Discussed Q and now SAT 5 F 2 B S A 41.3 6 F 2 N S D 41.8 7 H 3 N S C 41.9 8 H 2 N S A 41.1

a Rev 2 ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

9 H 3 N S C 41.7

• Change the stem to ask what 10 H 2 x B E C 41.5parameter will trip the plant first.

Revised stem and now SAT 11 H 3 N S D 41.7 12 H 3 M S A 41.7 13 H 2 x N U A 41.7B is not credible. Discussed the questions and now SAT 14 F 2 N S C 41.9 15 H 2 x N U B 41.9a is not credible. Question rewritten and now SAT.

The distractors are a combination 16 F 2 x x N U B 41.5of T/F statements. Question rewritten and now SAT Stem information needs more 17 H 2 x N E D 41.10info to make A and B credible distractors. Added information to the stem and now SAT 18 F 2 N S A 41.7 RW ST and CCW are not credible 19 F 2 x N U C 41.4distractors. Distractors changed and now SAT A is not a credible distractor. It is expected that ALL AFW throttle 20 H 2 X N U D 41.7valves open when AFW actuates.

A distractor analysis is incorrect.

OK as is after more discussion now SAT The question stem can be shortened. First part of 2x2 would say "The diesel generator must be rotating in the _____

direction when the output breaker is closed." This could also be 21 H 2 x N E D 41.5changed to ask "The diesel output voltage must be higher than the ________ volts."

running or incoming would be the answer choices. Rewrite will be provided. Rewritten and is now SAT 22 F 2 N S B 41.7 23 H 2 N S B 41.5

a Rev 2 ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

change the question stem. W hat 24 H 3 x N E D 41.11HX is the source of the leak?

The question was fixed and now SAT 25 F 2 N S A 41.7 26 F 2 N S A 41.4 A and D are not credible 27 F 2 x B U C 41.7distractors. The switch does not exist. Rewritten and now SAT 28 H 2 B S A 41.9 29 H 2 x N E C 41.2get rid of the word "will".

Corrected and now SAT W hy is distractor for B and D 2nd part wrong? If the applicant uses adverse CTMT parameters, what 30 H 3 x N E C 41.5is the adverse consequence?

Clarified distractors, added procedure reference, now SAT

31 F 2 N S B 41.9 *clarify explanation A & B are not credible distractors.

LO1403300, page 32 of 66 has a list of power supplies that would 32 F 2 x N U D 41.7be good distractors. Replace NB01 and NB02 with NG03C and NG04C. Changed distractors and now SAT 33 H 2 X N E A 41.7Add the word "maximum" to part

1. Corrected and now SAT 34 F 2 N S D 41.5 I want the 2nd part of the 2x2 edited to ask for the minimum 35 H 2 x x N E B 41.4steam pressure of sealing steam.

Distractors changed and now SAT Is the "protect main condenser from over pressure" a valid distractor? If it is, we should add 36 H 2 x N E A 41.4that there was a partial loss of the circ water pumps. Additional information was added to the stem and distractors are now plausible

a Rev 2 ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

D is not credible. W ould Porv or PZR safety leak by go to the 37 F 2 x N U B 41.7RCDT? IF so, add to the stem that the unidentified leakage is 0 gpm. Reworked to have "PRT" source replace D and now SAT 38 F 2 N S C 41.4 Change SCM to 45 F and add to 39 F 2 x N E A 41.10the stem that CTMT temperature is a value slightly higher than normal. Corrected and now SAT 40 F 2 x N E A 41.2W hy is B plausible? Discussed and now SAT

verify not in scenarios and where 41 H 3 N S C 41.5is reference for part B. Overlap is not an issue because the written will be taken first. Now SAT 42 H 3 N S A 41.10 A and D are correct choices. The fly wheel helps maintain RCS flow to avoid DNBR. The anti-43 F 2 x x N U A 41.7rotation devices prevent flow from being diverted through the stagnant loop to avoid DNBR.

W e need better distractors for B and C. Rewritten SAT eliminate the underlines in 44 F 2 x N E C 41.5answers or add underlines to first two distractors. Underlines removed and now SAT I want to edit the distractors to be 45 F 2 x N E D 41.8the alarm setpoints for the given parameters. Changed distractors and now SAT 46 F 3 M S B 41.7modified question from 2019 NRC exam underlines. C & D are same 47 H 3 x N U C 41.10action but different reason Rewritten now SAT.

• all parameters stable? W hat 48 F 2 x N E C 41.10about RCS tave? Borderline LOD 1. rewritten and now SAT

• change direct to immeadiate 49 F 2 x N E B 41.7discuss. Discussed and now SAT

a Rev 2 ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

50 F 2 x N E D 41.7add "designated" before rate.

Edited and now SAT can we cange this to a 2x2 51 H 3 x N E B 41.8question? W hy is a credible?

Rewritten SAT 52 H 3 B S D 41.5 53 H 3 N S C 41.5 RHR and SI are a subset of 54 F 2 x B E C 41.9ECCS. Is choice C a good distractor? Rewritten SAT

• "as necessary"? I want a rate 55 H 3 x N E A 41.10that is wrong. "feed at full flow" Correction made and now SAT 56 H 2 x B E B 41.10*change will to should.

Correction made and now SAT

• change will to should explain 57 H 2 x x N E C 41.1B&D credible with rising Tave?

Changed distractors and now SAT why is lowering turbine load a 58 H 3 x N E D 41.7credible distractor. Rewritten SAT is this not in scenario 2? W riten 59 H 2 N S D 41.11exam is being taken before the OP test. SAT 60 H 3 x N E B 41.10why are c&d credible? Rewrite SAT a is the same as "none of the 61 F 2 x N E D 41.13above" and not allowed.

Corrected and now SAT 62 F 2 M S C 41.14 used in 2011 NRC exam 63 H 3 N S B 41.8 C is not a plausible distractor.

64 F 2 X N E A 41.10Distractor C changed and now SAT 65 H 3 N S B 41.9 I would like to modify this to have a MTC of 0. I would then want to replace distractor B with CCW 66 H 3 x x B E D 41.1temperature lowers by 15 F due to a malfunction in the CCW system. Distractor D fixed to make it incorrect. B distractor changed and now SAT 67 H 3 N S C 41.5

a Rev 2 ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

68 F 2 x N U D 41.10cuing. D is in the question. A has nothing to do with oversight D is the same as "none of the 69 F 2 x N E B 41.10above". How can A be reperformed? Explain what A means. Rewritten and now SAT 70 H 3 N S D 41.2 71 F 2 N S D 41.11 72 F 2 B S C 41.12 73 H 2 N S A 41.10 add "required" in caps. D is an 74 F 2 x N U D 41.10"if/then" and this is usually correct. Corrected and now SAT

• discuss. Ensure that there are 75 F 2 x N E B 41.10not multiple "correct" answers.

Discussed and now SAT 76 H 2 N S C 43.2

• reference provided* is the 77 H 3 x N E C 43.2reference needed? Where is the reference. Reference removed 78 H 2 x N E D 43.5W hy is distractor B credible?

Discussed and now SAT discuss this. Distractor D looks 79 H 3 x N E D 43.2correct instantly. Discussed and now SAT distractors a and b don't have TS 80 H 3 x N U A 43.2actions and easy to eliminate. I want to change this to a 2x2.

Rewritten and now SAT discuss C. Do operators ever 81 F 2 x N E D 43.3review design requirements or is this a function of engineering?

Discussed and now SAT This is asking about normal procedures. W here is the link to 82 F 2 x B U B 43.6abnormal or emergency procedures? Changed the K/A and rewritten SAT most conservative choice is the correct answer. change the time from greater than 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> to 83 F 2 x N E D 43.4less than 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> but greater than 24. Changed the stem to make the distractors more plausible and now SAT

a Rev 2 ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

W hy would B ever be incorrect?

84 H 3 x x N U B 43.5You always transition to E-0 from an FR procedure. Discussed and now SAT add to the stem that the attachment is complete allowing 85 H 3 x B E A 43.5them to enter FR procedures.

Added information to the stem and now SAT why are distractors a,b,c 86 H 3 x N U B 43.5credible? Removed information from the stem and now SAT 87 H 3 N S A 43.2 eliminate the 2nd "take". Is there an actual "temporary diesel fire 88 H 3 x x N E B 43.5pump"? Does this meet K/A of determining the cause of loss of SW. Discussed and modified the stem. Now SAT discuss "licensing personel". Can we ask what the time limit for 89 F 2 x N E B 43.1making the written notification?

Changed the 2nd part of question to asking about 10CFR50.73 and now SAT 90 H 3 N S A 43.6 91 H 3 N S D 43.5 change "will" to should. Is B credible? W hen do you ever go 92 H 3 x x B E D 43.5from E-1 to E-2? Can we change B to go to FRZ-1 and add CET temp rising? Modifed distractor B and now SAT 93 H 3 B E Dwe need to add cfr to the table.

CFR added and now SAT 94 F 2 N S A 43.7 95 F 2 N S C 43.1

• when is the plant FIRST 96 H 3 x B E C 43.2required to be in mode 5.

Modified question and now SAT

a Rev 2 ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9

PV 2018 exam. I would like to modify this question. I want to change the 2nd part to ask what time the next surveillance is due 97 F 2 x x B E B 43.2and have in the stem that the last surveillance was completed 9.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> (1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> late) after the initial surveillance. Modified the question and now SAT 98 F 2 B S C 43.4 99 F 2 x N E A 43.4change the "to" to "into".

Corrected and now SAT Is C explanation correct?

100 H 3 B S A 43.5Distractor analysis corrected and now SAT

Results Table RO LOK -H 38 Avg RO LOD 2.29333 Flaws 10 CFR Distribution RO LOK-F 37 AVG SRO LOD 2.56 Stem focus 22 41.1 17 43.1 2 SRO LOK - H 16 Overall LOD 2.36 Cues 5 41.24 43.2 7 SRO LOK - F 9 T/F 1 41.3143.31

% % Cred Dist 35 41.4543.43 RO Bank 9 12 SRO Bank 8 32 Partial 2 41.5 12 43.5 8 RO Mod 3 4 SRO Mod 0 0 job link 0 41.6 2 43.6 2 RO New 63 84 SRO New 17 68 units 0 41.7 17 43.7 1

% minutia 0 41.84 Total Bank 17 17 backward 0 41.97 Total Mod 3 3 KA 2 41.1017 Total New 80 80 SRO-only 0 41.113

% % LOD = 1 0 41.12 1 RO Sat 33 44 SRO Sat 832 41.13 1 RO Unsat 14 18.67 SRO Unsat 416 Answ er Dist ( in %) 41.14 1 RO Edit 28 37.33 SRO Edit 1352RO-A 18 SRO-A 7

% % RO-B 17 SRO-B 6 Total Sat 41 41 Total Unsat 18 18 RO-C 21 SRO-C 5 Total Edit 41 41 RO-D 19 SRO-D 7

a Rev 2