ML21020A093

11 Draft Written Examination Comments

ML21020A093
Person / Time
Site:	Palo Verde
Issue date:	12/10/2020
From:	Greg Werner Operations Branch IV
To:	Arizona Public Service Co
References
Download: ML21020A093 (10)

Text

ES401 PV202011 Written Examination Review Worksheet Form ES4019 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

a. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

b. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

c. The answer choices are a collection of unrelated true/false statements.

d. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

e. One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

a. The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

b. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

c. The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

d. The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

Place the answer letter here Place the 55.41 or 55.43 item here

8. At a minimum, explain any Unsat ratings (e.g., how the Appendix B psychometric attributes are not being met).

Ans

1. LOK 2. LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6 7 CFR 8 Letter stem cred job #/ back- SRO- A/B/C/ 55.41x55 Q# (F/H) (1-5) cueing T/F partial minutia K/A B/M/N U/E/S Explanation focus dist link units ward only D .43y Appears to be too many "B" answers on the RO (26). There are also not enough "C" answers in the SRO portion (3).

Need to discuss why only one phase light causing a reactor trip plausible (A and C). In the entire 1 H 3 N S B 41.7 plant it seems like you need more than one of anything to generate a trip signal. Discussed/Q considered SAT Subcooling and superheat are two different things. Therefore, need to get subcooled out of the 2 H 2 X X N E C 41.14 stem since it doesn't match C or D. Discuss modifying distractors so all four are superheat.

Revised/Q now SAT a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 As written, all four distractors are correct as they are all below 73%

3 H 3 X N U A 41.10 RWT level. Add "a MAXIMUM of" at end of stem.Revised/Q now SAT ON 2016 NRC EXAM. Change 4 F 2 X B E D 41.10 "will's" to "should's" in stem.

Revised/Q now SAT Need LOD. Change "will" to "should" in stem. First part of distractors could be confusing if 5 H 3 X X N E A 41.3 applicant thnks "lower" and "higher" refer to pitch. Better to use "more noise" and "less noise" than normal. Revised/Q now SAT 6 H 3 N S C 41.7 In B and D, change "event of accident" to "event of a design 7 F 3 X N E A 41.7 basis accident."Revised/Q now SAT Change "will's" to "should's" in stem. Appears to be the same 8 H 3 X N U D 41.14 thing as Event 2 in scenario 3 (replace) Replaced/Q now SAT 9 F 2 N S D 41.7 Change "will" to "should" in stem.

1135 psia seems high. What is 10 H 2 X X N E A 41.5 normal operating SG pressure?

Discuss how K/A is met.Discussed/Q now SAT Change "will" to "should" in stem.

Don't quite understand explanations (opening aux spray 11 H 3 X N E A 41.5 stops a depressurization?).

Please explain.Explanations revised/Q now SAT ON 2016 NRC EXAM. Change 12 H 3 X B E A 41.7 "will's" to "should's" in stem.

Revised/Q now SAT a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 Don't see any real connection between instrument air and MSIS.

13 H 2 X N U B 41.4 Also, (1) and (3) both appear to be obvious results of an MSIS.Revised distractors to make more plausible. Q now SAT Confidence level not high that anyone would select CEAC over 14 H 3 N S A 41.4 COLSS (discuss).Discussed/Q considered SAT 15 H 3 N S A 41.8 Change "will's" to "should's" in 16 F 2 X N E D 41.4 stem. Revised/Q now SAT Change "will" to "should" in stem.

17 H 3 X N E D 41.4 Revised/Q now SAT Change "will's" to "should's" in 18 H 3 X N E A 41.5 stem.Revised/Q now SAT REFERENCE PROVIDED.

Change "will" to "should" in stem.

Knowledge that a periphery control rod would have less of an effect on actual reactivity than 19 H 3 X X N U A 41.6 one near the center of the core doesn't appear to have much discriminating value (distractors C and D not plausible).Replaced/Q now SAT Change "will" to "should" in stem.

Doesn't seem likely that anyone would think pusle counters would be used over Reed switch indication. Discuss possible 20 H 3 X X N E A 41.6 enhancement alternatives.

Explanations enhanced to explain better while pulse counters are plausible./Q now SAT Change "will's" to "should's" in 21 F 3 X N E B 41.11 stem. Revised/Q now SAT Please discuss explanations. RU-8 is mentioned in the explanation 22 F 2 N E B 41.11 for D even though it isn't part of the answer?Revised/Q now SAT Change "will's" to "should's" in 23 F 2 X N E B 41.8 stem.Revised/Q now SAT a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 Change "will's" to "should's" in stem. Please discuss 24 H 3 X N E D 41.4 explanations. B and D modified "isolate" to "restrict." /Q now SAT ON 2016 NRC EXAM. Student need only know the location and nothing else to correctly answer question. Not plausible that a 25 F 2 X B U D 41.11 contaiment area monitor would be detecting RCS activity better than an RCS monitor with no LOCA (distractors A and B).

Restructured/ Q now SAT 26 F 2 N S A 41.14 REFERENCE PROVIDED. Why is a reference really necessary here? Change "after the SG 27 H 3 X X N E B 41.10 blowing down and then isolated" to "after SG#1 has been isolated."Revised/Q now SAT Change "will" and "would" to 28 H 3 N E D 41.3 "should" in stem. Revised/Q now SAT OK, but distractor A is essentially 29 H 3 B S C 41.3 a "None of the Above" type distractor and should be avoided.

Change "will" to "should" in stem.

30 H 3 X B E D 41.3 Revised/Q now SAT 31 H 3 N S B 41.7 REFERENCE PROVIDED.

ON 2018 NRC EXAM. Change 32 H 3 X B E D 41.7 "will's" to "should's" in distractors.

Revised/Q now SAT Change "will's" to "should's" in stem. Possible two correct answers. It is not a "required" action to manually actuate CSAS on trend (should not shall), so if 33 F 2 X X N U C 41.7 the crew lets it auto actuate, that would be less preferred, but apparently not "wrong." Revised stem to clarify per procedural requirements./Q now SAT a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 Need period after "open" in stem.

34 H 3 X N E B 41.5 Change "will" to "should" in stem.

Revised/Q now SAT ON 2019 NRC EXAM. Change 35 H 3 X B E A 4.11 "will" to "should" in stem.Revised/Q now SAT 36 H 3 N S C 41.7 37 H 3 N S C 41.5 38 F 2 N S A 41.8 Change "will's" to "should's" in stem. Change "Channel A" to "Channel C" in stem and change 39 H 3 X X N E C 41.7 correct answer to "C" (more discriminating).Revised/Q now SAT 40 H 2 N S C 41.7 41 F 2 N S A 41.4 Explanation says a loss of PGA-L35 or PHA-M35 will cause a loss 42 F 2 X N U C 41.8 of the valve. Two correct answers (A and B)?Restructured/Q now SAT Question indicates it is modified 43 H 3 B E B 41.7 bank, but no original question is attached. Bank Q ON 2018 NRC EXAM. Change 44 H 3 X B E A 41.14 "will" to "should" in stemRevised/Q now SAT Change "will's" to "should's" in 45 H 3 X N E B 41.4 stem. Revised/Q now SAT C doesn't appear plausible. Why would someone presume that an 46 H 2 N S D 41.4 AFW pump would stay running to an isolated SG?Discussed/Q considered SAT Should the minimum temperature 47 F 2 X N E B 41.8 of 60° be in the stem?Revised/Q now SAT ON 2016 NRC EXAM. Change "will" to "should" in stem. Discuss 48 H 3 X B E D 41.7 modifications from original question.Bank Q/Revised/Q now SAT 49 H 3 N S B 41.7 a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 Change "will's" to "should's" in 50 H 3 X X N E A 41.11 stem and distractors.Revised/Q now SAT 51 F 3 M S D 41.7 52 H 3 B S C 41.4 ON 2016 NRC EXAM.

53 F 3 B S D 41.4 ON 2019 NRC EXAM.

54 F 2 N S A 41.4 B(2) and D(2) do not appear plausible when doing A(2) or C(2) 55 F 3 X N U A 41.9 would stop any potential leak.Restructured/Q now SAT 56 H 3 N S C 41.6 ON 2016 NRC EXAM. Seems far fetched that someone would think 57 F 2 B S B 41.14 delta T >65° in natural circulation (discuss)Discussed/Q considered SAT 58 H 3 N S B 41.6 59 F 2 N S C 41.2 60 F 3 B S D 41.13 ON 2016 NRC EXAM.

Discuss plausibility of hyrdogen recombiners being powered with 61 F 2 X N E B 41.8 4160 volts (distractors C and D).Discussed/Q considered SAT Provide power level in stem.

Discuss possible alternatives to 62 H 2 X X N E A 41.4 steam generator low level distractors SG level distractors replaced/Q now SAT Change "will" to "should" in stem.

Need to modify (1) in stem to 63 H 3 X N E D 41.1 specify SBCV's other than 1001 and 1004.Revised/Q now SAT 64 F 3 N S C 41.9 beta not very believable - maybe neutron instead? Also, B(2) and D(2) are essentially "none of the 65 F 2 X N E D 41.11 above" type distractors and should be avoided.Revised/Q now SAT a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 OK, but A(2) and C(2) are essentially "none of the above" 66 H 3 N S B 41.10 type distractors and should be avoided.

Change "will" to "should" in stem.

67 F 3 X N E B 41.10 Revised/Q now SAT ON 2016 NRC EXAM. Listed as 68 F 2 B E D 41.10 new question? Bank Q No correct answer as written.

Reference indicates double valve 69 F 2 X X N U B 41.10 isloation required for GREATER than 200 degrees or 500 psig.

Restructure Revised/Q now SAT Change "will" to "should" in stem.

A and B not credible, as "any SM" too ambiguous (any qualified SM?) + if "any SM" was correct, 70 F 3 X X N U C 41.10 then the Unit 1 SM would also be correct. Also, need to discuss plausibility of "Salt River Project."

Restructured/Q now SAT 71 F 3 N S C 41.9 ON 2016 NRC EXAM. Does not appear to be a modified bank 72 F 2 B E B 41.12 question (looks like same question, but the answer has changed?) Bank Q 73 F 2 N S D 41.12 74 F 3 N S C 41.10 It appears from the reference that the FTA needs to be both a qualified RO and Aux Operator? If 75 F 3 X N E C 41.10 so, then there could be two correct answers (C and D).

Discuss to make sure D is wrong.Revised/Q now SAT 76 F 3 N S D 43.4 77 F 3 N S B 43.2 78 F 3 N S B 43.2 a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 "2" is obviuosly correct (feed break/SG low level), which means "1" cannot be correct with the question as written. This 79 H 3 X N U B 43.2 automatically makes only two distractors possible (B and D)/(2) changed to "ctmnt press high"/Revised; Q now SAT Change "will" to "should" in stem.

Can't think of any circumstance where an SBO diesel is preferred 80 H 3 X N U C 43.5 over an available (normal) EDG.

C and D do not appear plausible.Replaced; Q now SAT Change "will" to "should" in stem.

81 H 3 X N E D 43.5 D explanation should be in C block./Revised; Q now SAT Distractors A and C don't appear very plausible with both boric acid makeup pumps OOS. Any 82 H 3 N S D 43.5 alternative?/Q determined OK based on procedural direction; Q SAT 83 H 3 N S B 43.2 ON 2016 NRC EXAM.

84 F 3 B E D 43.2 Explanation for C appears incomplete./Revised; Q now SAT Release exceeding federally approved limits does not appear plausible given there are zero indications of any abnormal 85 H 3 X N U D 43.4 radiation in stem (distractors A and C)./Add "design basis SGTR event" in stem/Revised; Q now SAT Change "will" to "should" in stem.

86 H 3 X N E A 43.5 Why is reference provided box X'd?/Typo/Revised; Q now SAT ON 2018 NRC EXAM. Distractors B and D suggest going to an 87 F 3 X X B E C 43.2 alarm response procedure, but there is no alarm in the stem (insert one)./Alarm inserted in stem/Revised; Q now SAT a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 REFERENCE PROVIDED (need to note in question). Change "will" 88 H 3 X N E A 43.5 to "should" in stem. /Revised; Q now SAT Fire main seems far fetched (distrators C and D). Discuss 89 H 3 X N U B 43.5 other potential sources (HPSI?)/Changed to HPSI/Revised; Q now SAT Change "will's" to "should's" in stem. If the CRS opted to energize PBB-S04, why would 90 H 3 X X N U B 43.5 that be "wrong?" (possible two correct answers).Add procedure in stem/Revised; Q now SAT 91 H 3 N S A 43.2 92 H 3 N S A 43.2 93 F 3 N S A 43.2 94 H 3 N S C 43.5 REFERENCE PROVIDED ON 2016 NRC EXAM. Question says modified bank. Don't see 95 F 2 B S C 43.7 any substantial modification./Remains a Bank Q Discuss. C and D did not appear very plausible. Why would you ever use an off watch SRO as the 96 F 3 X N U C 43.5 reactivity manager if the on shift licensed STA was allowed to?/B and D changed to Reg Gp.

1/Revised; Q now SAT 97 F 3 B S B 43.2 ON 2016 NRC EXAM 98 F 3 N S C 43.4 99 F 2 N S A 43.5 100 H 3 N S D 43.5 Results Table RO LOK -H 42 Avg RO LOD 2.64 Flaws 10 CFR Distribution RO LOK-F 33 AVG SRO LOD 2.92 Stem focus 38 41.1 11 43.1 0 SRO LOK - H 14 Overall LOD 2.71 Cues 1 41.2 1 43.2 10 SRO LOK - F 11 T/F 0 41.3 4 43.3 0

%  % Cred Dist 20 41.4 12 43.4 3 a Rev 4

ES401 PV202011 Written Examination Review Worksheet Form ES4019 RO Bank 16 21.33 SRO Bank 4 16 Partial 5 41.5 5 43.5 11 RO Mod 1 1.333 SRO Mod 0 0 job link 0 41.6 4 43.6 0 RO New 58 77.33 SRO New 21 84 units 0 41.7 15 43.7 1

% minutia 0 41.8 6 Total Bank 20 20 backward 0 41.9 3 Total Mod 1 1 KA 1 41.10 11 Total New 79 79 SRO-only 0 41.11 5

%  % LOD = 1 0 41.12 2 RO Sat 29 38.67 SRO Sat 14 56 41.13 1 RO Unsat 10 13.33 SRO Unsat 6 24 Answer Dist ( in %) 41.14 5 RO Edit 36 48 SRO Edit 5 20 RO-A 20 SRO-A 6

%  % RO-B 19 SRO-B 7 Total Sat 43 43 Total Unsat 16 16 RO-C 17 SRO-C 6 Total Edit 41 41 RO-D 19 SRO-D 6 a Rev 4