ML17080A186

2017-02 Draft Written Examination Comments

ML17080A186
Person / Time
Site:	Arkansas Nuclear
Issue date:	02/22/2017
From:	Vincent Gaddy Operations Branch IV
To:
References
Download: ML17080A186 (21)

Text

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation 1

H 3

X X

B E

S 2008 Bank Question Distractor B seems to be a subset of distractor D. (If one has D they would also have B.) This renders B implausible.

Rev. 1 Added EFW secured by DP to the distractor analysis for correct answer D. Enhanced distractor A analysis to help explain why SG level affect the characteristics of the line break. Changed distract D to the Main Feed Reg Valves. Question is now SAT 2

H 3

X N

U S

REFERENCES PROVIDED There is a hard return in the stem that should not be there, after "overheating of the" and before "Main Generator." This is a direct lookup question with the attached references, particularly Exhibit 2, which quickly eliminates distractors B and D.

Rev. 1 Corrected minor Edits. Deleted Exhibit 2 as a reference and lowered field amps to just above the nominal 100% value. Deleted extra generator parameters on voltage and current. Question is now SAT.

GENERAL COMMENT

S

1. Bulleted items in the question stem should not have periods at the end of them.

2. In stems, "will" and "would" should be "should",

3. Psychometric Flaws

4. Job Content Flaws

5. Other Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 3

H 4

X B

E E

S 2014 Bank Question Though the distractor analysis states that attachment 50 is in progress, it is impossible to know what step in the attachment is being performed, thus no guarantee that control power has been removed for the B and C pumps. In this case, distractor D could be incorrect if there was control power to the B pump, and distractor B could be correct.

Additionally, there is no need for the part 2 of the question, as part 1 each has unique answers. If the first concern is adequately addressed, then part 2 of the question should be eliminated.

Rev. 1 Reworded last bullet to state the attachment 50 has been completed up to othe step to start a condensate pump.

Rev. 2 Removed the second part of the stem and the second half of the distractors. Question is now SAT.

4 F

2 X

N E

E S

This appears to be SRO only knowledge, as knowledge of the Tech Spec Basis is required to answer the question. Why would anyone believe there is a loss of primary coolant on an excess steam demand event, which would make distractor A implausbile.

Rev. 1 Modified distractor A to state that a large amount of borated water is injected to overcome the rapid drop in pressurizer level. In 12/13/16 meeting with ANO, they stated that question was RO level knowledge.

Rev. 2 Removed the second part of the stem that referenced Hot Zero Power EOL core conditions and added " From 100% Power" to the beginning of the stem. Question is now SAT.

5 H

3 X

B E

S 2005 Bank Question Distractor D not plausible as it is the only distractor that requires no direct action by an operator, thus signaling that it is not correct.

Rev. 1 Added "proportional" to distractor B and removed the word "automatically" from distractor D to make it an action. Question is now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 6

F 2

X B

U E

S 2006 Bank Question Distractor A refers to high moisture environment, yet not mention of one in the stem (implausible distractor). If B were correct, it would make D correct (subset issue). Second part of distractor C not plausible as it is the only distractor that does not have a direct effect on the system, thus signaling that it is not correct.

Rev. 1 Changed to a 2x2 question, eliminating the second half of distractors A and C. Quesion is now EDIT.

Rev. 2 Changed the word "will" to "could" in the 2nd half of distractors A and B. Question is now SAT.

7 H

3 X

B U

E S

2008 Bank Question Why does distractor B say, "Immediately trip the reactor," and distractor D says, "Trip the reactor?"

All this question is really asking is if the reactor should be tripped on a loss of CCW. If an applicant chooses "No," then distractor C is cued as the correct answer because it is the only one of the two remaining that direct entry into a procedure.

Rev. 1 Removed the word "immmediately" from distractor B Reworded distractor D to direct entry into a procedure.

Rev. 2 Replaced Distractor B with another contingency action step 7.A out of the RCP Emergency Procedure which directs performing Standard Attachment 21 Step 4. Also updated the analysis for B. Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 8

H 3

X M

U E

S Modified 2003 Bank Question In stem, give an actual RCS leak rate instead of "in excess of charging pump capacity. Part 2 of distractor B not plausible. If MTS is a criteria to throttle, it is not plausible to throttle to restore MTS.

Distractor C not plausible as one cannot "control" RCS inventory and pressure with full HPSI flow, only "restore" Rev. 1 Added an actual leak rate to the stem. Added a bullet to state that all RCPs are secured. Editied distractor C to change "control" to "monitor."

Rev. 2 In distractor B, replaced the words" within limit" at the end with "below the MAXIMUM Limit" to be more plausible. Also in distractor D, replaced the words "within limits" at the end to "above the MINIMUM limit" to be more plausible. Question is now SAT.

9 H

4 X

N E

S After the conditions in the stem are given, the question should read: "Which of the following describes the starting and stopping of 2DG1?"

Remove the words without "Red Vital DC Power."

That cues that at least some controls will have to be performed locally. Also, is it possible in the plant conditions to state a breaker opened (i.e.), or somehow make the applicant determine that Red Vital DC Power is lost, rather than just stating it?

Rev. 1 Changed bullet #3 to a loss of 2D23.

Removed the words "without Red DC" from the stem. Question is now SAT.

10 H

4 X

B E

S 2008 Bank Question Is slowing and rapidly defined somewhere? Rather something is changing slowly or rapidly is a judgement call.

Rev. 1 Changed stem to compare conditions, and changed slowly and rapidly in distractors to rising or lowering faster or slower. Question is now SAT.

11 H

3 N

S

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 12 F

2 N

S Question should state, "Standard Attachment 25 requires that 120 VAC " (It is unnecessary to state

'2'.) Why are the 2RS1 & 2RS2 choices plausible?

The distractor analysis just states wrong buses, not why they are plausible.

Rev. 1 Made minor edits. Provided distractor analysis on 2RS1 and 2RS 2.

13 H

4 N

S 14 H

3 N

S 15 H

3 M

S Modified 2012 Bank Question 16 H

2 N

E S

Since the incorrect distractors are all random numbers, should Distractor D be greater than 500°F, which the AOP would direct tripping the reactor?

Rev. 1 Modified distractor numbers to allow distractor D to be above 500°F. Question is now SAT.

17 H

2 X

N U

E S

It doesn't seem plausible that anyone would think turbine first stage pressure would increase on an upstream steam line break.

Rev. 1 Modified distractors A & D to turbine first stage pressure remaining the same. Question now EDIT.

Rev. 2 Changed up the stem as requested adding a reference to the turbine being on the Load Limit Pot and COLSS operable. Added the word "Indicated" to the beginning of each distractor and answer.

Question now SAT.

18 H

3 X

N E

S Is there an instance where there is an issue during fuel movement where the first action is something other than "suspend moving irradiated fuel?"

Rev. 1 Modified distractor A so the correect answer is the second part of the TRM. Question now SAT.

19 F

2 X

X B

E S

2008 Bank Question Distractor A, change "cold water" to "feedwater."

Rev. 1 Changed cold water to feedwater.

Rev. 2 Changed "refill" to "add" in distractors A and B. Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 20 H

2 N

E S

REFERENCES PROVIDED Change "approximately" with "nearest to" in the stem. Overlap with scenario 3, where "ATC and BOP will perform RCS Leak rate determinations."

Replace question or change scenario.

Rev. 1 Complete rework on distractors, and the direction of the stem. Question now SAT.

Rev. 2 Made the last blank shorter to make the stem 2 lines instead of 3.

21 H

3 B

S 2008 Bank Question 22 H

3 X

M U

E S

Modified 2008 Bank Question Distractors C an D immediately discarded as they indicate a specific value (at 6.0 in.) versus exceeding a value (7.0 in.)

Rev. 1 Modified all distractors to be specific values.

Question now EDIT.

Rev. 2 Changed the stem question to a statement with blanks in a 2 X 2 format. Question now SAT.

23 H

3 N

U S

Distractors B and D can immediately be discarded because, if the SPTAs don't require emergency boration, then the available emergency boration paths are irrelevant.

Rev. 1 Reworded the stem, and changed part 2 of distractors B & D to more plausible conditions.

Question still UNSAT.

Rev. 2 Removed the 1st part of the distractors to determine if emergency boration is or is not required and changed the stem to just ask the second part.

Added "and/or any applicable standard attachment/exhibit directs" as the actual guidance for selecting the correct emergency boration flow path is in OP 2202.010, Standard Attachments, Exhibit 1.

Question now SAT.

24 F

2 M

S Modified 2009 Bank Question

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 25 H

2 B

E S

2009 Bank Question Please explain why anyone would choose "cntmnt aux" and not "cntmnt" when the question specifically asks for containment (distractors C and D).

Rev. 1 Changed references to accident being in containment, and chaned distractors C and D from CTMT AUX to PLANT.

Rev. 2 Added a bullet to say "OP-2502.001, Refueling Shuffle, Att. M, Refueling Accident is in progress". Reworded the first part of the stem to say "Per the procedure, to inform the required individuals to commence an evacuation". Added the word "only" to A and B. Added the words "and CNTMT" to C and D. Question now SAT.

26 F

1 X

X X

B U

E S

2012 Bank Question K/A Mismatch. The K/A requires knowledge of the effects of voltage changes on performance. The stem tells you what the effects are. LOD=1. Most conservative answer is the correct answer - D obviously correct. Question has no discriminating value. Also, the stem is flawed as it should as for what action is required vice "correct."

Rev. 1 Rewote entire question to meet the K/A.

Question now EDIT.

Rev. 2 Added the words "on SU Channel 2 NI" to the end of the 2nd bullet instead of "required by TS Surveillance 4.9.1". Question now SAT.

27 H

3 X

X M

U S

Modified 2015 Bank Question Regarding distractor D, concerned that someone may state that the stem does not specifically state that resin was replaced, and would be making an assumption. (Technically, the distractor is also a correct answer.) That knowledge can't be put in the stem or it would be cueing. Should there be a trend on the I-313 activity? Distractor C is implausible as it is too vague. No "chemical" is specified.

Rev. 1 Modified stem to ensure D not correct.

Modified distractor C. Question now SAT.

28 F

2 X

X M

S Modified 2015 Bank Question

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 29 H

3 B

E S

2014-2 Bank Question Listed as a modified bank question. Stem does not appear to be substantially different from the original bank question, and the correct answer is essentially unchanged. Should be listed as a bank question.

Rev. 1 Replaced the modified version of NRC Exam Bank 2118 with the exact Bank 2118 version as requested. Question now SAT.

30 F

2 B

S 2014-2 Bank Question 31 F

2 B

S 2011 Bank Question 32 F

2 X

X N

E S

Distractors A and B are easily eliminated. Since there can not be more than one correct answer, part 1 to A or B can not be correct because that would also make part 1 to C or D correct. Need to add "Minimum/Maximum" in the stem to avoid subset issues.

Rev. 1 Added "Minimum and Maximum" to stem to eliminate subset issues. Question now SAT.

Rev. 2 Capitalized the words minimum and maximum 33 H

1 X

M U

E S

Modified 2009 Bank Question It is also LOD=1 knowledge that two channels of PPS should not be placed in bypass at the same time. This leaves D as the only possible choice.

Rev. 1 Changed "AND" to "OR" and different channels to increase LOD. Question now EDIT.

Change "3 or 4" to "1 or 3" to be more realistic.

Rev. 2 Changed "3 OR 4" to "1 OR 3" in distractors A and C. Changed RPS to PPS in the stem.

Question now SAT.

34 F

2 N

S There appear to be multiple opportunities for performing boration during the scenarios. Ensure that there is no overlap.

Ensured no overlap.

35 H

3 X

X N

E S

The question states that both feed pumps will trip, thus it does not seem plausible that anyone would perform a rapid load reduction.

Rev. 1. Swapped the part 1 and part 2 distractors.

Modified the stem to indicate that IF the main feed pump suction pressure continues to degrade...

Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 36 H

3 X

X M

E S

Modified 2005 Bank Question Distractor A not plausible as it is apparent from the screen shot that the tech spec limits have not yet been exceeded, thus there is no restoring.

Distractor C should state to prevent "exceeding T.S.

3.6.1.4, Containment Internal Pressure/Temperature, limits" instead of just "TS limits" like the other distractors.

Rev. 1 Added "/humidity" to distractors C and D.

Added "exceeding TS. 3.6.1.4, Containment Internal Pressure/Temperature, limits." to distractor C.

Question still EDIT.

Rev. 2 Added the word "internal" to distractor D between Containment and temperature. Also removed the white "X" from the attachment as requested. Question now SAT.

37 F

2 X

N S

Please explain why it is plausible that someone would think a flywheel could prevent or mitigate a seal problem?

Rev. 1 Modified distractor B analysis to better explain the plausibility of distractor B 38 H

3 B

S 2002 Exam Bank Rev. 1 Added gauge number and minor edits on the stem.

39 H

3 X

B E

S 2008 Exam Bank Need to add MINIMUM to the stem to avoid subset issues.

Rev. 1 Added minimum to eliminate subset issues.

Question now SAT.

Rev. 2 Capitalized the word "minimum" 40 H

3 B

S 2009 Exam Bank This is listed as a modified bank question, but according to ES-401, page 8, there was not a pertinent change to the stem or the distractors to determine this is a modified bank question.

Rev. 1 Changed question from Modified to Bank

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 41 H

1 X

N U

S It is LOD=1 knowledge that HPSI pumps can't be used for shutdown cooling, thus distractors B and D are not plausible.

Rev. 1 Modified distractor analysis as to why HPSI pumps are plausible. Question still UNSAT.

Rev. 2 Removed the words "Red Train" from the beginning of the 2nd bullet. Changed "HPSI Pump 2P-89B" in distractors B and D to "Spray Pump 2P-35A". Question now SAT.

42 F

2 X

N E

S Add MAXIMUM to pecent level to avoid subset issues.

Rev. 1 Added the words " the setpoint of" before the 1st blank in the stem to eliminate subset issues.

Question now SAT.

43 H

3 X

B E

S 2009 Exam Bank What are chilled water and service water aligned to, and is it plausible that both would be aligned simultaneously?

Rev. 1 Changed the format of this question into a 2 X 2 and deleted references to the CC fans running and the position of the bypass dampers. Question now SAT.

44 F

3 N

S 45 F

2 X

N E

S There appears to be something wrong with the distractor analyses for C and D. Both state the smaller valve opening first, yet the distractors have different valves opening first. Why could one not argue that distractors C and D are correct, since they are accurate statements? Is there a way to word the stem such that they can be eliminated as correct answers? Something like, "... and the primary reason for the sequence?"

Rev. 1 Replaced the word "correct" with primary, and clarified the plausibility of distractor D. Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 46 F

2 N

E S

Since the normal bleedoff flow is to the VCT, distractor D is implausible because of the word "divert." Best to modify the stem to state something like, "which of the following describes where the controlled bleedoff flow will be directed?" and remove "Diverted to the" from the distractors.

Rev. 1 Modified the distractors such that they are all "directed to" rather than "diverted to."

Rev. 2 Revised the stem to Say "RCP Controlled Bleedoff (CBO) flow will be directed to the..."

Question now SAT.

47 H

3 X

N U

E S

How is it plausible that anyone would think SDC would be intentionally unavailable after a RAS signal (distractors B and D). In stem, remove the word "leak" from the 5th bullet.

Rev. 1 Removed the word "leak" from the 5th bullet and changed the stem/distractors and their associated analysis to SDC HX would be aligned automatically/manually. Question now EDIT.

Rev. 2 Changed "would " to "should" prior to the 2nd blank in the stem. In the 1st part of the distractors, changed "would and would not" to "is expected and not expected" Question now SAT.

48 H

3 N

S 49 F

1 X

N U

S LOD=1. It is not realistic to think that an applicant would not know what panel the PZR spreay controls are on.

Rev. 1 Removed the 1st part of each distractor and revised the stem to just ask for the 2nd part of the stem. Question now SAT.

50 F

2 X

N E

S Is it possible to give specific buses rather than noun names? Since distractor B is the only one that does not include "Vital," it is easily eliminated.

Rev. 1 Modified all distractors to give specific buses rather than noun names. Question now SAT.

51 F

2 N

S

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 52 H

3 X

N U

S Part 2 distractors are generally not plausible. It is not plausible that sump valves would be closed and RWT valves would be open after a RAS (distractors A and D).

Rev. 1 Added "With no operator action" to the beginning of the stem. Added the word "remain" after pumps in each distractor and answer. Added the word "remain" prior to Open in distractors A and D.

Question now SAT.

53 F

1 X

N U

S LOD=1. No discriminating value. The correct answer is the only choice that is a barrier to the RCS. The other things are easily dismissed as simply having to do with pump operability. Distractor C is a subset of B (if lube oil is overheating, will lead to bearings overheating).

Rev. 1 Rewrote the stem to eliminate questioning public health and safety. Modified distractor C to hydrostatic bearing which is cooled by RCS flow.

Question now SAT.

54 F

2 X

N S

This is an extremely confusing question.

Rev. 1 Changed the word "intensity" to "dose rates" in the stem. Removed the word "more" from distractor D prior to "RCS" 55 H

1 X

N U

S 2009 Bank Question Requal Exam Bank REFERENCES PROVIDED Fail to see how distractors A, B, and C are credible with a reference provided. Since the reference is provided, this is a simple arithmetic problem, and LOD=1.

Rev. 1 Formatted the question to a two by two and added a second part to the stem and made the distractors more plausible and more closely matched the K/A. Question still UNSAT.

Rev. 2 Resampled the K/A and substituted with a new bank question. Question UNSAT Not plausible to believe that an EDG would ever start on undervoltage, and the output breakers would remain open, making distractors A, B, and C implausible, and D as the only correct answer.

Rev. 3 Generated a new question by giving the final conditions in the stem and asking for the voltages in the choices. Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 56 H

3 X

X M

U S

Modified 2009 Bank Question Answer is in question. The stem says that the cooling tower basin is low, wich means it is going to need more makeup, making only B and C plausible.

Also, the word "squeeze" implies "close," cueing B as the correct answer.

Rev. 1 Renamed valve to proper noun name (not "squeeze"). Reworded part two distractors to question pressure rather than whether level will change. Question now SAT.

57 H

3 X

X N

U S

Distractors A and C are not plausible. It is obvious that pressure will increase. Please explain why anyone would think SG level would rise on an MSIV closure (distractors A and C). The two SG's are now assymetric - how does this not cue A or D as the correct answer?

Rev. 1 Added "prior to plant trip" to the stem.

Reworded distractor analysis for distractors A and C.

Question still UNSAT.

Rev. 2 Removed the 2nd half of the Rev. 1 question on all 4 distractors/answer. Revised the stem and question to a 2 X 2 format using the parameters on the "A" and "B" Steam Generators. Question now SAT.

58 H

4 X

N E

S REFERENCES PROVIDED Don't understand why a reference is neeeded. Also, the BAM pumps are not a source of boric acid, they just deliver boric acid from the source - distractors B and D should be BAM tanks.

Rev. 1 Deleted the "Pre-Fire Plan" Reference.

Deleted any words associated with emergency boration in the 3rd bullet and the stem. Revised the first part of the stem to say "Based on the above conditions, boration should be commenced from the" as suggested. Changed 1st part of A and C to "

RWT directly to the Charging pumps" and B and D to "BAMTs using the BAM pumps" Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 59 F

3 B

E S

2008 Bank Question Which is it? Distractor analysis does not correctly state why distractor A is credible. (The distractor analysis for distractor A is the same as for distractor D.) Does the closure occur on the alarm or the setpoint? Why does distractor A give a specific number of valves, and the other distractors just say "ALL?"

Rev. 1 Added "all six" to distractor B. Updated distractor analysis for distractor A.

Rev. 2 Added the word "only" to the end of distractor A. Question now SAT.

60 H

3 X

X N

E S

Answer is in question. Why would anyone think that a letdown rad monitor isolation valve would not isolate, but bypass?

Rev. 1 Added "at full power" to the stem. Replaced "has lowered to" with "is reading." Removed the word "isolation" from the valve noun name. Need to remove "letdown flow goes to maximum flowrate" from the stem. That cues the correct answer.

Rev. 2 Removed the 4th bullet from Rev. 1.

Question now SAT 61 F

2 X

N E

S Don't understand why anyone would think that area rad monitors monitor neutron radiation. Distractors B and D implausible.

Rev. 1 Added a specific ARM to make neutron reading more plausible. Question now SAT.

62 F

2 X

X N

E S

Leakage into containment cues the correct answer of "containment atmosphere."

Rev. 1 Changed the stem to a specific leak location and removed any reference to inside containment.

Changed main steam rad monitor to letdown line rad monitor. Question now SAT.

63 F

2 B

S 2014-2 Bank Question 64 H

3 X

N S

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 65 H

3 X

B E

S 2002 Bank Question Distractor A is not plausible, as it is a "none-of-the-above" type distractor, in that, after everything that goes on in the stem, nothing happens.

Rev. 1 Removed the status of the release from the distractors, and added the 2P-53A status to the distractors to have an action in each distractor.

Question now SAT.

66 F

2 X

X B

E S

2001 Unit 1 Bank Question Distractor A only discusses installation and distractor D only discusses preparer. The correct answer does not discuss installation, which the stem is in part asking about. Need to add "per procedure XXX' in stem. All of the incorrect distractors have "may" in them and only the correct answer has "must."

Reword distractors such as to not make correct answer so obvious.

Rev. 1 Added and/or to the step along with the procedure that requires the answer. Question now SAT.

67 F

2 X

N E

S As a procedure question, if you are unable to communicate the count rates, how are you going to communicate to suspend core alterations?

Rev. 1 Added the words "until communications have been restored" to the 2nd part of A and C. Changed the 2nd part of B and D to "suspend core alterations after 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> if communications have not been restored". Question now SAT.

68 F

2 X

B U

S 2008 Bank Question This is GFE level knowledge. Picture of a stop check valve is not "interpreting mechanical drawings." Regarding distractors A and B, why would any Unit 2 applicant expect to have a question regarding a Unit 1 print on their exam. Distractor D could also be argued correct, as a stop-check valve is a type of check valve.

Rev. 1 Resampled the K/A and substituted with a new bank question. (Initial question was NEW.)

Question is now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 69 F

2 X

N U

S 2012 Bank Question Distractors A and C not plausible. No one would believe that an ATC could leave early without the shift manager or CRS permission.

Rev. 1 Revised the question to delete the requirement to get permisssion to leave.

70 F

2 X

B E

S 2002 Bank Question Distractors B and D not plausible. No one would believe a green annunciator has priority over a red one.

Rev. 1 Revised all distractors so that a red annunciator is always the highest priority. Should change orange to amber, and consider changing yellow to something else since there are no yellow lights in the control room.

Rev. 2 Changed "Yellow" to "Amber" in distractor B and "Orange" to "Amber" in distractor D.

71 H

3 N

S 72 F

2 X

X N

E S

Get rid of ONE (1) in stem. Need ? at end of sentence in stem. Change "can" to "may" in distractors. Disctractor B is not credible, as many actions cannot be "undone" (such as a manual reator trip). Distractor C not credible - skip an EOP step with no SRO concurrence? No one would go for that. D is obviously the correct answer, as it is the only answer that includes any SROs Rev. 1 Removed one (1) from the stem. Changed "can be undone" to may be reversed in distractor B.

Changed ATC to CRS in distractor C.

Rev. 2 Added the words "without reversal" to distractor A. Added clarifying words to B,C, and D to balance all 4 choices. Capitalized the words "must" and "may" in the 4 choices. Question now SAT.

73 F

2 X

N E

S This is covered in great deal during the scenario portion of the operating test + has nothing to do with emergency procedures/plan.

Rev. 1 Added "during assessment of annunciators at the end of SPTAs" to the steam to tie the question to the EOPs. Question now SAT.

74 F

2 N

S

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 75 F

2 X

B U

E S

2012 Bank Question 2004 Bank Question Distractor B not plausible since the direction was to start "the" pump (singular), why would anyone believe the repeat back to start both pumps? Part 2 of the distactors is the same in all four distractors, so it is not necessary. This competency is already being tested thoroughly during the scenarios (replace)

Rev. 1 Changed distractor B to "start the EFW pump" and made the question into a statement that includes the second part of the distractors. Question still UNSAT Rev. 2 Resampled the K/A and substituted with a new bank question. Question UNSAT. Distractor A is also correct, thus there are two correct answers.

Rev. 3 Changed distractor A to a continuous action step description due to feedback that the Rev. 2 distractor A could be a potential correct answer.

Question now EDIT.

Rev. 4 Reworded Distractor D to the verbiage in step 4.20 of EOP/AOPuser guide. Added clarifying words to choices A, B, and C to balance the question and clarify the choices. Question now SAT.

76 H

3 N

S Why is distractor B not also correct? When implementing the ESD EOP, you FIRST come to step 21 which is to stabilize RCS pressure, then reduce pressure to maintain MTS less than 200°F.

Rev. 1 Added clarification of why distractor B is incorrect.

77 H

3 B

E S

2015 Bank Question "Woulds" should be "shoulds." Why are you mixing woulds with musts (inconsistent). Distractors C and D do not fit stem sentence grammatically.

Rev. 1 Changed would to should in distractors A and C. Added "the" to distractors C and D. Question now SAT.

78 H

3 M

S Modified 2014 Bank Question Change the word "diagnose" to "direct" in the stem.

(You don't really diagnose entry into SPTA.)

Rev.1 Changed diagnose to direct in the stem.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 79 H

3 N

S 80 H

4 X

B E

S 2015 Bank Question Need to add MAXIMUM in second half of stem to avoid subset issue.

Rev.1 Added maximum to the stem.

Rev. 2 Capitalized MAXIMUM. Question now SAT.

81 H

3 X

N E

S REFERENCES PROVIDED There are two bullets in the stem regarding pressurizer level. I would remove bullet 6, and replace it with bullet 2 (i.e., move bullet 2 down in the stem). Redundent to JPM A9 (either replace Q or replace JPM)

Rev.1 Combined bullets so that there is only one statement regarding pressurizer level and pressure.

Licensee to replace JPM A9. Question now SAT.

82 F

2 X

X N

E S

The use of the P&ID is unnecessary and distracting.

Part 2 distractors can simply be referred to as "the associated containment isolation valve."

Rev.1 Removed P&ID, and changed wording in stem to say "associated containment isolation valve."

Question now SAT.

83 H

3 X

N E

S Two correct answers. If D is correct, then B is correct.

Rev.1 Modified stem to add trend to aux building monitor readings. Added information on why B is not correct since the entire contigency step must be completed, not just substep A.

Rev. 2 Removed "if radiation levels continue to rise" from distractors C and D. Question now SAT.

84 H

3 N

S Knowledge of pressure boundary leakage is LOD=4?

Rev.1 Changed LOD from 4 to 3.

85 H

3 X

N E

S REFERENCES PROVIDED Need to identify "assuming no operator action in the stem" to ensure there is no confusion on if corrective actions could be taken.

Rev. 1 Added "assuming no operator actions have been taken" to the stem. Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 86 H

3 X

N E

S Why is the LOCA procedure plausible? The distractor analysis states due to RCS and containment conditions, but pressure is stable, level is rising, and the only containment parameter given in containment radiation levels.

Rev. 2 Moified distractor analysis to strengthen why the LOCA procedure is plausible. Should make a bus fault to make applicant have to diagnose loss of AFW pump.

Rev. 3 added "Bus 2A1 faults and cannot be re-energized" to de-energize AFW pump 2P75 Changed stem to 'B' EFW OOS for maintenance. De-energizing 2A3 would have also been an overlap issue with Q-51. Question now SAT.

87 H

4 N

S Capitalize MINIMUM in stem Rev.1 Capitalized minimum in stem.

88 H

3 X

N E

S Can the CRS be in Lower Mode Functional and an AOP at the same time? If so, the use of the words "must transition" and "can implement" can cause a multiple correct answers issue. In distractors, change "can" and "must" to "should."

Rev.1 Added "should" to the stem and removed can and must from the distractors, to avoid issue of multiple correct answers. Question now SAT.

89 H

3 M

S Modified 2014 Bank Question 90 H

4 N

S In the stem, Remove the word "Vital" from the last bullet Rev.1 Removed the word vital from the stem.

91 H

4 N

S 92 H

4 B

S 2011 Bank Question

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 93 H

3 N

U S

Why is the distractor order A, C, B, D? Add ONLY to the second distractor. Distractors B and C are out of order and are both implausible. B uses the remote shutdown procedure without using the remote shutdown instrument and C uses the remote shutdown instrument without using the remote shutdown procedure. This makes distractors A and D the only plausible choices.

Rev.1 Placed distractors in the correct oder.

Changed the stem from "Remote Shutdown Panel" to "Channel 2." Changed distractor B to a single procedure and instrument. Changed distractor C to both surveillances on a single instrument, plausible because a RSDP instrument is also PAM required.

Question now SAT.

94 F

3 X

N U

S Distractors C and D are only plausible choices as they are the only two disractors that contain an SRO.

Also, distractors A and B refer to the outage manager, who would have no specific knowledge of specific refueling activities.

Rev. 1 Corrected procedure number in stem.

Question still UNSAT.

Rev. 2 re-formatted the question adding "SRO in charge of refueling to the stem" and asked who's additional approval is required for a fuel shuffle.

Question now SAT.

95 F

3 X

N U

S Distractors C and D not plausible in that they are identifying positions in the ERO that are not manned yet, according to the stem. I don't think anyone would think 25 Rem would be OK just to save equipment and the SM is almost never second (if ever) to RP when it comes to on-shift approvals during an emergency.

Rev.1 Changed the stem to after the TSC is activated. This changes the correct answer to Emergency Plant Manager, and makes the radiological coordinator plausible. Question still UNSAT. Not plausible to think 25 R would be appropriate for saving equipment.

Rev. 2 changed the wrong limit from 25 rem to 5 rem, thus changing (original) distractors B and D.

Changed order making B correct. Question now SAT.

ES-401 AN2-2017-02 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S Explanation

3. Psychometric Flaws

4. Job Content Flaws

5. Other 96 F

3 B

S 2009 Bank Question 97 F

3 X

X N

E S

Fire where? What "large fire" would require evacuation of Unit 2 CR but not Unit 1? Don't see how "Unit 2 control room SM" is a viable choice given that the Unit 2 CR is uninhabitable. Maybe better if "control room" was removed from the distactors and just refer to the Unit 1 or Unit 2 SM.

Rev.1 Removed "large" for describing the fire.

Changed Control rooms to Shift Managers for part 1 distractors. Question now SAT.

98 H

1 X

M U

S Modified 2015 Bank Question With the success path tracking page, this is an LOD=1 question. The original bank question had a higher LOD without the attachment. Also, the only distractor that starts with a system in "jeopardy" is the correct answer, making it obvious. Need to put more stuff in jeopardy and have them prioritize those.

Removed the attachment and added conditions to the stem. Made an additional safety function jeopardized and changed the question to 2x2.

Question now SAT.

99 F

3 X

N E

S Need to add MINIMUM in stem to avoid subset issue.

Rev. 1 Added minimum in stem to avoid subset issue. Need to review new JPM A9 to ensure there is no cueing. Question now SAT.

Rev. 2 Capitalized MINIMUM.

100 F

3 N

S