Text

2015-12 Draft Written Exam Comments
	ML16028A019
Person / Time
Site:	Grand Gulf
Issue date:	12/14/2015
From:	Vincent Gaddy; Operations Branch IV
To:	; Entergy Operations
References
	50-416/15-012, 50-417/15-012
	Download: ML16028A019 (26)
	v • d • e

ES-401 GG-2015-12 Written Examination Review Worksheet Form ES-401-9 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any Unsat ratings (e.g., how the Appendix B psychometric attributes are not being met).

_Rev 1 Page 1 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only General Comments:

1.

No LOD on any question as required.

2.

Many questions (particularly SRO) do not have a complete 55.43 reference.

3.

Some worksheets not completed correctly (if no reference is to be provided, then put none (this is one example).

4.

Answers/Distracters with no actions required or nothing happens are not allowed by the NUREG-1021

5.

Lots of cueing issues in the various questions

6.

Several reverse logic questions

7.

When using questions from the bank or old NRC exams you have to verify that all distracters are credible since they werent required to be in older revisions of NUREG-1021 (ie rev8).

8.

There are too many references for the SRO exam.

ES-401 GG-2015-12 2

Form ES-401-9

_Rev 1 Page 2 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only

9.

You always have to send a References or hand-outs file that contains all the material you intend to hand out to support the questions that need references. We didnt see this and will need this provided for exam approval.

10. Putting What does the CRS direct in a stem does not make a question SRO only.

11. Per your transient mitigation strategy, a CRS is required to direct subsequent actions; however, these actions, if, based on RO knowledge, would still be RO level questions. You have to frame the question around procedure selection for the SRO-only guidance to be met or non-entry level conditions as bounded in the SRO-only guidance.

12. When you scanned the Ro and SRO exams into pdf, you scanned them as image files and therefore are not word searchable. We need them to be word searchable.

13. When you send a modified question you are required to send the parent question and you did not send these with the submittal. This affects the review of Q30, Q33, Q51, and Q82.

1 H

3 41.7 C

B S

S 55.41.b7 No LOD on worksheets.

Question was swapped out due to being too similar to Question 46.

2010 NRC exam New question is SAT SAT WITH REV 4 SUBMITTIAL 2

H 2

X 41.7 C

B E

55.41b7 2011 NRC I dont think that B or D are credible but since this question was previously approved on an NRC exam we would like some repairs made to it.

How about this: Take the 11DC out of the stem and make the choices be:

A.

Remains constant on the 11DB bus B.

Remains constant on the 11DC bus C.

Lowers by 5 to 10 volts on the 11DC bus D.

Lowers by 5 to 10 volts on the 11DB bus An even better question is this:

ES-401 GG-2015-12 3

Form ES-401-9

_Rev 1 Page 3 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

After a loss of AC power to both respective bus chargers, each ESF battery bank can supply its required DC loads without battery voltage dropping below :

A.

2.33 volts per cell for four hours on Div 1 and 2 B.

C2.33 volts per cell for two hours on Div 1 and 2 C.

2.13 volts per cell for four hours on Div 1 and 2 D.

2.13 volts per cell for two hours on Div 1 and 2 SAT WITH REV 4 SUBMITTIAL 3

F 2

41.6 C

N E

S 55.41b6 Remove the to in Turbine Control Valves to close in the stem OR stem edit to close to closing.

SAT WITH REV 4 SUBMITTIAL 4

H 3

41.6 B

X N

E 55.41b6 This is a negatively focused question, which is not the preferred way to write a question. See example D.2 on page B-21 of NUREG-1021, Appendix B for more details. Also, another way to think of this is the nugget of info you actually get from this question is when can rx water level ever be bypassed in RPS and the answer is never so you are really asking how does water level relate to SCRAM not reactor power to SCRAM.

A better way to write this question might be:

The unit is currently at rated power. Recirc flow is Wd = some value.

Which if the following will cause a reactor SCRAM on high reactor power if the coincidence is met (2 out of 4 voter)?

A.

APRM upscale neutron flux at 115.3%

B.

APRM upscale neutron flux at 117.3%

C.

APRM upscale Thermal (STP) at calc value D.

APRM upscale thermal (STP) at calc value Answer is B, make the calculations in C and D below the setpoint or just put in the clamp value and remove the recirc flow calc, or something like this. If it is a pure number then it would stay fundamental, if you use the formula and make them calc then it is a higher order question.

ES-401 GG-2015-12 4

Form ES-401-9

_Rev 1 Page 4 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

SAT WITH REV 4 SUBMITTIAL 5

F 3

X 41.7 B

B E

S 55.41b7 This has two subsets in it. A is a subset of C and B is a subset of D. Minor adjustments to fix:

Stem.

When placing this switch in the ON position, it isolates what equipment from the control room?

A.

Isolates ONLY the Div 1 powered equipment B.

Isolates ONLY the equipment controlled from RSP-P150 C.

Isolates BOTH the Div 1 and Div 2 powered equipment D.

Isolates BOTH equipment controlled from RSP-P150 and RSP-151 Also, notice that I put isolates from the control room in the stem and removed it from all four answers as required by the NUREG.

SAT WITH REV 4 SUBMITTIAL 6

H 3

X 41.10 A

B E

S 55.41b10 Distractor C is not credible. You would never trip the recirc pumps and force a SCRAM to occur. If you put in the actual SCRAM and then trip the pumps then you have two correct answers. Also, if the nugget you are going after is whether they know the difference in a partial and total loss of CCW based on CRD temp alone (any 1 of the 4 right) then the question might be:

Stem stuff..a reactor SCRAM and trip of both recirc pumps is required when:

A.

CRD pump oil temp cannot be maintained less than 135 degrees B.

CRD pump stator temp cannot be maintained less than 135 degrees C.

Recirc pump bearing temperatures cannot be maintained less than 135 degrees D.

Recirc pump cavity seal temperatures cannot be maintained less than 135 degrees If you use this, you need to ensure all distractors are not correct.

SAT WITH REV 4 SUBMITTIAL 7

H 3

41.7 D

N S

55.41b7

ES-401 GG-2015-12 5

Form ES-401-9

_Rev 1 Page 5 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only SAT WITH REV 4 SUBMITTIAL 8

H 3

X X

41.14 A

N E

E S

55.41b14 You did not assign 55.41XXXXX needs to be corrected 2011 NRC

. I dont think that B or C are credible when juxtaposed with A and D but since this question was previously approved on an NRC exam it needs some edits. Fuel cooling with full coverage versus drained down is an obvious GFE concept for cooling. Fuel that is partially covered will reach boiling much sooner than fuel that is submerged.

You might be able to modify it to use with the head on versus off or look at TS entry conditions (if above the line info can be used).

How about this:

Stem..

Of the following situations, which would result in the SHORTEST amount of time for reactor vessel water temperature to reach 200F and during the time of loss of all decay heat removal how often is reactor water temperature required to be monitored?

A.

prior to any core alterations; once every 60 minutes B.

prior to any core alterations; once every 30 minutes C.

after core alterations have been completed; once every 60 minutes after core alterations have been completed; once every 30 minutes SRO question 79 is very similar and with the reference provided in that question you can get this down to a 50/50 Initial revision still had several flaws Sent a revised question back to GG, the suggestion that we provided was not that great on further review. 11/2/2015 SAT WITH REV 4 SUBMITTIAL 9

F 4

41.13 B

X B

E S

55.41b13 You did not assign 55.41XXXXX needs to be corrected 2011 NRC K/a Mismatch, K/A asks about operational implications of inadvertent criticality not interlocks that prevent it. Marked as an E due to it being used on a previous exam.

Replacement question sent to facility

ES-401 GG-2015-12 6

Form ES-401-9

_Rev 1 Page 6 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only SAT WITH REV 4 SUBMITTIAL 10 F

3 41.7 A

N S

55.41b7 SAT WITH REV 4 SUBMITTIAL 11 H

3 41.6 D

B S

55.41b6 2014 NRC SAT WITH REV 4 SUBMITTIAL 12 H

3 41.10 D

B S

55.41b10 2008 NRC SAT WITH REV 4 SUBMITTIAL 13 H

2 41.6 C

B S

55.41b6 2011 NRC Handout (EP-1 cautions)

SAT WITH REV 4 SUBMITTIAL 14 H

3 41.10 C

N S

55.41b10 SAT WITH REV 4 SUBMITTIAL 15 F

3 41.8 A

N E

S 55.41b8 LOD = 1, too easy. This should involve something more than systems that can be used such as the set point for low level, maybe the min flow valve versus the test return, or even the possibility of gravity feed. For example, what is the significance of the 18.34 ft level it must be at to use RCIC to fill it? Distractor D explanation does not make sense. Can you fill the SP with Core Spray or not? If you can as it is worded, then this would make D correct as well.

Sat with revision SAT WITH REV 4 SUBMITTIAL 16 F

3 41.10 A

N S

55.41b10 SAT WITH REV 4 SUBMITTIAL 17 F

4 41.10 D

B S

55.41b10 2007 NRC SAT WITH REV 4 SUBMITTIAL 18 F

3 41.7 C

X B

E 55.41b9

ES-401 GG-2015-12 7

Form ES-401-9

_Rev 1 Page 7 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only E

S 2013 NRC KA mismatch but because we allowed it on the 2013 exam it is marked as E.

Knowledge of reasons for things are K3 KAs, this is a K2 KA. Also, this is in the EOP section of the catalog so it is asking the interrelations between an actual high off-site release rate and plant ventilation systems. It would be appropriate to ask the high-high rad setpoint or how Control room or SGTS realigns when rad levels reach a very high point during a big offsite release or ask about SGTS interrelations with the high rate, but you have to have a high rate occurring and something else to have the interrelation.

55.41(b)(7)

New question had subset issues and did not tie the question back to the K/A a proposed fix was sent to facility 11/4/2015 SAT WITH REV 4 SUBMITTIAL 19 H

2 41.10 D

B E

S 55.41b10 2012 NRC You did not send us the package of references to be provided to applicants (ie the official hand-outs) and this one looks like it needs a lot of stuff to answer. The question is marked as E because it was previously approved but it is not a balanced 2 X 2. Once you know the CO2 part, distractors A and D are eliminated. This means that knowledge of the other parts of them are not needed to answer the question.

This could have been made easier with just a fire in the room with the equipment and ask what equipment is affected by the fire. Reference provided (Whole bunch).

Question modified and is Sat with revision Reference (10-S-03-2 cover, attachment III and attachment IV page 125-135)

SAT WITH REV 4 SUBMITTIAL 20 F

2 41.7 B

X N

E 55.41b7 Question: Do you need the alarm window information to reach the same conclusion with the values provided? If not the alarm windows are not needed and the KA is not met. Are the alarm windows colored based on priority? If so this might be a way to setup the question. Is this operationally valid to ask it this way with no turbine trip alarm in? In pedigree the reactor screams. I dont think we want a screaming reactor that might be bad.

SAT WITH REV 4 SUBMITTIAL

ES-401 GG-2015-12 8

Form ES-401-9

_Rev 1 Page 8 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

21 H

3 X

41.5 A

B E

S 55.41b5 2013 NRC B and C distractors need editing. B is different than the other choices in that it does not have a reason for the change. It needs to have a reason added to it to be balanced. Something like due to less thermal driving head. C needs to be more credible (increased flow not credible) as follows:

A.

Answer B.

Flow rate will decrease initially and then increase to a new equilibrium value slightly less than the original flow rate due to less water (head)

C.

Flow rate will not be significantly affected because the decrease in water (head) will be offset by the increase in thermal driving head D.

Leave as is SAT WITH REV 4 SUBMITTIAL 22 H

2 X

41.10 D

N E

S 55.41b10 Edits needed Should not use abbreviations per NUREG unless you have a key for them. RPV is okay. I dont think CTMT is an approved abbreviation.

A.

Okay B.

add s to end of system in this one C.

Drywell pressure 1.2 psig. Vent primary containment to control pressure.

D.

Containment temperature 98F. Operate all available containment coolers SAT WITH REV 4 SUBMITTIAL 23 H

3 X

41.1 A

N U

S 55.41b1,5,7 General fundamental question, KA 292003 K1.05 and K1.09. Need to incorporate specific instruments used at GG to read these values or give them a trace and have them read it to get there (whatever you guys have) to make it plant specific.

SAT WITH REV 4 SUBMITTIAL 24 H

2 X

41.10 A

N U

55.41b10 If you put in the noun name for Attachment 20 (which is required by the NUREG) it

ES-401 GG-2015-12 9

Form ES-401-9

_Rev 1 Page 9 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

cues to your answer. More importantly, the reason that you defeat the control rod insertion blocks is to get rods in for the ATWS. The reason the rod blocks could be impacting rod insertion is why you do it-such as the turbine 1st stage pressure, rod pattern, etc. That is the way the question should be written.

SAT WITH REV 4 SUBMITTIAL 25 H

3 41.11 A

B S

55.41b11 2011 NRC SAT WITH REV 4 SUBMITTIAL 26 H

3 41.8 D

B S

55.41b8 2012 NRC SAT WITH REV 4 SUBMITTIAL 27 H

2 41.7 C

B S

55.41b7 SAT WITH REV 4 SUBMITTIAL 28 F

2 41.8 D

B E

S 55.41b8 2008 NRC This is not Higher order question. Change to fundamental because all the operator has to know is the shutoff head of the pump and the pressure of the RPV near that point but just below for flow to go in.

SAT WITH REV 4 SUBMITTIAL 29 H

2 41.7 C

N S

55.41b7 SAT WITH REV 4 SUBMITTIAL 30 F

3 41.10 B

B E

E S

55.41b10 2011 NRC Need to add why C is the correct answer in explanation This question had more than one correct answer the way it was written I sent a revised stem to the facility for review SAT WITH REV 4 SUBMITTIAL 31 F

2 X

41.7 D

B E

55.41b7

ES-401 GG-2015-12 10 Form ES-401-9

_Rev 1 Page 10 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

2007 NRC Distractor C makes no sense-if the valve is initially closed and we are asking when it would auto open, why would we state that it was closed in the distractor C (that seems trivial and obvious and therefore not credible). Make the distractors be:

A.

System flow is 1300 gpm and discharge pressure is 125 psig B.

System flow is 1200 gpm and discharge pressure is 125 psig C.

System flow is 1300 gpm and discharge pressure is 135 psig D.

System flow is 1200 gpm and discharge pressure is 135 psig SAT WITH REV 4 SUBMITTIAL 32 H

3 X

41.6 C

N E

E S

55.41b6 The first part of distractors C and D are not credible. For safety related equipment it is obvious that these would be on EDG power. A simple fix is do they auto restart on loss of power, yes or no?

Question still lacked credibility, I sent a revised question that the facility accepted on 11/5/2015 SAT WITH REV 4 SUBMITTIAL 33 H

2 41.6 D

B S

55.41b6 SAT WITH REV 4 SUBMITTIAL 34 F

2 41.10 A

B S

55.41b10 SAT WITH REV 4 SUBMITTIAL 35 F

2 41.8 C

N S

55.41b6 SAT WITH REV 4 SUBMITTIAL 36 H

2 X

41.6 A

B E

S 55.41b6 2011 NRC Not credible for a full scram on only one instrument failure, at best a half scram due to only one instrument failure, this is considered an edit because it was allowed on a previous NRC exam SAT WITH REV 4 SUBMITTIAL

ES-401 GG-2015-12 11 Form ES-401-9

_Rev 1 Page 11 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 37 H

3 41.6 A

N S

55.41b6 SAT WITH REV 4 SUBMITTIAL 38 H

2 41.8 B

N S

55.41b8 SAT WITH REV 4 SUBMITTIAL 39 H

2 X

41.7 B

B E

S 55.41b7 2010 NRC D is not a credible distractor as the control room was evacuated in the stem of the question.

SAT WITH REV 4 SUBMITTIAL 40 F

2 41.7 C

B S

55.41b7 SAT WITH REV 4 SUBMITTIAL 41 F

2 X

41.7 B

B E

S 55.41b7 2012 NRC Edit since approved on 2012 NRC exam however cueing must be fixed. In stem have CTMT in switch name and the only choice with containment in it is the answer. To fix, modify as follows:

Depressing the CTMT-DRWL ISOL 1(2) MAN INIT isolation pushbuttons on P870 will A.

Initiate Group 2 valve isolations B.

Initiate Group 8 valve isolations C.

Initiate Group 7 valve isolations D.

Initiate Group 10 valve isolations SAT WITH REV 4 SUBMITTIAL 42 B

F 2

X 41.3 B

N E

55.41b3 Distractor A is not credible with other choices in the list. To correct, make the following changes:

A.

Safety and ADS Relief modes only B.

Leave as is C.

Safety and Low-Low Set Relief modes only D.

Leave as is

ES-401 GG-2015-12 12 Form ES-401-9

_Rev 1 Page 12 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

SAT WITH REV 4 SUBMITTIAL 43 H

3 X

41.8 B

N E

S 55.41b8 Need to bold and capitalize Minimum in stem for emphasis. Need to get tense correct in all distractors. Example, In distractor A A.

ADS actuation has occurred The Division 1 ADS logic must be reset and the respective ADS MANUAL INHIBIT keylock switch placed in the INHIBIT position SAT WITH REV 4 SUBMITTIAL 44 F

3 X

41.7 B

N E

S 55.41b7 Remove hard fail from stem. An operator would not be given that information in the actual control room. Make it operationally valid by changing stem to:

The C34 Feedwater Flow transmitter fails upscale to 12.75 mlb/hr. What effects will directly result from this condition?

A.

The feedwater system will switch to MANUAL level control B.

The feedwater system will disable three-element control C.

An estimated feedwater flow signal replaces the failed transmitter with no change in control D.

Leave as is SAT WITH REV 4 SUBMITTIAL 45 H

2 X

41.7 A

B E

S 55.41b7 2010 NRC Listed as an edit as it was on previous NRC exam Stem has a que in it that needs corrected Reword as follows:

Post-scram hi level control problems have resulted in a trip of both RFPTs.

Everything else okay SAT WITH REV 4 SUBMITTIAL 46 D

F 3

41.7 D

X N

E 55.41b7 This needs work to meet the KA. The stem is not setup for A LOSS OR

ES-401 GG-2015-12 13 Form ES-401-9

_Rev 1 Page 13 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

MALFUNCTION of SGTS.I am not sure what caused all of the stem info to happen. Is this operationally valid as given?

SAT WITH REV 4 SUBMITTIAL 47 H

2 X

41.7 C

N E

S 55.41b7 Distracters A and B are not credible with ESF power regardless of what you give them from PDS. I think the KA is asking if you can use PDS to evaluate the actual AC dist system such as transformer temps, amps, status, etc, not the power supply to condensate pumps (this is the K2 KA).

Reference (PDS screen shot included with question)

SAT WITH REV 4 SUBMITTIAL 48 H

3 41.7 C

X N

E S

55.41b7 This is a reverse or backward logic question. You give them the answer to the first part of the KA in the stem. One suggestion to fix this question is the following:

You could put several conditions in the stem (voltage and current values on both sides of the inverter). Then ask What is the impact on the UPS static switch for the static inverters?..then keep the second part for answering the b part of the KA.

A.

UPS auto transferred due to undervoltage B.

UPS auto transferred due to overcurrent You might also consider a choice of inverters-some auto swap and some do not.just a thought.

SAT WITH REV 4 SUBMITTIAL 49 H

2 41.7 D

B E

S 55.41b7 Need to complete reference field in question with None SAT WITH REV 4 SUBMITTIAL 50 H

2 41.7 B

X N

U S

55.41b7 K/A is asking for the cooling water for the diesel SAT WITH REV 4 SUBMITTIAL 51 F

3 41.4 C

M E

55.41b4

ES-401 GG-2015-12 14 Form ES-401-9

_Rev 1 Page 14 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

Need parent to see if it is modified or Bank question, otherwise Sat SAT WITH REV 4 SUBMITTIAL 52 H

3 X

X 41.10 D

N E

S 55.41b10 2010 NRC This question is highly challengeable as written. The stem does not say initial effect or long term effect, also C is not credible from a thermodynamics perspective. If I get cooling going why is it credible to have the rate of increase going up and temp going up with no other information given?

Sat new question was submitted SAT WITH REV 4 SUBMITTIAL 53 H

2 X

41.10 B

B E

S 55.41b10 2015 BRQ exam Distractors C and D not credible with recirc pumps in the choices for A and B and no scram in the stem. How about this:

Recirc pumps and..

A.

RWCU NRHX only B.

CRD pump oil coolers only C.

RWCU NRHX and CRD pump oil coolers only D.

RWCU NRHX and FPCC HX only SAT WITH REV 4 SUBMITTIAL 54 H

2 41.3 C

N S

55.41b3 SAT WITH REV 4 SUBMITTIAL 55 H

2 41.6 C

B S

55.41b6 SAT WITH REV 4 SUBMITTIAL 56 H

2 41.10 A

N S

55.41b10 SAT WITH REV 4 SUBMITTIAL 57 H

2 41.10 B

B S

55.41b10

ES-401 GG-2015-12 15 Form ES-401-9

_Rev 1 Page 15 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only Reference (05-1-02-III-3, Recirculation System Flow, Figure 1)

SAT WITH REV 4 SUBMITTIAL 58 F

2 41.10 A

X N

U S

55.41b10 Backward Logic, You have the purpose in the stem and the KA is the purpose New Question submitted SAT WITH REV 4 SUBMITTIAL 59 H

2 41.7 C

B S

55.41b7 SAT WITH REV 4 SUBMITTIAL 60 F

3 41.9 D

N S

55.41b9 SAT WITH REV 4 SUBMITTIAL 61 H

3 41.7 C

B E

S 55.41b7 2008 NRC Need to put valve after numerical names (ie F066A valve). Need to have pump after RHR A in each choice.

For distracter D need to make it the same as the other three choices by modifying it to:

RHR A pump will restart in LPCI mode after the F066 valve auto closes and the F004A valve auto opens SAT WITH REV 4 SUBMITTIAL 62 F

2 41.7 A

B U

S 55.41b7 LOD=1, too easy. Main control valves would have to have an analog indication for control. Hence the name. Also, this would be cueing with the green lights only as a choice that these arent correct based on the valve name alone. The panel location is fine but maybe ask the values you might expect at full power at the proper panel from the MCVs.provided they are not all at 100% open.

Replaced with Bank Question from 2011 exam Sat SAT WITH REV 4 SUBMITTIAL 63 H

2 41.7 C

N E

55.41b7 Change distracters A and B to be A.

Throttle open direction Up

ES-401 GG-2015-12 16 Form ES-401-9

_Rev 1 Page 16 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

B.

Throttle open direction Remains same SAT WITH REV 4 SUBMITTIAL 64 H

2 41.5 A

B S

55.41b5 2010 NRC SAT WITH REV 4 SUBMITTIAL 65 F

2 X

41.7 D

N E

S 55.41b7 Distractor C needs some work to make it more credible Change distracter C to an AC BOP bus for credibility.

SAT WITH REV 4 SUBMITTIAL 66 F

2 41.10 B

B S

55.41b10 SAT WITH REV 4 SUBMITTIAL 67 F

2 X

41.10 B

N E

S 55.41b10 To make A and C more credible you need a command and control position when you use authorize for the control room in the stem, it cues to authority positions. Use CRS in place of ATC in these answers and it will be fine.

SAT WITH REV 4 SUBMITTIAL 68 F

2 41.10 C

N S

55.41b10 SAT WITH REV 4 SUBMITTIAL 69 F

2 X

41.10 D

B E

S 55.41b10 2010 NRC Cueing in stem with Tests and only answer with test in it is the answer. This needs work. Given an E since it was previously approved on NRC exams.

SAT WITH REV 4 SUBMITTIAL 70 F

3 41.10 A

B S

55.41b10 2012 NRC SAT WITH REV 4 SUBMITTIAL 71 F

2 41.10 B

B S

55.41b10

ES-401 GG-2015-12 17 Form ES-401-9

_Rev 1 Page 17 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only SAT WITH REV 4 SUBMITTIAL 72 H

3 41.10 B

B S

55.41b10 2011 NRC SAT WITH REV 4 SUBMITTIAL 73 F

2 41.11 B

N S

55.41b11 SAT WITH REV 4 SUBMITTIAL 74 F

3 X

41.10 A

B E

S 55.41b10 Distracter D doesnt answer the question and is therefore not credible. A better question might be who goes where? Example RO1 to P150, RO2 to P-151, etc. if this isnt covered to this specificity in your procedures then just fix D.

SAT WITH REV 4 SUBMITTIAL 75 H

2 41.10 D

B S

55.41.b10 Having the statement that an RO does this function is the only thing that allows this to count otherwise it is an SRO level question are you sure you want it on the exam?

I would prefer that you ask a different question that is clearly at the RO level per the CFR. Otherwise it is extremely appealable.

Sat with new explanation SAT WITH REV 4 SUBMITTIAL 76 H

3 43.5 C

N S

55.43b5 Handout SAT WITH REV 4 SUBMITTIAL 77 H

3 43.5 D

X N

U S

55.43b5 KA is to determine or interpret reactor water level as they apply to control room abandonment. The question is asking what do you do at <160 inches wide range Licensee accepted NRC re-written Q77 and is now Sat.

SAT WITH REV 4 SUBMITTIAL

ES-401 GG-2015-12 18 Form ES-401-9

_Rev 1 Page 18 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 78 H

3 43.5 B

X N

E S

55.43b5 You are still asking a pressure value which is not SRO-only, even if GG states that per the Trans. Mitigation Strategy procedure on who is required to direct these actions, the knowledge is still RO and it isnt selecting a procedure or sub-section of this procedure. This is also not a comprehensive question since you are asking one piece of information. This is not selecting a subset of a procedure either. All you are really asking is the pressure in this question. You did not assign a CFR 55.43.

Because this was a conceptual error with TMS procedure at GGNS, we credited them as edits for these errors since they were repetitive for several SRO only questions.

Licensee accepted NRC re-written question and Q78 is now Sat.

SAT WITH REV 4 SUBMITTIAL 79 H

3 43.1 C

N E

S 55.43b1 With the given material this becomes a borderline direct lookup question and would not be acceptable for inclusion on an initial exam. Also, licensee did not assign a CFR 55.43. Handout needed with this question. Reference given also provides information that might help answer RO question 8.

Because we have no 55.43b1 questions we recommend a question that focuses on 55.43b1 and licensee agreed and so NRC wrote new question and is now Sat. No reference is now needed.

SAT WITH REV 4 SUBMITTIAL 80 H

3 X

43.5 A

X N

E S

55.43b5 You did not assign 55.43XXXXX needs to be corrected Has cueing in second part of choices with information in the stem related to SP level Enter EP-2, if an actual choice, would be SRO only, however, you give them this in all four choices therefore not SRO only.

Change the stem of the question to All of the EP-3 actions have been performed or attempted to have been performed to raise Suppression Pool level. For clarity and in each distractor remove Enter EP-2 Licensee made corrections and is now Sat.

SAT WITH REV 4 SUBMITTIAL

ES-401 GG-2015-12 19 Form ES-401-9

_Rev 1 Page 19 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 81 H

3 X

43.4 C

N E

S 55.43b4 You did not assign 55.43XXXXX needs to be corrected Partial Cueing from stem to answer. MSL issues and failure of Group 1 isolation in stem and this is the answer.

HANDOUT with old version.

Licensee made repair to answer by removing words for cueing and handout not needed and question is now Sat.

SAT WITH REV 4 SUBMITTIAL 82 H

3 43.2 B

X M

E S

55.43b2 2014 NRC The last NRC exam field is meant to be what years NRC exam, not YES.I dont think you should give them all of the proposed references, otherwise it is a direct look up with LOD=1.

I do not believe that you have a correct answer for the question. In the explanation According to my calculations the correct answer should be Day 2 at 1500 hours0.0174 days 0.417 hours 0.00248 weeks 5.7075e-4 months . This is not an option but is what is described in the explanation section. The AC/DC weekly line up should not be given as a handout. You can modify the values to make them further outside the acceptable range like Baxter Wilson at 528 KV, 62 Hz and Port Gibson going to 122 KV. With the LCO portion deleted on the handout that means all information above the line correct?

Also I need to see the parent question to see how the question was modified to determine if it is a bank or modified question.

Sat with Revision Handout SAT WITH REV 4 SUBMITTIAL 83 H

3 43.4 A

N E

S 55.43b4 You did not assign a CFR 55.43 and it needs to be corrected Handout Done and is now Sat.

ES-401 GG-2015-12 20 Form ES-401-9

_Rev 1 Page 20 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only SAT WITH REV 4 SUBMITTIAL 84 H

3 X

43.2 B

N E

S 55.43b5 By providing the reference it makes distractor C and D not credible as there is only one action item for 3.1.4 and it has a 12 hour1.388889e-4 days 0.00333 hours 1.984127e-5 weeks 4.566e-6 months time. In addition the Alarm Response is not needed as you give in the stem the information that is requested in the alarm response instructions. The above the line information is RO level knowledge and it is the key to answer the question. Is the blank full core display necessary as an attachment? Operators should be able to create a grid evens on bottom and odds going vertical.

Distracter A not allowed-no action is not allowed as a choice or answer. See NUREG-1021 for specific details.

With the information that is being provided in the handout this becomes a direct lookup question. Handout not needed if make it basis question.

Recommend changing the question as to the basis for not allowing two rods with slow time to be adjacent to each other to make this an SRO level question. Handout needed with old version of this question.

Sat with Revision and no handout now.

SAT WITH REV 4 SUBMITTIAL 85 H

3 X

43.5 C

X X

N U

55.43b5 You did not assign 55.43XXXXX needs to be corrected The stem where is states ALL level indication Should be more specific to state that all reactor vessel level indication has failed upscale. Need the noun names for the attachments, Operators are not required to memorize the numerical names of procedures and on an NRC exam the noun name is required to be with the numerical name. In this case it lowers the level of difficulty to RO level knowledge. Based on the information in the stem and system knowledge of interlocks that are required to be bypassed is less than that required of an SRO given the procedure name.

Question does not meet the K/A in that the mitigation strategy is given in the stem where the K/A asks about knowledge of the mitigation strategy. There is no way we can allow the Tech Spec basis for 3.5.1 to be included in the handout. This is a

ES-401 GG-2015-12 21 Form ES-401-9

_Rev 1 Page 21 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only E

S Systems question on what interlocks you need to bypass for the given conditions to do what the question is telling you to do. (Could be a really good system question for an RO)

Cueing from stem to answer. Need to add only to answer so it looks like the other choices. Also knowing the major purpose of Attachments is RO knowledge. Knowing when to use them in deep EOP situations is SRO. Need to discuss this one.

Will be Sat with the slight modification I sent you SAT WITH REV 4 SUBMITTIAL 86 H

3 43.2 D

N S

55.43b5 SAT WITH REV 4 SUBMITTIAL 87 H

2 X

43.1 C

X X

N U

55.43b2 You did not assign 55.43XXXXX needs to be corrected By providing the Tech spec it is a systems knowledge and a direct look-up question.

Ro only knowledge is required to determine effect on the plant and the Tech spec is given so distractors A and D are not plausible the only way to get to the condition H is through condition D.1 which is if action of conditions A, B, or C are not met.

Not SRO Only level as this is just reading the tech spec with system knowledge of the RPS system.

Suggest a question with regards to a tech spec that is in place to ensure a safety limit is not exceeded. Like you have instrument failures such that the requirements of Condition C of LCO 3.3.1.1 is not met requiring you to restore the RPS trip capability within 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months . This RPS trip functions is designed to ensure that the Reactor safety limit for. List 4 safety limits from tech spec and back up the answer from the TS basis.

By doing this you do not need a reference (already too many) and it is a TS basis question so it meets the K/A and SRO only criteria.

KA mismatch because it does not involve a safety limit type LCO.

ES-401 GG-2015-12 22 Form ES-401-9

_Rev 1 Page 22 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

Sat with the revision Handout SAT WITH REV 4 SUBMITTIAL 88 H

3 43.5 B

X N

E S

55.43b2 This is RO knowledge-knowing Transient Mit strategy and who directs it is one thing but it is another thing to ask system knowledge. This is an error carried forward conceptual error and therefore was marked as E but need to fix. Since licensee cant write SRO question on this topic, resampled to system 2390002 SRV and pick A2 to keep outline consistent and submit new question.

55.41b5 New question is Sat SAT WITH REV 4 SUBMITTIAL 89 H

3 43.5 D

N E

E S

55.43b5 You did not assign 55.43XXXXX needs to be corrected In the distractors please provide the noun name and numerical name for the procedures.

D answer needs ONEP at end to be complete and to prevent negative cueing.

Question had several formatting errors Question was replaced due to being too similar to question 78 Handout SAT WITH REV 4 SUBMITTIAL 90 H

3 43.2 B

N E

U 55.43b5 This question has overlap with the Op test so need to change System and KA.

Licensee agreed to swap to SGTS 261000 and get A2 KA to fix.

55.43b2

ES-401 GG-2015-12 23 Form ES-401-9

_Rev 1 Page 23 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only S

Question submitted to replace was unsat. It did not contain the correct answer and was not credible with the distractors listed Modified the question to fix the correct answer and make it credible sent to GG on 11/5/2015 SAT WITH REV 4 SUBMITTIAL 91 H

3 43.5 A

N E

S 55.43b5 You did not assign 55.43XXXXX needs to be corrected Done and is now Sat.

SAT WITH REV 4 SUBMITTIAL 92 H

2 43.7 D

B E

S 55.43b7 Way too many references on the exam. Re write question to get to the basis portion without giving the references. Handout with old question. 55.43b2 old CFR link.

Also, we need a refueling question to get a 55.43b(7) so resampled 234000 from T2G2 and licensee is writing a new question on refueling to fix the CFR distribution for the SRO-only exam.

55.43b7 Changed from new to a Bank question no reference is required New question is Sat SAT WITH REV 4 SUBMITTIAL 93 H

3 43.1 B

N S

55.43b5 SAT WITH REV 4 SUBMITTIAL 94 H

3 43.2 C

N E

S 55.43b10 Typos on distracters A and D (should be on not of).

Done and is now Sat.

SAT WITH REV 4 SUBMITTIAL

ES-401 GG-2015-12 24 Form ES-401-9

_Rev 1 Page 24 of 26 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

/

units Back-ward Q=

K/A SRO Only 95 H

2 43.6 B

M E

S 55.43b1 You did not assign 55.43XXXXX needs to be corrected 2013 NRC Need the parent question to determine if it is modified or Bank question.

Done and is now Sat.

SAT WITH REV 4 SUBMITTIAL 96 F

2 43.3 A

N S

55.43b3 SAT WITH REV 4 SUBMITTIAL 97 F

3 43.3 D

N S

55.43b3 SAT WITH REV 4 SUBMITTIAL 98 A

F 3

43.4 A

B E

S 55.43b5 Dont need commas before only. Reference for this question is bizarre-5.4.3 batch release on page 7 states Error! Reference not found.

SAT WITH REV 4 SUBMITTIAL 99 F

2 43.5 B

B E

S 55.43b5 You did not assign 55.43XXXXX needs to be corrected Borderline too easy.

Pulled a new Generic KA and replaced with a new bank question New question was Sat SAT WITH REV 4 SUBMITTIAL 100 F

2 43.5 C

B S

55.43b5 SAT WITH REV 4 SUBMITTIAL