ML14017A038

2013-11-Draft Written Exam Comments

ML14017A038
Person / Time
Site:	Grand Gulf
Issue date:	11/15/2013
From:	Garchow S Operations Branch IV
To:	Entergy Operations
laura hurley
References
ES-401, ES-401-9
Download: ML14017A038 (8)

Text

ES-401 GG-2013-11 Written Examination Review Worksheet Form ES-401-9 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).

Page 1 of 8 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 1

H 3

N S

A, CR, 41.5 2

H 3

B S

D, CR, 41.10 3

F 3

E B

E S

B, CR, 41.8 - Distractor A is also correct.

The stem was revised 4

F 3

E N

E S

B, CR, 41.10 - Distractors C and D are also correct.

The stem was revised to make only one answer correct.

5 F

2 N

S B, CR, 41.2 6

H 3

N S

C, CR, 41.7 7

H 3

B S

B, CR, 41.7 8

H 4

N S

B, CR, 41.7 - Edit; Delete wrong from last line of explanation.

ES-401 GG-2013-11 2

Form ES-401-9 Page 2 of 8 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 9

H 3

N S

C, CR, 41.3 10 F

4 B

S C, CR, 41.11 11 F

3 B

S A, CR, 41.9, NRC 2012 12 H

3 E

N E

S C, CR, 41.10 - Not sure distractor D could not be argued to also be correct. I.e., how would the candidate know the safety limit had not been exceeded from the information provided?

Distractor D was revised.

13 F

3 E

N E

S B, CR, 41.10 - Distractor C could also be argued as correct (the TS entry condition is met and before SP reaches 110 a manual scram is required). Also, entering the TS is not, in and of itself, the reason for the scram - the reason for the scram is in the TS basis document for LCO 3.6.2.

This question was revised.

14 F

3 B

S A, CR, 41.10 15 F

3 B

S A, CR, 41.9, NRC 2012 16 F

3 N

S B, CR, 41.10 17 H

3 N

S D, CR, 41.7 18 F

2 B

S C, CR, 41.7 19 F

3 U

B U

S D, CR, 41.10 - This is a KA mismatch because it does not address the reason for the response.

This question was revised.

20 H

3 E

B E

S D, CR, 41.4 - Distractor C could also be correct depending if you start off with a leading or lagging power factor.

This question was revised.

21 H

2 B

S B, CR, 41.3 22 H

3 E

N E

S A, CR, 41.7 - Distractor B could also be correct since a level of <-41.6 could be interpreted as less than level 1. Easy fix is to change <-41.6 to -50 inches.

This question was revised.

23 F

3 N

S C, CR, 41.10 24 H

3 U

B U

S A, CR, 41.6 - This is a KA mismatch because it doesnt have anything to do with RC&IS.

ES-401 GG-2013-11 3

Form ES-401-9 Page 3 of 8 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only This question was revised.

25 H

4 N

S D, CR, 41.10 - Edit; align the margins in the distractors.

Done.

26 H

3 N

S D, CR, 41.11 27 H

4 N

S B, CR, 41.11 28 H

2 B

S C, CR, 41.7, NRC 2012 29 H

3 N

S B, CR, 41.7 30 H

3 N

S D, CR, 41.4 31 F

3 U

N U

S D, CR, 41.7 - This is a KA mismatch because the question does not involve a loss or malfunction of the LPCS system.

This question was revised.

32 H

4 N

S B, CR, 41.7 - Edit; Need to clarify the stem to make it clear the loss of 15AA follows the other initial conditions.

Clarification was made.

33 F

3 U

N U

S D, CR, 41.10 - The distractors are a collection of true false statements.

This question was revised.

34 F

3 N

S A, CR, 41.6 35 H

3 E

N E

S D, CR, 41.6 - If I missed this question, I would argue there are no correct answers.

Distractors A and C are incorrect because RPS A remains de-energized. Distractors B and D are incorrect because both the normal and alternate supplies to RPS are available. I think the power supplies are or are not available to RPS regardless of the status of the EPAs.

This question was revised.

36 H

2 B

S C, CR, 41.6 37 F

2 N

S D, CR, 41.6 38 F

2 E

B E

S B, CR, 41.5 - Change the stem to read A normal reactor startup is in to preclude someone thinking the SRMs could be in single coincidence logic (making D also correct).

This question was revised.

39 H

2 E

E N

E C, CR, 41.6 - It is not credible that there would be an upscale alarm with power

ES-401 GG-2013-11 4

Form ES-401-9 Page 4 of 8 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only S

indicated less than 5%. Change distractors B and C so that they are limited to APRM channel 1. What is Mode 2 scram setpoint? CAF.

Changes were made. Distractor B is credible since a downscale could be misinterpreted as an INOP alarm.

40 H

3 B

S C, CR, 41.7 41 H

3 B

S A, CR, 41.7 42 H

3 B

S D, CR, 41.4 43 F

3 B

S A, CR, 41.7 44 F

3 N

S C, CR, 41.4 45 F

3 B

S C, CR, 41.11 46 F

3 N

S A, CR, 41.4 47 F

2 U

B U

S C, CR, 41.7 - This is a KA mismatch because it does not involve paralleling or synchronizing AC power sources.

This question was revised.

48 F

3 N

S A, CR, 41.7 49 H

3 N

S D, CR, 41.10 50 F

3 B

S B, CR, 41.4 - Edit; Distractor D is always true (even in this case with no control power and the breaker wont trip). Recommend changing D to read, will still have automatic trip capability.

This question was revised.

51 F

3 N

S B, CR, 41.7 - CAF; Is RHR/SWS B off of Div III?

52 F

3 N

S A, CR, 41.4 53 F

3 B

S C, CR, 41.7 54 F

3 N

S C, CR, 41.6 55 F

3 N

S A, CR, 41.3 56 F

3 N

S B, CR, 41.4 57 H

3 B

S A, CR, 41.2 58 F

3 B

S C, CR. 41.9

ES-401 GG-2013-11 5

Form ES-401-9 Page 5 of 8 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 59 H

3 N

S D, CR, 41.10 - Edit; Change last line of the stem to read procedure(s)

This question was revised.

60 F

3 B

S A, CR, 41.4 61 H

2 B

S C, CR, 41.4 62 H

4 B

S C, CR, 41.4 63 H

4 N

S B, CR, 41.13 64 H

4 N

S A, CR, 41.4 - Edit; Is the second line of the stem correctly written (i.e., pressure going down and rising)?

This question was revised.

65 H

3 E

B E

S B, CR, 41.11 - Distractor A is a true statement; there are two correct answers.

This question was revised.

66 F

2 B

S B, CR, 41.10 67 F

3 B

S B, CR, 41.10 68 F

3 E

N S

A, CR, 41.10 - Recommend changing distractor B to read when in Mode 5 instead of when fuel is in the reactor.

This question was revised.

69 F

3 B

S D, CR, 41.10 - CAF; is distractor C the same as de-coupling?

70 H

3 E

B E

S A, OR, 41.10 - Good question, but there is a lot of potential cueing between this question and question 50. CAF. Recommend changing one or the other.

The question was replaced.

71 H

2 B

S D, CR, 41.10 72 F

2 B

S B, CR, 41.12 73 H

2 B

S B, CR, 41.12 74 H

2 N

S D, CR, 41.10 75 H

3 N

S D, CR, 41.10 76 H

4 N

S A, OR, 43.2 77 F

3 N

S A, CR, 43.2

ES-401 GG-2013-11 6

Form ES-401-9 Page 6 of 8 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 78 H

2 U

N U

S A, CR, 41.10 - This is an RO question since it can be answered with only knowing the required action (i.e. you dont have to know what procedure to enter to get the correct answer).

This question was revised.

79 F

3 N

S C, CR, 43.2 80 H

3 N

S C, CR, 43.5 81 H

2 N

S D, OR, 43.5 82 H

3 N

S B, OR, 43.2 83 H

2 N

S D, OR, 43.2 84 H

3 E

N E

S C, OR, 43.2 - Change what is the tech spec completion time to What is the maximum tech spec completion time.

This question was revised.

85 H

3 E

N E

S B, CR, 43.2 - We need to talk about this one.

Exam author will clarify the stem.

This question was revised.

86 H

3 N

S A - CR, 43.5 - Edit; Should the PSP indicator be red instead of green on the EOP Overview screen?

No it is correct as displayed.

87 H

3 N

S C, CR, 43.5 88 F

2 N

S B, CR, 43.2 89 H

3 N

S B, CR, 43.5 90 H

3 N

S C, OR, 43.2 91 F

3 N

S A, CR, 43.2 92 H

3 N

S D, CR, 43.5 93 H

4 N

S A, CR, 43.5 94 H

3 U

B U

S D, CR, 43.5 - This is a KA mismatch because it is a Tier 3 question and it needs to be generic in nature and this question is a specific scenario.

The generic concept is to exit all EOPs upon entry into the SAPs and the question is ok as written.

ES-401 GG-2013-11 7

Form ES-401-9 Page 7 of 8 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 95 F

2 N

S C, CR, 43.6 96 H

2 N

S B, OR, 43.2 97 F

3 N

S B, CR, 43.2 98 H

3 N

S D, CR, 43.4 99 F

3 E

N E

S A, CR, 43.2 - I dont expect candidates to know the specific TRM chemistry limits for a 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> LCO from memory.

Will provide the TRM actions but not the limits.

100 H

3 N

S A, CR, 43.2 RO TOTALS:

B= 36 F= 37 (49%)

E= 11 Additional Notes:

M= 0 H= 38 (51%)

U= 4 N= 39 SRO TOTALS:

B= 1 F= 7 (28%)

E= 2 Additional Notes:

M= 1 H= 18 (72%)

U= 3 N= 23

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N

2.

Chief Examiner resolution comments are indicated in blue.

3.

Average difficulty is _2.91 on the RO exam and 2.92 on the SRO exam.

4.

The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 0 41.2 = 0 43.2 = 14 41.3 = 3 43.3 = 0 41.4 = 13 43.4 = 1 41.5 = 3 43.5 = 8 41.6 = 5 43.6 = 1 41.7 = 18 43.7 = 0 41.8 = 2 41.9 = 3 41.10 = 21

ES-401 GG-2013-11 8

Form ES-401-9 Page 8 of 8 OBDI 202 - IOLE Process 41.11 = 5 41.12 = 2 41.13 = 1 41.14 = 0

5.

The answer distribution is: RO / SRO A = 18

/ 8 B = 21

/ 6 C = 18

/ 6 D = 18

/ 5

6.

There are 1 / 7 questions with attachments provided.