ML090700413

GG-2009-03 - Draft Written Exam Comments

ML090700413
Person / Time
Site:	Grand Gulf
Issue date:	03/11/2009
From:	Ryan Lantz Operations Branch IV
To:	Douet J Entergy Operations
References
50-416/09-301
Download: ML090700413 (8)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 GG-2009-03 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).

Page 1 of 8 OBDI 202 - IOLE Process

3. Psychometric Flaws

4. Job Content Flaws

5. Other Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

6.

B/M/N

7.

U/E/S

8.

Explanation 1

F 2

N S

Edited stem for clarity 2

F 2

N S

3 F

3 N

S 4

H 2

N S

5 H

3 E

N E

S Need to status the Recirc Pumps in the stem since the cooldown is initiated while attempting to restart the pumps and prevent thermal stratification (steps 3.9.4 and 3.9.5).

The stem was revised to include the status of the recirc pumps.

6 F

2 N

S Edited stem for clarity.

7 F

2 N

S 8

H 3

N S

ES-401 2

Form ES-401-9 GG-2009-03 Page 2 of 8 OBDI 202 - IOLE Process

3. Psychometric Flaws

4. Job Content Flaws

5. Other Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

6.

B/M/N

7.

U/E/S

8.

Explanation 9

F 3

N S

Edited stem for clarity 10 F

3 U

N U

S KA mismatch. Does not address focus on refuel accidents.

The stem was revised to reinforce the applicability to refueling accidents.

11 H

3 N

S 12 H

3 N

S Question was replaced after audit exam.

13 F

2 N

S 14 F

2 N

S 15 H

2 U

N U

S KA mismatch. The only item needed to answer this question is knowledge of the design maximum pressure of containment.

This question was rewritten.

16 F

3 N

S Edited stem for clarity 17 H

3 U

U N

U S

Since there is no flow/head correlation curve given, the applicant could default to zero gpm. The elicited response needs to focus on knowledge of the effects on LCPS. As written, the applicant could incorrectly decide that LPCS has not started and get the answer correct. Need to change answers to pump status and (discharge valve or flow status).

This question was revised to address this comment.

18 H

3 E

N E

S As written, there could be no correct answer. The current pressure is within the nominal control band. Level is currently passing through the upper portion of the ATWS control band, so I would expect no reductions in pressure to be made.

This question was revised.

19 F

2 N

S 20 F

2 N

S 21 H

3 N

S 22 H

2 U

N U

S KA mismatch. Low water level (ie EOP entry condition) does not exist. Lots of examples in EOP for this KA. (Fuel zone level/temperature effects etc.)

This question was revised.

23 H

2 N

S 24 F

2 U

N U

S KA mismatch. Does not address increased cooling all coolers; containment spray.

This question is ok as written because the reason for initiating containment sprays is

ES-401 3

Form ES-401-9 GG-2009-03 Page 3 of 8 OBDI 202 - IOLE Process

3. Psychometric Flaws

4. Job Content Flaws

5. Other Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

6.

B/M/N

7.

U/E/S

8.

Explanation to maintain containment temperature below the design temperature limits which is the same reason for increasing containment cooling per the EOPs.

25 F

3 N

S 26 H

2 U

N U

S KA mismatch. This is a system level question, not E/APE Loss of CRD pumps. Right now, the question centers on both CRD pumps running and the effects this will have on the CRD system.

This question is ok since rod drifts will result in reactivity control issues as defined by the KA.

27 H

3 N

S This question was rewritten to address the comment on question 95.

28 H

2 N

S 29 F

2 N

S 30 H

2 U

N U

S Answer A is a give-away. The answer is modified with never before used, and cues in on an infrequent procedure.

This question was replaced.

31 F

2 N

E S

Recommend rewriting the stem as indicated on the markup.

This question is ok as is.

32 H

3 N

S 33 F

3 N

S 34 F

2 N

S 35 H

3 N

S 36 F

3 N

S 37 F

3 N

S 38 H

2 N

S Minor edits of answers completed.

39 H

3 N

S 40 H

3 N

S 41 F

2 N

S Minor edit to stem completed.

42 H

3 N

S 43 H

3 N

S

ES-401 4

Form ES-401-9 GG-2009-03 Page 4 of 8 OBDI 202 - IOLE Process

3. Psychometric Flaws

4. Job Content Flaws

5. Other Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

6.

B/M/N

7.

U/E/S

8.

Explanation 44 H

3 N

S 45 H

4 U

N U

S As written, this parallels K4.05, paralleling interlock design features. Need to focus on fundamentals of paralleling 2 AC sources.

This question was rewritten.

46 H

2 N

S 47 F

2 N

S 48 F

2 N

E S

See markup.

This question is correct as written.

49 F

3 N

U S

Verify this answer in the references. Explanation does not support the given answer.

The explanations were revised to address this comment.

50 H

2 N

S 51 F

2 N

E S

See markup.

Comment was incorporated.

52 H

3 N

E See markup.

Comment was incorporated.

53 F

2 N

S 54 F

2 N

E S

Add ONLY to the stem.

This question is OK as written.

55 F

2 N

S 56 H

3 N

S 57 H

3 N

S 58 H

2 N

S 59 F

2 N

S 60 H

3 N

S 61 F

3 U

N U

S This is a K2 question as written. KA mismatch.

This question is OK as written. The KA requires the question focus on the physical connections between the main turbine control system and DC power and the question tests on this.

ES-401 5

Form ES-401-9 GG-2009-03 Page 5 of 8 OBDI 202 - IOLE Process

3. Psychometric Flaws

4. Job Content Flaws

5. Other Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

6.

B/M/N

7.

U/E/S

8.

Explanation 62 H

3 U

N UE S

Both C and D are correct. Change D to The suction valve closing after 15 seconds.

Distractor C was rewritten.

63 H

3 N

S 64 H

3 N

S 65 F

3 N

S 66 F

2 N

S 67 H

3 N

S 68 F

2 N

S 69 F

3 N

S 70 F

3 N

S 71 H

3 N

S 72 F

2 N

E S

Need to add maximum to stem, or verify that the distractor values are not below the maximum values for the respective conditions.

This comment has been incorporated.

73 F

2 N

S 74 H

2 N

S 75 H

2 N

S 76 H

2 N

S 77 H

3 N

S 78 H

4 N

S 79 H

2 U

N U

S KA mismatch. Does not address reactor pressure.

This question was replaced.

80 F

2 U

N U

S RO knowledge. The applicant only needs to know how ED is performed.

This question is ok as written because it tests on procedure selection and a choice of methods of depressurizing. Both options are procedurally allowed but only one is correct in this situation.

81 H

3 N

E Same knowledge requirement as Q78 - when level cant be determined, enter EP-5

ES-401 6

Form ES-401-9 GG-2009-03 Page 6 of 8 OBDI 202 - IOLE Process

3. Psychometric Flaws

4. Job Content Flaws

5. Other Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

6.

B/M/N

7.

U/E/S

8.

Explanation S

This question was rewritten.

82 F

2 U

N U

S Series of true/false statements.

This question is ok as written - the distractors are not stand alone true/false statements.

83 F

2 N

S 84 F

3 U

N U

S This does not clearly address the reactor pressure in the KA Revised 85 H

4 N

S Removed the hours after the 0700 86 H

2 N

S 87 H

2 N

E S

Do not supply reference since all actions are <= 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />.

This question was modified. The reference is needed as not all the distractors have actions that are one hour or less. This question is testing on selecting the correct action statement.

88 H

2 N

E S

Remove all of the hours in this question.

This comment was incorporated.

89 H

3 N

S 90 H

2 N

S 91 F

2 N

S 92 H

3 N

S 93 F

3 N

S 94 F

2 N

S 95 F

3 N

U S

This question is the same as one of the RO questions ->enter SAPS based on H2 Rewrote question 27 on the RO exam.

96 F

2 N

S 97 F

2 N

E S

All of the explanations are incorrect. This is borderline RO since MCPR is a 55.41(14) knowledge requirement.

The explanations have been corrected. This is an SRO question because it tests on the specific tech spec bases for this safety limit.

ES-401 7

Form ES-401-9 GG-2009-03 Page 7 of 8 OBDI 202 - IOLE Process

3. Psychometric Flaws

4. Job Content Flaws

5. Other Q#

1.

LOK (F/H)

2.

LOD (1-5)

Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

6.

B/M/N

7.

U/E/S

8.

Explanation 98 H

2 U

N U

S RO knowledge. SRO should have to evaluate something below the line in TS after choosing the correct TS.

This question is ok as written.

99 F

2 N

S 100 F

2 N

S Question was revised after validation.

B=

F= 36 E=

M=

H= 39 U=

RO TOTALS:

N=

Additional Notes:

B=

F=

E=

M=

H=

U=

SRO TOTALS:

N=

Additional Notes:

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.

2.

Chief Examiner comments are indicated in blue.

3.

Average difficulty is __ ___ on the RO exam and ___ on the SRO exam.

4.

The 10CFR55.41/43 distribution is: RO / SRO 41.1 =

43.1 =

41.2 =

43.2 =

41.3 =

43.3 =

41.4 =

43.4 =

41.5 =

43.5 =

41.6 =

43.6 =

41.7 =

43.7 =

41.8 =

41.9 =

41.10 =

41.11 =

41.12 =

41.13 =

41.14 =

ES-401 8

Form ES-401-9 GG-2009-03 Page 8 of 8 OBDI 202 - IOLE Process

5.

The answer distribution is: RO / SRO A = 13 (17%)

/

6 (24%)

B = 16 (22%)

/

7 (28%)

C = 21 (28%)

/

5 (20%)

D = 25 (33%)

/

7 (28%)

6.

There are questions with attachments provided.