ML16204A022

2016 Prairie Island Nuclear Generating Plant Initial License Examination Form ES-401-9

ML16204A022
Person / Time
Site:	Prairie Island
Issue date:	06/10/2016
From:	Mcneil D NRC/RGN-III/DRS/OLB
To:	Northern States Power Company, Minnesota
Shared Package
ML15274A272	List: IR 05000282/2016301 ML16111A449 ML16111A488 ML16204A022 ML16204A039 ML16204A050 ML16204A054 ML16204A100 ML16204A102 ML16246A035
References
50-282/OL-16-301, 50-306/OL-16-301
Download: ML16204A022 (6)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. questions reviewed in depth.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

1. 1 H

2 N

S 2

H 2

N S

1. 3 H

2 N

S 4

H 2

B S

1. 5 F

2 N

S 6

H 2

N S

1. 7 H

2 B

S 8

H 2

N S

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved K/A, and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6.

Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7.

Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8.

At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2

Form ES-401-9 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 9

F 1

N U

LOD = 1 Replaced Question 10 H

2 M

S

1. 11 M

2 B

S 12 M

2 N

S

1. 13 M

2 N

S 14 H

2 B

S 15 F

2 N

S 16 F

2 N

S

1. 17 F

2 N

E One implausible distractor. Establishing RCS bleed and feed wont involve Aux Feed Replaced distractor.

18 F

2 N

S

1. 19 H

2 N

S 20 H

2.5 N

S 21 H

2 M

E Bullet #4: No operator actions have been taken. Moved bullet to better location.

22 H

2 N

S

1. 23 H

2 N

S 24 H

2.5 N

S 25 F

2 N

S 26 H

2 N

S 27 H

2.5 N

S 28 F

2 N

S

1. 29 H

2 N

S 30 H

2 N

S

1. 31 F

2 B

S

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 32 F

2 N

S 33 F

1 B

U LOD=1 Reworked question.

34 F

2 N

S 35 H

2 N

S 36 H

2 N

S

1. 37 H

1 B

U E LOD=1. I was able to guess the correct answer without looking at OPT: Modified one distractor and made question satisfactory.

38 H

1 N

U LOD-1. Doesnt completely test the K/A. (2X2) Reworked distractors - matches K/A.

39 F

2 N

S 40 H

2 N

S

1. 41 H

2 N

S 42 H

2 N

S

1. 43 H

2 N

E Not fundamental. Requires applicant to determine a correct response to an error.

44 F

2 N

U S QK/A. The Q does not accurately test the K/A. In order to use the procedure to correct the condition, the impact has to be known.

Q does match the K/A.

45 H

2 N

S 46 F

1 N

U LOD=1. Reworked question.

1. 47 H

2 N

S 48 H

2 N

S

1. 49 F

2 B

S 50 F

2 N

S

1. 51 H

2 N

S 52 F

2 B

S

1. 53 F

2 N

S 54 F

2 B

S

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 55 H

2 B

S 56 H

2 N

S

1. 57 H

2 N

S 58 F

2 N

S

1. 59 H

2 N

S 60 H

2 N

S

1. 61 F

2 N

S 62 F

2 N

S 63 F

1 N

U LOD=1. Reworked question.

64 H

2 M

U QK/A Reworked question.

65 F

1 N

U S LOD=1. Discussed w/ licensee, Q is LOD = 2.

66 F

1 N

U LOD=1. Replaced K/A, wrote new question.

1. 67 F

1 N

U S LOD=1 Discussed question - LOD = 2.

68 H

2 N

S 69 H

2 N

U Two distractors contain Digital Rod Control IN positions when the control rods have to be withdrawn. Implausible distractors.

Reworked distractors.

70 H

2 N

S

1. 71 F

2 N

E Do we have any immediately action statements in section 3? If not, change to some time limit. Suggest going to a (1) ; (2) format. Many Tech Spec Immediately actions. Fixed format.

72 F

2 N

U Distractors C & D are implausible. The picture shows that valve CV-31415 is associated with B SG Blowdown, not SG 11. Replaced question.

1. 73 F

2 B

E Distractor c states the purpose is to minimize the cooldown capability of the non-faulted S/G. Why would we want to do that? Appears to be implausible. See if we can replace. Replaced distractor -

changed one initial condition.

74 F

2 N

S 75 H

1 N

U LOD=1 Reworked question.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only

1. 76 H

2.5 B

S 77 H

1 N

U LOD=1. Direct look-up. Replaced question.

78 H

1 M

U LOD=1. Direct look-up. Replaced question.

79#

H 2

N S

80 H

2 N

S 81 H

2.5 N

S 82 H

2 M

S LOD=1 or 2 implausible distractors - procedure is a give-away.

Procedure is not a give-away. Changing bullet one changes correct procedure. LOD = 2.

1. 83 H

2 N

S 84 F

2 N

S 85 H

2 N

E Add the rad monitor annunciators to comply with the K/A Statement.

Added rad monitors.

86 H

2 B

U S LOD = 1. Discussed with licensee, determined the question was LOD = 2.

1. 87 H

2 N

S 88 H

2 N

S 89 H

2.5 B

S 90 H

2 N

S

1. 91 H

2 M

S Verify this is not a direct look-up. Not a direct look-up.

92 H

2 N

U S Q does not address a portion of the K/A. Discussed question -

determined question meets the K/A.

93 H

2 B

S Modify question to more fully address the K/A.

94 H

2 B

S 95 F

1 N

U S LOD=1. After discussion with licensee, determined LOD = 2 96 H

2 N

S

1. 97 F

2 B

S 98 H

2 N

S

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 99 H

2 N

S What is the correct answer? The Answer key says one distractor, but the word description for justifying the distractor states another distractor is the correct answer. Explanation corrected.

100 H

2 N

S