ML14242A026

From kanterella
Jump to navigation Jump to search
2013-03 Draft Written Exam Comments
ML14242A026
Person / Time
Site: Callaway Ameren icon.png
Issue date: 03/18/2013
From: Vincent Gaddy
Operations Branch IV
To:
Union Electric Co
References
Download: ML14242A026 (7)


Text

ES-401 CW-2013-03 Written Examination Review Worksheet Form ES-401-9 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).

Page 1 of 7 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws
4. Job Content Flaws
5. Other
6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

  1. /

units Back-ward Q=

K/A SRO Only 1

F 2

X N

E A, 41.10. Does CW FSAR differentiate between Leak and SBLOCA? If so, would need to alter stem for consistency with the explanation for Distracter A. Change to SB LOCA.

2 H

2 B

E A, 41.10, OR.

3 H

3 X

N E

B, 41.8. Distracter D is effectively All of the Above, which shouldnt be used.

Revised 4

H 3

X B

E C, 41.2. Does Distracter A mean that heat transfer is via forced circ? Change A to forced circ + condenser +10% atm steam dumps. Add only to the end of B and C.

Done 5

H 3

N E

B, 41.10.

6 H

3 N

S C, 41.5 /.7 7

H 3

B E

D, 41.7. Setpoints required? Removed

ES-401 CW-2013-03 2

Form ES-401-9 Page 2 of 7 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws
4. Job Content Flaws
5. Other
6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

  1. /

units Back-ward Q=

K/A SRO Only 8

F 2

X N

E B, 41.5,.10. Is D plausible? Revised 9

H 3

B E

A, 41.10. Consider making one of the distracters Unidentified Leakage < 1gpm, so that all 3 incorrect answers arent the same category (Identified Leakage < 10 gpm).

Revised to unidentified leakage.

10 H

3 B

E D, 41.10.

11 F

3 N

S A, 41.7.

12 F

3 N

S B, 41.7.

13 H

3 N

S A, 41.7 /.10.

14 H

3 B

E B, 41.7. Does NN01 have an alternate supply? Yes 15 H

3 N

E A, 41.10. Add in accordance with E-1? to the beginning of the stem question.

Done 16 H

4 X

N U

S A, 41.10. K/A M/M. Tests on use of figures, not components. Replaced question 17 H

4 B

E B, 41.10.

18 H

2 B

S B, 41.10. OR.

19 H

2 N

E A, 41.1 20 H

4 N

E C, 41.10. Knowledge of FSAR BAST limit from memory? Yes. Delete first sentence in stem, re-order distractors.

21 H

2 X

N U

S D, 41.7. K/A requires applicant to determine the occurrence of a fuel handling incident, but that is given to the applicant in the stem. Deleted 3rd bullet, revised stem.

22 H

3 X

N U

S D, 41.10. B and C are not credible. For B, keeping RCS pressure lower puts it closer to saturation. For C, keeping RCS pressure lower than SG ensures backfill. Revised B and C 23 H

4 N

E B, 41.10.

24 H

3 N

S B, 41.10.

25 H

2 N

E A, 41.7. Borderline LOD= 1. Revised

ES-401 CW-2013-03 3

Form ES-401-9 Page 3 of 7 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws
4. Job Content Flaws
5. Other
6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

  1. /

units Back-ward Q=

K/A SRO Only 26 F

3 X

B N

U S

D, 41.10. Does not test on ability to operate or monitor components or functions of control and safety systems. Rewrote question.

27 H

3 N

E A, 41.10. Add /hr after each cooldown rate. Add IAW ES-0.2 to stem.

28 H

2 X

B E

D, 41.2. Add amplification to stem that we are looking for the final state, since initially both MFRV and MFBV are controlling.

29 F

3 N

E C, 41.7.

30 H

3 B

E C, 41.7. Specify NCP discharge on distracters A and D.

31 F

3 B

S B, 41.7.

32 H

2 M

E C, 41.8. Copy of original Bank question? Added 33 H

3 B

E A, 41.3.

34 H

4 B

E D, 41.10 35 H

3 B

S C, 41.7 36 H

2 N

S B, 41.7 37 F

2 B

S D, 41.7 38 F

H 1

3 X

N U

S C, 41.10 LOD=1 Rewrote 39 F

2 X

N U

S A, 41.7 LOD = 1 Modified stem 40 F

2 X

N E

C, 41.10, Recommend in stem replacing more limiting values with adverse containment values.

41 F

H 2

4 X

B N

U S

D, 41.7, Will Distracter A valves receive OPEN signal if they are initially in a closed position? Distracter B not plausible. Wrote new question 42 F

3 N

S D, 41.10 43 H

3 N

E A, 41.4 44 H

3 B

S B, 41.1 45 H

2 B

E C, 41.4, Change stem to Which SGs are being supplied by AFW?

ES-401 CW-2013-03 4

Form ES-401-9 Page 4 of 7 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws
4. Job Content Flaws
5. Other
6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

  1. /

units Back-ward Q=

K/A SRO Only 46 H

3 X

B U

S D, 41.5, Need to explain what makes Distracters A B C credible. Otherwise LOD=1.

Rewrote question.

47 F

3 N

S C, 41.7 48 F

4 X

N S

B, 41.8. Verify that you expect your operators to have DG Surveillance parameters memorized. On OJT qual card.

49 F

3 B

S D, 41.8, 50 H

4 N

S A, 41.8 51 H

2 N

E D, 41.7, maybe a better question is which alarm will have the highest priority?

52 H

2 N

E D, 41.11. Isnt B also correct given the explanation for D? Enhanced explanation for D 53 H

3 N

S D, 41.8, 54 H

3 B

S C, 41.4 55 F

3 N

S D, 41.12 56 H

3 X

B U

S B, 41.10, If the reference procedure is used, cant get to initial conditions in stem.

Step 1.b RNO isolates letdown. Revised question 57 H

3 N

S B, 41.8, 58 H

2 B

S B, 41.6 59 H

2 B

S A, 41.4 60 H

2 B

S D, 41.7 61 H

4 N

E A, 41.10 62 H

2 B

S C, 41.5 63 H

3 N

E A, 41.4, Change temperatures to 557 on B and C response.

64 H

3 M

S C, 41.4 65 H

3 N

S D, 41.4 66 F

2 N

S C, 41.10

ES-401 CW-2013-03 5

Form ES-401-9 Page 5 of 7 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws
4. Job Content Flaws
5. Other
6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

  1. /

units Back-ward Q=

K/A SRO Only 67 F

2 X

N B

U S

C, OR, 41.10, Given the reference, this appears to be a direct lookup. Only knowledge needed is voltage. Replaced KA and question 68 F

4 N

S A, 41.10 69 F

1 2

X N

B U

S D, 41.10, Well known surv requirement. LOD=1. Replaced 70 F

2 B

S C, 41.10 71 F

2 N

S C, 41.10 72 F

3 B

E C, 41.12, Suggest one answer be 1500 mRem, Best distracter. Eliminate D (4500 mRem) Revised 73 F

3 N

S D, 41.12 74 F

1 2

X B

U S

C, 41.10, Answer given in stem. Functional Restoration Guidelines are FRGs Replaced question 75 F

2 B

S A, 41.10 76 H

2 B

E A, 43.5, As written, I can easily eliminate response C and D and only need to decide the correct procedure. Swap first part of response (1) between B and C. B will now read (1)hot leg break and C will be (1) Vapor Space Break. Fixed 77 H

3 N

E B, 43.5 78 H

3 N

S C, 43.5 79 H

3 N

S D, 43.2 80 H

3 B

E B, 43.5, Change to operate both containment spray pumps 81 H

3 N

S C, 43.2 82 H

3 X

N U

S B, 43.2. What automatic safety function did not take place? C11 was a control function and appears to be removed from CW by CMP 95-1008. Replaced KA 83 F

2 N

E D, 43.5 Make question gender neutral (he/she) 84 H

3 B

S A, OR, 43.5 85 H

3 B

E D, 43.5

ES-401 CW-2013-03 6

Form ES-401-9 Page 6 of 7 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws
4. Job Content Flaws
5. Other
6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

  1. /

units Back-ward Q=

K/A SRO Only 86 H

3 N

S C, 43.1 & 5 87 H

3 B

E D, 43.2 C explanation needs cleaning up. Change D When only 1 AFW train is restored, be in Hot Standby in the following 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />. Done 88 H

3 N

E C, 43.5 89 H

3 B

E C, 43.5 Change B.1 and C.1 distracters to Lube oil temperature will be low This is better distracter. Done 90 H

3 N

E B, 43.5 The procedure reference is for the Containment Isolation CCW Bypass valves. Correct all CRS Actions for these valves to say bypass valves. Done 91 F

2 X

B U

S D, 43.7, Is the EDO a specific person, or a position that might be filled by a SRO?

See explanation B. Changed B and C to Director Nuclear Ops 92 H

3 N

E A, 43.5, Change (1) to say, what direction is provided to mitigate the above given conditions? Done 93 H

3 B

E B, 43.5 94 F

2 N

E A, 43.1 For C1 and D1, change to read due to the declaration of an emergency Done 95 F

3 N

E A, 43.5 96 F

3 N

E D, 43.5 Match capitalization in answers to the procedure.

97 F

3 N

S D, 43.5 98 F

2 N

E D, 43.4 99 H

3 X

N U

S A, 43.5 Question is not generic. See ES-401 D.2.a, last sentence of first paragraph.

Rewrote question 100 F

3 N

E D, 43.5 Can applicant argue that Probable and Imminent are the same for answer C? No For D - capitalize CODE RED as in procedure. Done RO TOTALS:

B= 31 (41%)

F= 25 (33%)

E= 31 (41%)

Additional Notes:

M= 2 (3%)

H= 50 (67%)

U= 12 (16%)

N= 42 (56%)

S= 32 (43%)

ES-401 CW-2013-03 7

Form ES-401-9 Page 7 of 7 OBDI 202 - IOLE Process SRO TOTALS:

B= 8 (32%)

F= 8 (32%)

E= 16 (64%)

Additional Notes:

M= 0 H= 17 (68%)

U= 3 (12%)

N= 17 (68%)

S= 6 (24%)

GENERAL COMMENT

S:

1. Chief Examiner comments are indicated in red.
2.

Average difficulty is 2.77 on the RO exam and 3.04 on the SRO exam.