ML051040429

Init Exam - 02/2005 - Draft Written Exam Comments

ML051040429
Person / Time
Site:	Diablo Canyon
Issue date:	10/25/2004
From:	Rueger G Pacific Gas & Electric Co
To:	NRC Region 4
References
50-275/05-301, 50-323/05-301
Download: ML051040429 (6)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 S 2 H 3 S 3 H 3 E KA refers to a Large Break LOCA. Answer explaination identifies a small break LOCA 4 H 2 S 5 H 3 U Direct look up from references. Stem points directly to applicable reference.

6 F 2 S 7 F 2 S 8 H 3 S Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 9 H 3 S 10 F 2 S 11 F 2 E GFES level as asked, need to rework.

12 F 2 U KA is for loss of bus, not restoration. Need to rework.

13 F 2 U Not EOP related as per KA, need to rework 14 F 2 S 15 F 2 E Are distractors valid/creditable?

16 F 2 E low discriminatory value, need to rework 17 F 2 S 18 F 2 S 19 H 2 S 20 H 3 E low discriminatory value, need to rework 21 H 3 S 22 F 2 S 23 F 3 S 24 F 3 S 25 H 3 S 26 H 2 S 27 H 3 E low discriminatory value, need to rework, Rediagnosis?

28 F 2 S 29 F 3 S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 30 F 2 S 31 F 2 U Heat Exchanger not in question 32 H 3 S 33 H 2 S 34 H 3 S 35 H 3 E Direct look up from references. Stem points directly to applicable reference.

36 F 2 S 37 H 2 E Change PORV to 474 in distractor B. Typo?

38 F 2 S 39 F 2 S 40 H 2 S 41 F 3 E Direct look up from references. Stem points directly to applicable reference.

42 F 2 E Answer should also identify either loop suction valves closed 43 H 3 E Stem should also identify that rods are in manual control.

44 F 3 S 45 F 3 U not MFW related.

46 H 3 S 47 F 3 E/U 063 is DC power, not AC. Wrong KA?

48 H 2 E Direct look up from references. Stem points directly to applicable reference.

49 H 2 S 50 F 3 U This question matches K4.02. KA mismatch or typo?

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 51 F 2 S 52 F 2 S 53 H S 54 F 2 S 55 H 3 S 56 H 3 S 57 F 2 E/U Typo on KA?

58 H 3 S 59 F 2 U mix or vital and non vital is a cue, use of 4160V is not credible.

60 F 2 S 61 H 2 S 62 H 3 S 63 F 3 S 64 H 2 S 65 F 2 S 66 F 3 S 67 F 2 S 68 F 2 E proposed answer does not read correctly 69 F 2 S 70 H 2 S 71 F 2 E GFES level as asked, need to rework.

72 F 2 S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 73 H 2 S 74 F 2 S 75 F 3 S 76 F 3 S 77 H 3 S 78 H 3 S 79 F 2 U Not a 55.43.5 question, RO level.

80 H 3 S 81 H 3 S 82 F 2 U KA mismatch, distractors are not credible.

83 H 3 E Not sure if this addresses SI termination 84 H 3 S 85 F 3 S 86 H 2 U RO level question 87 H 2 U RO level question 88 H 3 U Does not reference procedure in question 89 H 3 U 90 H 3 U Direct look up from references. Stem points directly to applicable reference.

91 H 3 U RO level question 92 H 2 U KA mismatch 93 H 2 U KA mismatch

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 94 H 2 S Direct look up from references. Stem points directly to applicable reference.

95 F 3 S 96 F 2 S 97 F 3 S 98 H 3 E Direct look up from references. Stem points directly to applicable reference.

99 F 2 S 100 H 2 S