ML14119A394

NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)

ML14119A394
Person / Time
Site:	FitzPatrick
Issue date:	04/01/2014
From:	David Silk Operations Branch I
To:	FitzPatrick
Jackson D
Shared Package
ML13330B159	List: IR 05000333/2014301 ML14093A641 ML14119A390 ML14119A391 ML14119A392 ML14119A393 ML14119A394 ML14119A395 ML14119A396 ML14119A397 ML14141A187 ML14164A212
References
TAC U01887
Download: ML14119A394 (6)

Text

ES-401 FITZPATRICK Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 3 N s 2 F 3 N s 3 H 2 N s 4 F 2 N s 5 H 3 c M E Make C plausible. Change SRM C to 300 cps.

6 H 2 N s 7 F 2 N s F vs H 8 F 2 c N E Make C plausible. Use an actual EDG trip.

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] II

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy- difficult) rating scale (questions in the 2-4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

.. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) .

.. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) .

The answer choices are a collection of unrelated true/false statements .

. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable .

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) .

4.

..Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content) .

.. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) .

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) .

The question requires reverse logic or application compared to the job requirements .

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dis!. Link units ward KIA Only 9 F 3 N s 10 H 2 X B E Add "loss of' to "coolant accident."

11 H 2 N s 12 H 3 N s 13 H 2 X D B E Modify 2"ct bullet and change D.

14 H 2 N s 15 H 2 N s 16 H 3 X N E Add lAW procedure.

If u

II v f'J s 18 F 2 X N E Edit C&D- delete "If the light does illuminate" & replace with "and."

19 H 2 N s 20 F 2 c N E C not plausible because IRM readout is not cps.

21 F 2 N s 22 H 2 N s 23 F 3 N s 24 H 2 N s H vs F.

25 H 3 N s 26 H 3 X N E Balance the size of the dis tractors.

27 F 2 N s 28 F 2 B s

ES-401 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 29 H 3 B s 30 H 3 M s 31 H 2 N s 32 H 2 N s 33 F 3 N s F vs H.

34 H 3 N s 35 F 2 X N E 12MOV-69 common to all choice. Remove from choices.

36 H 2 X N E Add valve operation to A&B.

37 F 3 N s 38 H 3 N s 39 H 2 N s 40 F 2 D N u In D change "or" to "and" to avoid second correct answer.

41 H 3 N s 42 F 2 N s 43 H 2 N s 44 H 2 N s 45 F 2 N s 46 F 2 A,B N u Heat generation Rx Bldg does not seem plausible for SBO concerns.

47 F 1 B X N u Low LOD. B is a normal condition vs accident. KAMM. Reselect.

1.-118 H 3 X B E Provide initial plant conditions.

ES-401 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO 8/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 H 2 N s 50 H 2 N s 51 H 2 M s 52 H 2 X N E Add bullet about drift alarm.

53 H 2 B s 54 F 2 A,B,D N u Replace or reselect.

55 H 1 X N u Low LOD. KAMM- not using reference material. Reselect.

56 H 2 A N E Modify A regarding rod block.

57 H 2 8 s 58 F 2 c N E Partial loss of breaker control implausible. Fix C.

59 H 2 N s 60 H 2 X N E Change D to "purge to the drywell."

61 H 2 N s 62 F 2 N s 63 H 3 N s 64 F 3 N s 65 F 2 A N E Modify A.

66 F 3 X N E Capitalize "and" in stem.

67 H 2 M s Amplify justification.

J:;q9;8 H 2 N s H vs F

- ~--

ES-401 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO 8/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only 69 F 3 B s 70 F 3 B,D B u 23.5% oxygen is higher than normal air. Lower value.

71 F 3 B s 72 F 2 N s 73 H 2 M s 74 F 2 N s 75 H 1 M u Rework or reselect.

76 H 2 X N E Specify the EOP implemented to meet SRO-only format.

77 H 2 X N E Remove graph from Q and add to general reference package.

78 H 2 X N u Can be answered with systems knowledge. Replace/reselect.

79 H 2 N s 80 H 2 X N E Meets format of 55.43(b)5 by adding second part to Q (next steps).

81 H 2 N s 82 H 3 N s Add explanation to justify as SRO-only.

83 H 3 N s 84 H 3 N s 85 H 2 N s 86 H 2 X N E Add "immediate" to stem.

ES-401 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO 8/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 87 H 2 X N u Q can be answered with systems knowledge. Replace/reselect.

88 H 3 X N E Time frame missing from C.

89 H 2 N s 90 H 2 N s 91 F 2 N s 92 H 2 N s Add justification regarding SRO-only Q.

93 F 2 M s 94 F 3 M s 95 H 3 M s 96 F A N E Make A "Installed PRMs only".

97 F 2 X C,D M u Add "Initial" to stem. Modify C & D- eliminate SAE.

98 F 2 A N E Replace A.

99 F 2 N s 100 F 2 D N E D not plausible.