ML14246A231

From kanterella
Jump to navigation Jump to search
NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)
ML14246A231
Person / Time
Site: Beaver Valley
Issue date: 07/01/2014
From: Brian Fuller
Operations Branch I
To:
FirstEnergy Nuclear Operating Co
Shared Package
ML14058A120 List:
References
U01884
Download: ML14246A231 (6)
v • d • e


Text

ES-401 Beaver Valley Unit 1 FY 14 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N UIE/S Explanation Focus Dis!. Link units ward KIA Only 1 H 3 y N s Editorial comments incorporated.

2 F 2 y N s 3 F 2 y N s 4 H 3 y N s 5 H 3 y N s Q meets KIA by providing control board indications which are considered as control room references 6 F 3 y N s 7 H 2 y N s KIA is for proportional heaters in manual, Q matches KIA for manual manual by failing the MPC 8 F 2 y B s Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy- difficult) rating scale (questions in the 2- 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

.. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) .

. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) .

The answer choices are a collection of unrelated true/false statements .

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable .

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) .

4.

..Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content) .

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) .

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) .

The question requires reverse logic or application compared to the job requirements .

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 9 H 3 y N s Editorial comments incorporated 10 F 3 y N s 11 F 3 y N s 12 F 3 y N E Rewrote stem to correct wording 13 H 2 y N s 14 H 2 y N E Wrote new distractor for C 15 H 3 y N s 16 F 3 y N s 17 H 4 y B E Wrote new distractor for A.

18 F 3 y N s Added a provided reference to Q 19 H 4 y N s 20 H 4 y N s Added justification for KIA match, relays actuate control room annunciators 21 H 2 y N s 22 F 3 y N u KIA MM, wrote new Q 23 F 2 y N s 24 F 3 y N s 25 H 3 y N s 26 H 4 y N s 27 H 3 y N s 28 F 3 y N s

ES-401 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only 29 H 2 y B s 30 F 2 y N s 31 F 3 y N s 32 F 3 y N E Wrote new Q to better match the KIA 33 H 2 y N s 34 F 2 y N s 35 F 2 y N s 36 H 3 y N s 37 H 2 y N s 38 F 2 y N s 39 H 3 y N s 40 F 2 y N s 41 F 2 y N s 42 H 3 y N s 43 H 3 y N s 44 F 3 y N s 45 H 2 y N s 46 F 2 y N s 47 H 3 y N s 48 H 3 y N E Deleted ONL Ys from all choices

ES-401 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 H 3 y N s y

50 F 2 N s Editorial comments incorporated y

51 F 2 N s Editorial comments incorporated y

52 F 2 N s y

53 F 3 N s y

54 H 3 N s y

55 H 3 B u KIA MM, wrote new Q y

56 H 2 N s y

57 H 2 N s y

58 H 3 N s y

59 F 3 N s y

60 H 3 N s y

61 F 2 B s y

62 H 2 N s y

63 H 2 N s y

64 H 2 N E Wrote new combinations of distractors y

65 F 3 N s y

66 F 2 N s Editorial comments incorporated y

67 H 2 N s y

68 F 2 N s

ES-401 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO 8/M/N UIE/S Explanation Focus Dist. Link units ward KIA Only y

69 H 2 N E Rewrote distractor choices y

70 H 3 N s y

71 F 2 N s y

72 H 3 N s Editorial comments incorporated y

73 F 3 N s y

74 H 2 N s y

75 H 2 N s Editorial comments incorporated y

76 H 3 y N s y

77 H 3 N N s Editorial comments incorporated y

78 H 2 N N s Editoiial comments incoipoiated y

79 H 2 N N E Rewrote distractors to restructure answers (2 parts vice 3) y 80 H 3 y N s y

81 H 3 y N s y

82 F 3 y N s y

83 H 2 y N s Editorial to correct justification y

84 H 2 y N s y

85 H 2 y N s y

86 H 3 y N E Q did not directly test KIA, unable to write adequate Q. Selected new I KIA, wrote new Q.

y 87 H 2 y N s Editorial comments incorporated y

88 H 2 y N E Editorial comments incorporated

ES-401 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only y

89 H 3 y N E Added determination of operability to Q y

90 H 3 y N s Editorial comments incorporated y

91 H 3 y N E Rewrote stem and distractor part 2's y

92 H 3 y N s Editorial comments incorporated y

93 F 2 y N s y

94 F 3 y N E Rewrote stem to eliminate implausible choice, renumbered distractor combinations to match y

95 H 2 y N s Editorial comments incorporated y

96 H 2 y B s Editorial comments incorporated in justification y

97 H 3 y N E Oversampied event classification (w/ Q#83). Wroie new Q y

98 H 2 y N s y

99 F 2 y N s y

100 H 3 y N s

Retrieved from "https://kanterella.com/w/index.php?title=ML14246A231&oldid=2451379"