ML22355A650

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)

ML22355A650
Person / Time
Site:	FitzPatrick
Issue date:	08/30/2021
From:	Brian Fuller Operations Branch I
To:	Kelly D Exelon FitzPatrick
References
EPID: L-2021-OLL-0000
Download: ML22355A650 (1)

Text

ES-401 FitzPatrick 2021 Written Examination Review Worksheet Form ES-401-9

2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source 7. Status Q 1. LOK (F/H) LOD Stem (B/M/N) (U/E/S) 8. Explanation (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia #/Units Backward Q - K/A SRO Only

1 H 2 N S

2 F 3 N S

3 H 4 B S

4 F 2 N E Rework the 1st part of options C & D (use info from ST eliminate potential for 2 correct)

5 H 3 X B E Remove ref pic (give valve pos instead)

6 H 4 B S

7 F 2 B E Change level indication options to NPSH

8 H 4 M S

Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.

3. Check the appropriate box if a psychometric flaw is identified:

• Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).

• T/F: The answer choices are a collection of unrelated true/false statements.

• Cred. Dist.: The distractors are not credible; single implausible distractors s hould be repaired, and more than one is unacceptable.

• Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).

4. Check the appropriate box if a job content flaw is identified:

• Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• Minutia: The question requires the recall of knowledge that is too specific for the closed -reference test mode (i.e., it is not required to be known from memory).

• #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• Backward: The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license -level mismatches are unacceptable.)

6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.

7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source 7. Status Q LOK LOD Stem (B/M/N) (U/E/S) 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only

9 H 2 N S

10 H 3 N E Enhance verbiage to improve clarity

11 F 2 B S

12 F 2 B S

13 H 3 N S

14 F 2 N S

15 H 2 B S

16 H 2 N S

17 F 3 N S

18 H 4 B S

19 H 3 N S

20 H 4 B S

21 H 3 B S

22 F 2 N S

23 F 2 B S

24 F 2 B S

25 H 3 N E Revise plausibility statement and change option A verbiage to match B ES-401 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source 7. Status Q LOK LOD Stem (B/M/N) (U/E/S) 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only

26 F 2 B S

27 F 3 N S

28 H 3 N S

29 F 2 N S

30 H 3 B S

31 H 3 B S

32 F 2 N S

33 H 4 B S

34 F 2 N S

35 H 2 M S

36 H 3 B E Change to 2 double & 2 single options

37 H 2 M E Make column headers look the same

38 H 3 B S

39 F 2 N S

40 F 2 B S

41 H 4 B S

42 H 3 B S ES-401 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source 7. Status Q LOK LOD Stem (B/M/N) (U/E/S) 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only

43 H 3 B E Change to 100% power

44 H 2 N S

45 F 3 B E Change option B to cooling for Recirc Pumps

46 H 2 N S

47 H 3 B S

48 H 2 N S

49 F 2 N S

50 H 3 N S

51 H 2 N S

52 H 2 B S

53 H 3 N S

54 F 3 M S

55 F 3 N E Fix typo

56 F 3 B S

57 F 2 B S

58 H 3 B S

59 H 2 N S ES-401 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source 7. Status Q LOK LOD Stem (B/M/N) (U/E/S) 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only

60 H 2 B S

61 F 2 B S

62 H 3 M S

63 H 3 M S

64 H 4 M E Eliminate unnecessary words

65 F 3 B S

66 F 3 B S

67 F 2 B S

68 F 2 N S

69 H 2 B S

70 F 2 M S

71 H 2 B S

72 F 2 B S

73 H 2 B S

74 F 2 B S

75 F 2 B S

76 H 3 M S ES-401 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source 7. Status Q LOK LOD Stem (B/M/N) (U/E/S) 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only

77 H 2 B S

78 H 2 B E Add 100% power, refueling preparations

79 H 3 B S

80 H 2 N S

81 H 2 N S

82 F 3 N S

83 H 2 N E Combine 2nd & 3rd columns into one

84 H 2 N S

85 H 2 B S

86 H 2 N E Add SRO level justification

87 F 2 N S

88 H 2 M S

89 H 2 B S

90 H 2 X N U Changed to give voltage values vs. unsat

91 F 2 N S

92 H 4 M S

93 H 3 N S ES-401 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source 7. Status Q LOK LOD Stem (B/M/N) (U/E/S) 8. Explanation (F/H) (1-5) Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only

94 H 2 N S

95 F 2 N S

96 H 3 X X N U Rewrite to match K/A. Question replaced

97 F 2 N S

98 H 2 B S

99 H 3 B S

100 F 2 N S