ML15352A064

NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)

ML15352A064
Person / Time
Site:	Three Mile Island
Issue date:	09/25/2015
From:	Peter Presby Operations Branch I
To:	Exelon Generation Co
Shared Package
ML15135A253	List: IR 05000289/2015301 ML15267A771 ML15292A369 ML15292A370 ML15335A327 ML15335A328 ML15335A329 ML15335A330 ML15335A331 ML15335A332 ML15352A064 ML15352A065
References
TAC U01913
Download: ML15352A064 (5)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 N u Overlap with Scenario. Reselected KIA, wrote new Q.

2 F 3 N u Key answer is incorrect. Replaced Q.

3 F 3 B E Revised component naming in stem.

4 H 3 N E Modified Q to improve clarity.

5 H 3 B E Typo in stem.

6 H 3 B E Modified Q and choices to improve distractor plausibility.

7 F 3 B E Minor Q rewording.

8 F 3 B E Minor edits of capitalization and abbreviation.

9 F 3 N E Modified to improve distractors and ensure no overlap with a SRO Q ..

10 F 2 N E Modified stem conditions to improve distractor plausibility.

11 F 3 M E Improved distractors by changing order of priority in choices.

12 H 3 B E Minor Q wording enhancement.

13 H 3 N E Low LOO. Two distractors not credible. Modified stem conditions, choices and provided reference to raise LOO and fix distractors.

14 F 3 x N u KA mismatch. Rewrote the Q.

15 H 3 N E Modified stem to ensure not leading.

16 H 3 B s 17 H 3 N E Minor Q wording change.

18 F 3 N E Minor distractor improvement.

19 F 3 x N u Two implausible choices. Rewrote the Q.

20 H 3 B s 2014 NRC Exam Q#21 21 F 2 B E Reworded Q and choices to raise LOO.

Page 1

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only F 3 B E Okay Q in appropriate topical area, but poor KIA match. Tied Q to a 22 different KIA per CE. Minor edits.

23 H 2 N E Edited to improve plausibility of distractors.

24 H 3 x B u Weak KIA match. Implausible distractors. Wrote new Q.

25 H 3 N u Low LOO. Not discriminating. Reselected KIA. Wrote new Q.

26 F 3 x N u KIA mismatch. Wrote new Q.

27 H 3 N E Modified wording for clarity.

28 H 3 N E Addressed ambiguity in 2nd part of the Q.

29 H 3 N s 30 F 2 B s 31 H 3 B s 2014 NRC Exam Q#31. Minor stem component naming change.

32 H 3 N E Minor rewording.

33 H 3 x N u KIA mismatch. Rewrote the Q.

34 H 3 M E Minor wording change.

35 F 3 B E Re-ordered choices.

36 F 3 N E Minor typos. Enhanced distractors.

37 F 3 N E Modified Q and distractors to improve discriminatory value.

38 F 3 N E Added ONLY to choices to ensure no choice is a subset of any other.

39 F 3 B E Reworded Q and choices to improve distractor plausibility.

40 H 3 B E 2012 NRC Exam Q#9. Minor stem wording change.

41 F 3 B E Choices revised to address problem with grammar and structure.

Page 2

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 42 F M E Fixed typos.

43 H 3 B s 44 H 3 B E Minor stem rewording to improve clarity.

45 F 3 N E Reworded choices to improve discriminatory value.

46 H 3 x N u Reworded stem and choices to address two implausible distractors.

47 H 3 N E Minor rewording of two choices to improve clarity 48 H 3 B s 49 H 3 M s 2014 NRC Exam Q#50 50 H 3 B s 51 F 3 B s 52 H 3 N s 53 H 3 x N u KIA mismatch. Wrote new Q.

54 H 3 N s 55 F 3 B E Minor clarification wording changes.

56 F 3 M E Modified to ensure Q cannot be answered by knowing only one part.

Minor order change for symmetry.

57 H 3 M E Minor wording changes.

58 H 3 M E Minor wording changes to improve clarity.

59 H 3 B E Minor edits for clarification.

60 H 3 N E Typo in stem.

61 F 3 B s 62 F 3 B s Page 3

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 63 H 3 N E Modified Q to improve distractor plausibility.

64 F 3 M s 65 H 3 B s 66 H 3 B E Modified Q and distractors to improve discriminatory value.

67 H 3 M s 68 F 3 B s 2014 NRC Exam Q#70 69 H 3 M s 70 F 3 B E Modified stem wording to make more precise.

F 3 B E Removed some information from answer choice to improve 71 discriminatory value of the Q.

72 F 3 B E 2014 NRC Exam Q#73, minor word change for consistency 73 F 3 N E Minor wording changes.

74 H 3 B s 75 H 3 B s 2012 NRC Exam Q#72 76 H 3 N s 77 F 3 N E Removed unnecessary information from stem.

78 H 3 B s 2014 NRC Exam Q#76 79 H 3 N E Minor changes to improve clarity, discriminatory value 80 F 3 N s 81 H 3 M E Added information to stem to support distractor plausibility.

82 F 3 B E Minor wording change In distractors.

83 H 3 M E Modified stem to improve plausibility of distractors.

Page4

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 84 H 3 N E Minor edits.

85 F 3 N s 86 F 3 N E Fixed one distractor. Other minor wording changes.

87 H 3 M E Minor edits.

88 H 3 N E Reworded to improve discriminatory value of distractors.

89 F 3 M E Minor edits.

90 H 3 N E Changes to improve distractors.

91 F 3 N E Minor edit.

92 H 3 B s 93 H 3 N E Modified to improve distractors.

94 H 3 B E Minor stem modification.

95 F 3 B s F 3 N E Okay Q in appropriate topical area, but poor KIA match. Tied Q to a 96 different KIA per CE. Minor edits.

97 F 3 B E Minor wording change.

98 H 3 B s 99 H 3 B E Modified distractors to improve plausibility.

100 H 3 B s Page 5

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 N u Overlap with Scenario. Reselected KIA, wrote new Q.

2 F 3 N u Key answer is incorrect. Replaced Q.

3 F 3 B E Revised component naming in stem.

4 H 3 N E Modified Q to improve clarity.

5 H 3 B E Typo in stem.

6 H 3 B E Modified Q and choices to improve distractor plausibility.

7 F 3 B E Minor Q rewording.

8 F 3 B E Minor edits of capitalization and abbreviation.

9 F 3 N E Modified to improve distractors and ensure no overlap with a SRO Q ..

10 F 2 N E Modified stem conditions to improve distractor plausibility.

11 F 3 M E Improved distractors by changing order of priority in choices.

12 H 3 B E Minor Q wording enhancement.

13 H 3 N E Low LOO. Two distractors not credible. Modified stem conditions, choices and provided reference to raise LOO and fix distractors.

14 F 3 x N u KA mismatch. Rewrote the Q.

15 H 3 N E Modified stem to ensure not leading.

16 H 3 B s 17 H 3 N E Minor Q wording change.

18 F 3 N E Minor distractor improvement.

19 F 3 x N u Two implausible choices. Rewrote the Q.

20 H 3 B s 2014 NRC Exam Q#21 21 F 2 B E Reworded Q and choices to raise LOO.

Page 1

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only F 3 B E Okay Q in appropriate topical area, but poor KIA match. Tied Q to a 22 different KIA per CE. Minor edits.

23 H 2 N E Edited to improve plausibility of distractors.

24 H 3 x B u Weak KIA match. Implausible distractors. Wrote new Q.

25 H 3 N u Low LOO. Not discriminating. Reselected KIA. Wrote new Q.

26 F 3 x N u KIA mismatch. Wrote new Q.

27 H 3 N E Modified wording for clarity.

28 H 3 N E Addressed ambiguity in 2nd part of the Q.

29 H 3 N s 30 F 2 B s 31 H 3 B s 2014 NRC Exam Q#31. Minor stem component naming change.

32 H 3 N E Minor rewording.

33 H 3 x N u KIA mismatch. Rewrote the Q.

34 H 3 M E Minor wording change.

35 F 3 B E Re-ordered choices.

36 F 3 N E Minor typos. Enhanced distractors.

37 F 3 N E Modified Q and distractors to improve discriminatory value.

38 F 3 N E Added ONLY to choices to ensure no choice is a subset of any other.

39 F 3 B E Reworded Q and choices to improve distractor plausibility.

40 H 3 B E 2012 NRC Exam Q#9. Minor stem wording change.

41 F 3 B E Choices revised to address problem with grammar and structure.

Page 2

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 42 F M E Fixed typos.

43 H 3 B s 44 H 3 B E Minor stem rewording to improve clarity.

45 F 3 N E Reworded choices to improve discriminatory value.

46 H 3 x N u Reworded stem and choices to address two implausible distractors.

47 H 3 N E Minor rewording of two choices to improve clarity 48 H 3 B s 49 H 3 M s 2014 NRC Exam Q#50 50 H 3 B s 51 F 3 B s 52 H 3 N s 53 H 3 x N u KIA mismatch. Wrote new Q.

54 H 3 N s 55 F 3 B E Minor clarification wording changes.

56 F 3 M E Modified to ensure Q cannot be answered by knowing only one part.

Minor order change for symmetry.

57 H 3 M E Minor wording changes.

58 H 3 M E Minor wording changes to improve clarity.

59 H 3 B E Minor edits for clarification.

60 H 3 N E Typo in stem.

61 F 3 B s 62 F 3 B s Page 3

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 63 H 3 N E Modified Q to improve distractor plausibility.

64 F 3 M s 65 H 3 B s 66 H 3 B E Modified Q and distractors to improve discriminatory value.

67 H 3 M s 68 F 3 B s 2014 NRC Exam Q#70 69 H 3 M s 70 F 3 B E Modified stem wording to make more precise.

F 3 B E Removed some information from answer choice to improve 71 discriminatory value of the Q.

72 F 3 B E 2014 NRC Exam Q#73, minor word change for consistency 73 F 3 N E Minor wording changes.

74 H 3 B s 75 H 3 B s 2012 NRC Exam Q#72 76 H 3 N s 77 F 3 N E Removed unnecessary information from stem.

78 H 3 B s 2014 NRC Exam Q#76 79 H 3 N E Minor changes to improve clarity, discriminatory value 80 F 3 N s 81 H 3 M E Added information to stem to support distractor plausibility.

82 F 3 B E Minor wording change In distractors.

83 H 3 M E Modified stem to improve plausibility of distractors.

Page4

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOO Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 84 H 3 N E Minor edits.

85 F 3 N s 86 F 3 N E Fixed one distractor. Other minor wording changes.

87 H 3 M E Minor edits.

88 H 3 N E Reworded to improve discriminatory value of distractors.

89 F 3 M E Minor edits.

90 H 3 N E Changes to improve distractors.

91 F 3 N E Minor edit.

92 H 3 B s 93 H 3 N E Modified to improve distractors.

94 H 3 B E Minor stem modification.

95 F 3 B s F 3 N E Okay Q in appropriate topical area, but poor KIA match. Tied Q to a 96 different KIA per CE. Minor edits.

97 F 3 B E Minor wording change.

98 H 3 B s 99 H 3 B E Modified distractors to improve plausibility.

100 H 3 B s Page 5