ML113330734

NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)

ML113330734
Person / Time
Site:	Salem
Issue date:	09/22/2011
From:	NRC Region 1
To:	Public Service Enterprise Group
JACKSON D RGN-I/DRS/OB/610-337-5306
Shared Package
ML110030673	List: IR 05000272/2011301 ML111610244 ML112590037 ML112650617 ML113330710 ML113330734 ML113330777 ML113330798 ML113330805 ML113340036 ML113340040 ML113340044 ML113340065 ML113500280 ML12167A076
References
TAC U01830
Download: ML113330734 (8)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

---

--------

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Creel. Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 F N U Overlap with SRO #16. replaced question.

Modified Choices C and D to enhance discriminatory value.

2 H N E Added additional information to justification section.

Recategorized as memory level, not higher order.

3 F X N U Modified stem and distracters to make distracters credible.

Modified stem, added information relating to A TWS condition to 4 H B E enhance credibility of the 'two-pump' choices. Modified choices to improve discriminatory value.

(Draft 0#18) 5 M N E Question replaced because of overlap with tested knowledge in Questions 29 and 56.

(Draft Q#5) 6 F B E Minor distractor wording change. Changed value in distractor to improve plausibility. Added to justification to explain why motor temp may not be increasing with 10 flow alarm.

(Draft Q#6) 7 F B E Replaced one implausible distractor, reordered choices.

(Draft 0#7) 8 H B E Removed "references provided" statement from the stem. Modified choice values to prevent overlap between choices.

(Draft 0#8)

Reworded several choices to improve plausibility and eliminate overlap with Q#20.

9 F N E This question matches KIA for applying system limits because the heat up of seal injection to 225°F in 5.3 minutes is a system limitation as documented in a loss of CCW calculation and in the Technical Basis Document for S2.0P-AB.CC-0001.

Page 1

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

0# LOK LOD (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back 0= SRO B/M/N utE/S Explanation Focus Dist. Link units ward KIA Only (Draft 0#9) 10 F B E Removed "hotter" and "cooler" from Choices BID. Unnecessary, could be confusing and could cue the correct answer.

(Draft 0#10) 11 F N E Changed running pump in the stem to enhance distractor credibility.

Reordered answer choices.

(Draft 0#11) 12 H B E Removed unnecessary, leading information ("since it is not being discriminated out") from both Choice C and Choice D.

13 H B S (Draft 0#12)

(Draft 0#13) 14 M N E Fixed typo in Choice D. Modified wording in Choices A and B for symmetry. Corrected reference page number.

15 H N S (Draft 0#14)

(Draft 0#15) 16 H N E Replaced O. Too many GFE level questions on test.

(Draft 0#16) 17 F X N U Modified choices to ensure each choice is mutually exclusive of the corresponding choice.

(Draft 0#17) 18 H 3.0 N E Minor editorial changes in distracters.

Page 2

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (FIH) (1-5) Stem Cues TlF Credo Partial Job- Minutia #1 Back- Q= SRO BIMtN utEtS Explanation Focus Dist. Link units ward KIA Only 19 H B S 20 F N S Modified Choice C to prevent overlap with Q#9 (Draft Q#8).

21 F B E Replaced one non-credible distractor. Reordered choices.

22 H B S Reclassified as higher cognitive level question.

23 M B U Low LOD. Replaced question.

24 H B S H B S Added to justification to explain that lockout switch is plausible 25 because it is a Unit difference.

26 F N S 27 F N S Reclassified as a lower cognitive level question.

Corrected minor typo in stam.

28 H N E Added "initial" to stem question.

Modified wording of generator MW choice to make clearly wrong.

Minor stem wording change.

29 H N S Added to justification to elaborate on why Choice A is wrong.

30 H B S 31 H X X B U Weak KIA match and limited operational validity. Replaced Q.

32 H N S Added to question justification to explain the KIA match.

33 H N S Fixed typos in justification section.

34 H N S 35 F M S 36 H B S Page 3

ES-401 2 Form ES*401*9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N UlE/S Explanation Focus Dist. Link units ward KIA Only Modified conditions in stem to remove the leak and replace with 37 H B E different fault. Changed to address the overlap between 0#37 and 0#3.

H X B E Added transmitter O/S in stem to improve distractor plausibility and 38 increase LOD.

39 F B S 40 F N S

• 41 H B S Minor distractor wording change, rearranged choices.

• 42 H B S

• 43 H N S

• 44 H N S 45 H B S 46 F B S 47 F N S 48 F X B U Collection of TruelFalse statements. Rewrote Q to fix psychometric flaw.

--

49 H N S Too many GFE level Qs on RO exam. Replaced with new Q.

50 F N E Modified stem to make more operationally valid.

51 H N S

• 52 F N U Low LOD based on initial reactor trip EOP immediate action, practiced many times in simulator scenarios. Question replaced.

53 F N E Added "lAW ... " to stem.

• 54 H N E Rewrote question and distracters to improve plausibility of choices.

55 H N S Page 4

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD 0# (FtH) (1-5) Stem Cues TtF Credo Partial Job- Minutia #t Back- 0= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only 56 H B S 57 F B S 58 H N S I I

F B S Corrected typo in justification section. Modified justification for 59 correct answer to explain TS.

F N E Replaced distractor containing 2R1A because discriminatory value 60 reduced by 0#61. Reordered choices.

61 H N S 62 H M S 63 H B S 64 H B E Replaced one implausible distractor. Reordered choices.

65 F B E Modified stem and one distractor to improve distractor plausibility.

66 F N E Modified stem to enhance distractor plausibility.

67 F N S 68 H N S Facility confirmed this is expected RO level knowledge.

H B E Modified justification to align with key answer. Reduced total pri to 69 sec leakage in stem to ensure per-generator value is within limits.

70 H B S .

SRO 0#22 helps to answer this question. However, RO exam is  !

71 H B S collected before SRO exam distributed to applicants. Therefore, no i need for change to SRO 0#22. I Removed "the" from Choice C. Added to "Unit 1 trips" choice, to 72 H B E provide information for assessment that emergency not in progress.

Reordered choices.

73 F B S Page 5

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only F B E Modified a distractor to remove leading information. Replaced one 74 distractor with a plausible incorrect choice.

75 F X N U Modified stem. Replaced ChOices Band D with plausible distracters.

(SRO#1) 76 H M E Deleted mode information from the stem.

(SRO #2) 77 H N E Provided RCS temperature and pressure parameters, vice plant mode, in the stem.

(SRO #3) 78 H N E Modified the choice wording to ensure question requires procedure knowledge to answer correctly.

(SRO #4) 79 H N E Changed Choice C from "2FR" to "1 FR" to align with the designated unit Improved stem clarity by changing to " ... and which would require ._."_

80 H N S (SRO#5)

(SRO#6)

If "O/S" means out of service, which could be interpreted as inoperable, then Choice C is not credible since it wouldn't change 81 H N E conditions. Perhaps better to state 22 RHR pump is in standby.

Choice D could be argued as a second correct answer, based on assumption that the channel is inoperable because of its audible function.

82 H B S (SRO#7) 83 H N s (SRO#8) 84 H N S (SRO#9)

(SRO #10) 85 H B E Rewrote stem and distrac!ers to better test procedure knowledge.

Page 6

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only (SRO #11) (DRAFT SRO #12) 86 H X N E Replaced Choice A distractor. Added explanation of KIA match.

87 H N S (SRO #12) (DRAFT SRO #13)

._-. ~.---~

(SRO #13) (DRAFT SRO #14) 88 H X N U Minor stem wording change. Reworded Choice A to eliminate the 2 00 correct answer.

(SRO #14) (DRAFT SRO #15) 89 H X N U KA mismatch. KA is for ESF AS. Question tests RPS. Wrote new question to same KA.

(SRO #15) (DRAFT SRO #16) 90 H B E Fixed typos. Minor stem rewording. Reworded distracters for clarity and to improve discriminatory value.

(SRO #16) (DRAFT SRO #11) 91 H X N U Not SRO only. Replaced Q.

(SRO #17) 92 H N E Minor wording changes in Choices C and D.

93 H M S (SRO #18)

(SRO#19) 94 H X B U Could be answered with system knowledge only. Replaced question. i (SRO #20) 95 H X B U Not SRO-only. Can be answered with basic GFE knowledge.

Replaced question.

96 H B S (SRO #21)

Page 7

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utE/S Explanation Focus Dist. Link units ward KIA Only (SRO#22)

Minor stem wording change to ensure all choices are supported by the question.

97 H B E This question contains information that could help to eliminate two distracters on RO 0#71 by narrowing the annual PSE&G limit to between 1900 and 2350 mR. However. RO exam is collected before SRO exam distributed to applicants. Therefore, no need for change.

(SRO#23) 98 H B E Rewarded stem and choices to improve distractor plausibility.

(SRO#24) 99 H X B E Rewrote stem conditions and choices to enhance discriminatory value of distracters.

100 H N S (SRO #25)

Page 8