ML113330734

NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)

ML113330734
Person / Time
Site:	Salem
Issue date:	09/22/2011
From:	NRC Region 1
To:	Public Service Enterprise Group
JACKSON D E RGN-I/DRS/OB/610-337-5306
Shared Package
ML110030673	List: IR 05000272/2011301 ML111610244 ML112590037 ML112650617 ML113330710 ML113330734 ML113330777 ML113330798 ML113330805 ML113340036 ML113340040 ML113340044 ML113340065 ML113500280 ML12167A076
References
TAC U01830
Download: ML113330734 (8)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Creel. Partial Job-Minutia #1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 F N U Overlap with SRO #16. replaced question.

2 H N E Modified Choices C and D to enhance discriminatory value. Added additional information to justification section. 3 F X N U Recategorized as memory level, not higher order. Modified stem and distracters to make distracters credible.

4 H B E Modified stem, added information relating to A TWS condition to enhance credibility of the 'two-pump' choices. Modified choices to improve discriminatory value. (Draft 0#18) 5 M N E Question replaced because of overlap with tested knowledge in Questions 29 and 56. (Draft Q#5) 6 F B E Minor distractor wording change. Changed value in distractor to improve plausibility.

Added to justification to explain why motor temp may not be increasing with 10 flow alarm. 7 F B E (Draft Q#6) Replaced one implausible distractor, reordered choices. (Draft 0#7) 8 H B E Removed "references provided" statement from the stem. Modified choice values to prevent overlap between choices. (Draft 0#8) Reworded several choices to improve plausibility and eliminate 9 F N E overlap with Q#20. This question matches KIA for applying system limits because the heat up of seal injection to 225°F in 5.3 minutes is a system limitation as documented in a loss of CCW calculation and in the Technical Basis Document for S2.0P-AB.CC-0001.

Page 1 ES-401 Written Examination Review Worksheet Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. 0# LOK LOD (F/H) (1-5) Stem Cues TIF Credo Partial Job-Minutia #1 0= SRO B/M/N utE/S Explanation Focus Dist. Link units ward KIA Only (Draft 0#9) 10 F B E Removed "hotter" and "cooler" from Choices BID. Unnecessary, could be confusing and could cue the correct answer. (Draft 0#10) 11 F N E Changed running pump in the stem to enhance distractor credibility.

Reordered answer choices. (Draft 0#11) 12 H B E Removed unnecessary, leading information

("since it is not being discriminated out") from both Choice C and Choice D. 13 H B S (Draft 0#12) (Draft 0#13) 14 M N E Fixed typo in Choice D. Modified wording in Choices A and B for symmetry.

Corrected reference page number. 15 H N S (Draft 0#14) 16 H N E (Draft 0#15) Replaced O. Too many GFE level questions on test. (Draft 0#16) 17 F X N U Modified choices to ensure each choice is mutually exclusive of the corresponding choice. 18 H 3.0 N E (Draft 0#17) Minor editorial changes in distracters.

Page 2 ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. LOK LOD Q# (FIH) (1-5) Stem Cues TlF Credo Partial Job-Minutia #1 Back-Q= SRO BIMtN utEtS Explanation Focus Dist. Link units ward KIA Only 19 H B S 20 F N S Modified Choice C to prevent overlap with Q#9 (Draft Q#8). 21 F B E Replaced one non-credible distractor.

Reordered choices. 22 H B S Reclassified as higher cognitive level question.

23 M B U Low LOD. Replaced question.

24 H B S 25 H B S Added to justification to explain that lockout switch is plausible because it is a Unit difference.

26 F N S 27 F N S Reclassified as a lower cognitive level question.

Corrected minor typo in stam. 28 H N E Added "initial" to stem question.

Modified wording of generator MW choice to make clearly wrong. 29 H N S Minor stem wording change. Added to justification to elaborate on why Choice A is wrong. 30 H B S 31 H X X B U Weak KIA match and limited operational validity.

Replaced Q. 32 H N S Added to question justification to explain the KIA match. 33 H N S Fixed typos in justification section. 34 H N S 35 F M S 36 H B S Page 3

--2 Form ES*401*9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. LOK (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N UlE/S Explanation Focus Dist. Link units ward KIA Only Modified conditions in stem to remove the leak and replace with 37 H B E different fault. Changed to address the overlap between 0#37 and 0#3. Added transmitter O/S in stem to improve distractor plausibility and 38 H B E increase LOD. 39 F B S 40 F N S

• 41 H B S Minor distractor wording change, rearranged choices.

• 42 H B S 43 H N S *

• 44 H N S 45 H B S 46 F B S 47 F N S Collection of TruelFalse statements.

Rewrote Q to fix psychometric 48 F B U flaw. 49 H N S Too many GFE level Qs on RO exam. Replaced with new Q. 50 F N E Modified stem to make more operationally valid. 51 H N S

• Low LOD based on initial reactor trip EOP immediate action, 52 N U practiced many times in simulator scenarios.

Question replaced.

53 N E Added "lAW ... " to stem. F

• 54 H N E Rewrote question and distracters to improve plausibility of choices. 55 H N S Page 4 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. LOK 0# (FtH) Stem Cues TtF Credo Partial Job-Minutia #t Back-0= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only 56 B S 57 B S 58 N S I I Corrected typo in justification section. Modified justification for 59 B S correct answer to explain Replaced distractor containing 2R1A because discriminatory 60 N E reduced by 0#61. Reordered choices. 61 H N S 62 H M S 63 H B S 64 H B E Replaced one implausible distractor.

Reordered choices. 65 F B E Modified stem and one distractor to improve distractor plausibility.

66 F N E Modified stem to enhance distractor plausibility.

67 F N S 68 H N S Facility confirmed this is expected RO level knowledge.

Modified justification to align with key answer. Reduced total pri to 69 B E sec leakage in stem to ensure per-generator value is within limits. 70 H B S . SRO 0#22 helps to answer this question.

However, RO exam is ! 71 H B S collected before SRO exam distributed to applicants.

Therefore, no i need for change to SRO 0#22. I Removed "the" from Choice C. Added to "Unit 1 trips" choice, to 72 H B E provide information for assessment that emergency not in progress.

Reordered choices. 73 B S Page 5 ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only 74 F B E Modified a distractor to remove leading information.

Replaced one distractor with a plausible incorrect choice. 75 F X N U Modified stem. Replaced ChOices Band D with plausible distracters.

76 H M E (SRO#1) Deleted mode information from the stem. (SRO #2) 77 H N E Provided RCS temperature and pressure parameters, vice plant mode, in the stem. (SRO #3) 78 H N E Modified the choice wording to ensure question requires procedure knowledge to answer correctly. (SRO #4) 79 H N E Changed Choice C from "2FR" to "1 FR" to align with the designated unit Improved stem clarity by changing to " ... and which would require ._."_ 80 H N S (SRO#5) (SRO#6) If "O/S" means out of service, which could be interpreted as inoperable, then Choice C is not credible since it wouldn't change 81 H N E conditions.

Perhaps better to state 22 RHR pump is in standby. Choice D could be argued as a second correct answer, based on assumption that the channel is inoperable because of its audible function.

82 H B S (SRO#7) 83 H N s (SRO#8) 84 H N S (SRO#9) 85 H B E (SRO #10) Rewrote stem and distrac!ers to better test procedure knowledge.

Page 6 ES-401 2 Form ES-401-9 5. Other 6. 7. Q= SRO B/M/N U/E/S 1. 2. 3. Psychometric Flaws LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Focus Dist. 86 H X 87 H 88 H X H 89 90 H 91 H 92 H 93 H 94 H 95 H H 96 4. Job Content Flaws Job-Minutia #1 Link units ._-. 8. Explanation (SRO #11) (DRAFT SRO Replaced Choice A distractor.

Added explanation of KIA (SRO #12) (DRAFT SRO (SRO #13) (DRAFT SRO Minor stem wording change. Reworded Choice A to eliminate the 2 00 correct (SRO #14) (DRAFT SRO KA mismatch.

KA is for ESF AS. Question tests RPS. Wrote question to same (SRO #15) (DRAFT SRO #16) Fixed typos. Minor stem rewording.

Reworded distracters for clarity and to improve discriminatory value. (SRO #16) (DRAFT SRO #11) Not SRO only. Replaced Q. (SRO Minor wording changes in Choices C and (SRO Could be answered with system knowledge only. Replaced (SRO Not SRO-only.

Can be answered with basic GFE Replaced (SRO KIA Only X X X X N E N S N U N U B E N U N E M S B U B U B S Page 7 i ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. LOK LOD Q# (F/H) (1-5) Stem Cues TIF Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N utE/S Explanation Focus Dist. Link units ward KIA Only (SRO#22) Minor stem wording change to ensure all choices are supported by 97 H B E the question.

This question contains information that could help to eliminate two distracters on RO 0#71 by narrowing the annual PSE&G limit to between 1900 and 2350 mR. However. RO exam is collected before SRO exam distributed to applicants.

Therefore, no need for change. 98 H B E (SRO#23) Rewarded stem and choices to improve distractor plausibility. (SRO#24) 99 H X B E Rewrote stem conditions and choices to enhance discriminatory value of distracters.

100 H N S (SRO #25) Page 8