ML113350201

NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)

ML113350201
Person / Time
Site:	05000363
Issue date:	09/30/2011
From:	Presby P A Operations Branch I
To:	Armstrong L Dominion Nuclear Connecticut
JACKSON D E RGN-I/DRS/OB/610-337-5306
Shared Package
ML102850247	List: IR 05000336/2011301 ML103120114 ML112800407 ML113340068 ML113340071 ML113340095 ML113340098 ML113350197 ML113350201 ML113350205 ML113400006 ML113500249 ML113500251 ML12031A207
References
TAC U01834
Download: ML113350201 (7)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 i. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N UlEtS Explanation Focus Dis!. Link units ward KIA Only 1 H B S Deleted 2nd part of question and answers. Applicants will 2 H M E demonstrate understanding of the why if they correctly determine downstream temperature.

Question data modified to better match expected RCS conditions.

3 F B S 4 F M S 5 F N S Adjusted capitalization of "Bleedoff Flow" in the choices for consistency.

6 H B S Clarified how the Q matches the KIA in the justification section. 7 H M S 8 H X N U Multiple correct answers. Rewrote the question.

Reworded one distractor to make plausible.

9 H M E Regarding KA match, the proportional heaters are powered from vital (emergency) buses. The backup heaters are powered from non-vital (non-emergency buses). 10 H X M U Implausible distractor wording changed to make distractors plausible.

11 H N E Added to Q, "procedurally required per EOP2534".

Removed extra line between Choices A and B. 12 F M S 13 H X X M E Question did not match KA for demonstrating knowledge of the reasons and two choices were implausible.

Choices expanded to include the "why" and modified to make plausible.

14 H N S Minor editorial change 15 F N E Modified choice wording to improve distractor plausibility.

16 H X N U KA mismatch.

Question replaced.

Page 1

---

ES*401 Written Examination Review Worksheet Form ES*401*9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 00 LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N utE/S Focus Dist. Link units ward KIA Only 17 H X B E 18 H N E 19 H N E 20 F B S 21 F N E 22 H B E 23 F B E 24 F M E 25 F N E 26 F M E 27 H X 8 U I ?R F X X X N U II --I I I I I I I I I I I I I I I

8. Explanation Choice B implausible.

Choice replaced.

Reworded question and choices to improve clarity and eliminate cueing. Changed stem conditions and distractors to ensure only one correct Added to justification to explain how KA "metroscope" is Modified stem and choices to improve KA match and plausibility of Choice D modified to eliminate overlap with Choice Added to justification to explain KIA Minor modification to stem to better support the Rewrote question to improve clarity and better match the Added current EOP to the stem, corrected procedure names made other editorial Changed choices to eliminate the second correct Recategorizd as higher Question re-written to match KA, enhance distractor credibility I remove cues. Page 2 ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other LOK LOD 0# (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 0= SRO Focus Dist. Link units ward KIA Only 29 H 30 F 31 H X 32 H 33 H 34 F X 35 F 36 F 37 H 38 F X 39 H X 40 H 41 F 42 H 43 F 44 H X 45 F 46 H X 6. 7. 8. B/M/N UtE/S Explanation M E Modified stem and choices for clarity. N E Reworded stem and distractors.

Added valve numbers. Modified key answer wording to match the reference document.

N U KIA mismatch and backward logic. Replaced with new question.

N E Modified question to require assessment of VA-10 loss consequences.

N S B E Reworded distractor to ensure plausibility.

Other minor editorial changes. M S N E Reworded distractors to improve balance. N S M E Reworded choices to improve distractor plausibility.

M E Minor modification to stem and editorial changes to choices to eliminate cueing for 0#40. M E Modified choices for symmetry.

N E Changed choice wording for symmetry and to ensure wrong choices are defensibly wrong. N E Modified Choice 0 to ensure distractor is wrong.

M S N E Modified choices to improve plausibility.

M S M E Modified distractors to improve balance, plausibility.

Page 3

--2 Form ES-401-9 6. 7. B/M/N U/E/S 1. 2. 3. Psychometric Flavvs LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Focus Dist 47 H 48 H X 49 F 50 F 51 H 52 H 53 F 54 H 55 H 56 F 57 H 58 F I 59 H I I I 60 H 61 H X 62 H 63 H 64 H 4. Job Content Flavvs 5. Other Job-Minutia #1 Back-Q= Link units ward KiA I 8. Explanation Recategorized as higher order. Modified choices to Reworded Choices A and C to prevent Corrected units in two Removed unnecessary information in stem and streamlined Corrected grammer in Choice Clarified question in stem to require sequential Reworded choices to ensure they are mutually Minor editorial Modified several distractors to clarify and to make wrong answers wrong Changed restoration time in stem from 6 days to 36 hours4.166667e-4 days <br />0.01 hours <br />5.952381e-5 weeks <br />1.3698e-5 months <br /> to ensure RWST could absorb the heat of the S FP. Modified stem and choices to clarify conditions and ensure plausible distractors.

Annotated question pedigree data to indicate reference to be provided for the question.

Corrected valve name and designator and modified choice wording to enhance symmetry.

Added ARP to list of question references.

Modified Choice C to enhance symmetry.

M N N N B N B B N B N N M B N N B M E E E S S E S E E S S S E E U S E E ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. LOK LOO Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/MIN U/E/S Explanation Focus Oist. Link units ward KIA Only Modified stem question statement to match syntax of the first three 65 H N E choices and modified the wording of Choice 0 to match the syntax of the other three choices. 66 H M S 67 H X B U KIA mismatch.

Replaced with a bank question.

I 68 H M E Modified Q and choices to improve plausibility and ensure it is not a direct lookup. 69 H B S 70 F X M U Modified choices to ensure choices are mutually exclusive.

71 F X M U Question had two correct answers. Modified question and choices. 72 F B E Reworded choices to eliminate psychometric flaws. 73 F X N U Correct answer too obvious. Stem setup not valid for plant conditions.

Could not repair. Replaced question.

74 F X B U Question tested minutia, was poor KIA match, had low operational validity and correct answer was cued. Replaced question.

I 75 F X N E Added to justification to better explain sequence of actions in stem. i 76 H N E Question modified to ensure technicai accuracy and Clarity. I 77 F X X N U Reworded question and choices to make distracters credible and to meet SRO-only criterion of cannot be answered solely with RO system knowledge.

78 F X X M U Modified the question and the answer choices to address the implausibility of 2 distractors.

79 H X N U Implausible distractors. Rewrote question.

KIA mismatch.

Implausible distractors.

Question and choices rewritten.

80 F X X N U New question is KA match because knowledge of AOP actions applied to determine system alignment as reflected by TS operability call. Page 5 ES*401 2 Form ES*401*9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. LOK LOD Q# (F/H) (1-5) Stem Cues TtF Credo Partial Job-Minutia #t Back-Q= SRO B/MIN UlEtS Explanation Focus Dist. Link units ward KIA Only 81 H X N U NOT SRO-only.

Could be answered by application of system knowledge.

Replaced with bank question.

82 F X X N U Implausible distractors and no correct answer. Question and choices modified.

83 F X X N U Implausible distractor.

and 3 correct answers. Modified stem and distractors to improve plausibility.

84 H X N U NOT SRO-only.

Rewrote the question and the choices. 85 H X X M U NOT SRO-only.

modified stem and choices. 86 H X N E Reworded stem and Choice B to enhance plausibility.

87 H X B U Multiple correct answers. Question replaced.

88 H N E Minor typographical corrections.

Added to distractor justification.

89 F X X N U Implausible distracters and not SRO-only.

Could be answered with system knowledge of battery function.

Modified stem and choices to make SRO-only and to make distracters plausible.

90 H X X N U Reworded distracters to make plausible.

91 H M E Reworded stem and choices to improve diScriminatory value. 92 F X X N U KIA mismatch.

Q was not related to NNI system. Distractor C was not a logical choice. Replaced with new question 93 F X N U Multiple correct answers. Based on Q wording ("should"), applicant could make a case for all choices as correct answers. Developed new question to improve distractor plausibility.

94 H B E Reworded stem and choices to improve plausibility.

95 H X X B U Replaced question, original did not match the KIA. 96 H X B U Question had correct answers. Rewrote stem and distractor.

97 H X N U Question was not SRO-only.

Could be answered with RO system knowledge.

Replaced the question.

Page 6

---

ES-401 2 Form ES-401-9 --_._._---

---_._.. _..........

_-----.. -.......------_._._.. _....... -1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. LOK Q# (F/H) (1-5) Stem Cues T/F SRO B/M/N utEtS Credo Partial JOb-J::Utia

1. 1 Back-Q= Focus Dist. Link units ward KIA Only I Question was not SRO-only.

Could be answered based on 98 F X N U application of system knowledge.

Replaced question.

! 99 H X B E Reworded implausible distractor.

I *100 F X B E Reworded choices to improve plausibility.

Page 7