ML12339A158
| ML12339A158 | |
| Person / Time | |
|---|---|
| Site: | Limerick  |
| Issue date: | 11/15/2012 |
| From: | Wasong A Exelon Generation Co |
| To: | Caruso J Operations Branch I |
| Jackson D | |
| Shared Package | |
| ML120230224 | List: |
| References | |
| TAC U01858 | |
| Download: ML12339A158 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Q#
- 1.
LOK
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
(F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N utEtS Explanation Focus Dist.
Link units ward KIA Only 1
F 3
Y N
M S
Minor editorial changes to Q stem 2
H 4
Y N
N S
Minor editorial changes to Q stem and answer choices 3
H 4
Y N
B S
Minor editorial changes to Q stem 4
H 3
Y N
B S
Original question replaced to more closely fil selected KIA 5
H 4
Y N
N S
Technical change to Q stem related to RETS pressure setpoint and added partial reference to include scram logic and MSV logic prints based on validation comments.
6 F
3 Y
N N
S Minor editorial changes to Q stem 7
H 4
Y N
N S
Minor editorial changes to Q stem 8
F 3
Y N
N S
Minor editorial changes to reflect plant labeling 9
H 4
Y N
N S
Technical changes to Q stem to identify minimum RPV level to prevent Thermal Stratification and determine if adequate DHR exist based on licensee comments. Maintained original KIA and KIA topic 10 F
3 Y
N B
S Minor editorial changes to Q stem 11 H
4 Y
N N
S Minor editorial changes to Q stem and answer choices 12 H
3 Y
N B
S Minor editorial changes for stem clarity 13 H
4 Y
N N
S Minor editorial changes to Q stem 14 F
3 Y
N M
S Changed the part of question stem from determining the availability of RECW system to identifying minimum required suction pressure of DWW system based on validationllicensee comments.
15 H
4 Y
N B
S Minor editorial changes to Q stem 16 H
4 Y
N M
S Minor editorial changes for stem clarity 17 H
4 Y
N M
S Minor editorial changes to Q stem 18 F
3 Y
N M
S Minor editorial changes to Q stem
a#
- 1.
LOK
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
(F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-a=
SRO B/M/N UlE/S Explanation Focus Dist.
link units ward KIA Only 19 F
3 Y
N N
S Minor editorial changes to a stem 20 H
3 Y
N M
S Minor editorial changes 21 H
4 Y
N M
S Changed answer choices to indicate pressure trend of "Steady" or "Cycling". Also made minor editorial changes to a stem 22 H
4 Y
N N
S Changed the a stem to add Hotwell and Equipment Drain collection capacity information, and made minor editorial changes to Q stem and answer choices 23 F
3 Y
N N
S Changed the a stem to ask the applicant response of OW unit cooler fans as a result of large steam leak based on validation comments. Changed Q answer choices to reflect the technical stem change 24 H
4 Y
N N
S Minor editorial changes to Q stem 25 H
4 Y
N M
S Changed the question to test required actions associated with DWCW momentary loss of power restoration actions based on validation comments. Changed a answer choices to reflect the technical stem change 26 H
4 Y
N M
S Added embedded reference (portion of SP/L leg ofT-102) to a and made minor editorial changes to answer choices. Changed a KIA topic from EK3.02 to EK2.01 (based on licensee commentsl additional review determined to be better fit).
27 F
3 Y
N N
S Minor editorial changes to a stem 28 F
3 Y
N M
S Minor editorial changes to a stem 29 H
3 Y
N M
S Stem edited for clarity and to streamline question 30 F
3 Y
N N
S Minor editorial changes to a stem and answer choices 31 H
4 Y
N B
S Minor editorial changes to a stem 32 H
4 Y
N M
S Minor editorial changes to a stem 33 H
3 Y
N M
S Minor stem edits for clarity 34 H
3 Y
N B
S Minor editorial changes for stem clarity
- i.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N utEtS Explanation Focus Dist.
Link units ward KIA Only 35 H
3 Y
N B
S KIA changed to reflect original selection, incorrect KIA documented on draft question 36 F
3 Y
N M
S Minor stem edit to enhance clarity 37 F
3 Y
N N
S Removed the reference of shorting links to Q stem 38 H
3 Y
N N
S Minor edits to stem for clarity and answers edited to more clearly reflect the concept being tested 39 F
3 Y
N N
S Minor editorial changes to stem 40 H
4 Y
N N
S Editorial changes to stem for clarity 41 F
3 Y
N M
S KIA resampled to EK4.04 and question replaced 42 H
3 Y
N N
S Stem re-written based upon licensee feedback and distracters modified to suit 43 H
4 Y
N M
S Minor editorial changes to Q stem 44 F
2 Y
N B
S Editorial changes to distracters to more accurately reflect terminology 45 H
4 Y
N M
S Minor editorial changes for stem clarity 46 F
2 Y
N N
S No changes 47 F
3 Y
N M
S Minor editorial changes to answer choices 48 H
3 Y
N M
S Editorial changes to stem for clarity 49 F
3 Y
N M
S Minor editorial changes to stem 50 F
2 Y
N M
S Editorial changes to timeline for stem clarity 51 F
3 Y
N N
S Stem and distracters edited to more clearly reflect the concept being tested 52 H
4 Y
N M
S Changed the entire question to better match KIA due to original question being detailed knowledge towards instrument air compressors.
53 H
3 Y
N N
S Minor stem changes for clarity
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 54 F
3 Y
N N
S No changes 55 F
3 Y
N B
S Minor editorial changes to Q stem 56 H
3 Y
N B
S No changes 57 F
3 Y
N M
S Minor editorial changes to answer choices 58 H
3 Y
N N
S Stem conditions reorganized for clarity 59 F
3 Y
N N
S Revise KIA topic to better reflect Q content while meeting sample plan requirement. Changed the bus for over current conditions and revised Q stem and answer choices to reflect the bus change. Other minor editorial changes to Q stem and answer choices.
60 H
4 Y
N M
S Minor changes in stem for distracter plausibility 61 F
2 Y
N N
S Minor editorial changes to distracters 62 F
2 Y
N B
S Minor editorial changes for technical accuracy 63 H
3 Y
N N
S Stem and distracters edited to more clearly reflect the concept being tested 64 F
3 Y
N B
S Changed the entire question to a bank Q that better matched KIA, due to original Q LOD too high based on requal exam open reference question source.
65 F
3 Y
N N
S Minor editorial changes to Q stem 66 F
2 Y
N N
S No changes 67 F
3 Y
N N
S Minor editorial changes to Q stem 68 H
4 Y
N N
S Minor distracter edit for technical accuracy and clarity 69 H
3 Y
N M
S Minor editorial changes 70 H
3 Y
N N
S No changes 71 F
3 Y
N N
S Minor editorial changes to Q stem 72 F
2 Y
N B
S KIA replaced due to low LOD of original KIA and question 73 F
2 Y
N N
S Minor editorial changes to stem
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
lOK lOD (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- /
Back-Q=
SRO B/M/N UlE/S Explanation Focus Dist.
Link units ward KIA Only 74 H
4 Y
N N
S Revised Q stem to incorporate Trip procedure into answer choices, and revised initial conditions to better reflect answer choices 75 H
3 Y
N N
S No changes 76 H
4 Y
Y N
S Changed Q stem to identify the 8T for 125 VDC battery and provide Terminal voltage and float current information to determine the TS actions. (based on licensee/validation comments) 77 H
4 Y
Y N
S Minor editorial changes to Q stem 78 H
4 Y
Y N
S Minor editorial/format changes to Q stem. Q concept remained unchanged.
79 F
3 Y
Y N
S No changes 80 H
4 Y
Y N
8 Minor editorial changes to Q stem 81 H
4 Y
Y N
S Minor editorial changes to answer choices 82 H
4 Y
Y N
8 Added embedded reference of partial T-101 RC/P leg, changed SBlC failure, deleted SlC level information, and added IRM and Control rod information following 30 min to Q stem.
83 H
4 Y
Y B
S No changes 84 H
4 Y
Y N
S Minor editorial changes to Q stem 85 H
4 Y
Y N
S Changed site boundary dose assessment value to 101 mREM and deleted the reason for dose assessment performance question statement. Changed the question statement to identify the procedure that required dose assessment. Changed Q stem and answer choices to reflect the technical change.
86 H
4 Y
Y B
S No changes 87 H
4 Y
Y M
S Changed from Unit 1 to Unit 2 in stem 88 H
4 Y
Y M
S Changed Core Spray Pump discharge pressure from 140 psig to 130 psig based original value being too close to setpoint for ADS logic. Also made minor editorial changes to Q stem 89 H
3 Y
Y M
S Stem conditions edited to clarify question and distracters Simplified
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues TlF Credo Partial Job-Minutia
- 1 Back-Q=
SRO BfMfN UfE/S Explanation Focus Dist.
Link units ward KIA Only 90 H
4 Y
Y M
S Stem edited to add initial conditions and distracters edited to reference included reference 91 H
4 Y
Y B
S Stem edited to add more specific initial conditions 92 H
3 Y
Y N
S Stem conditions edited to remove ambiguity 93 H
4 Y
Y M
S Minor editorial changes to Q stem 94 F
3 Y
Y M
S Question replaced due to lack of clarity in procedure 95 H
3 Y
Y M
S No changes 96 F
3 Y
Y B
S Changed question focus within same procedure and provided reference 97 F
3 Y
Y N
S Changed the Q stem to better reflect LS-AA-104 procedure 50.59 changes examples. Q concept remained same.
98 F
3 Y
Y B
S No changes 99 H
3 Y
Y M
S No changes 100 H
4 Y
Y N
S Minor editorial changes to Q stem
(\\
, )
-1)<£ v'~ \\.v ~
j)~cL.t,,"!::!,, -rilod-'!l'If,J1! V i4 C"t r..J '1r~III'~rJ:~ S'd.."rmi ij!)~ of tJ!lzd(
/v...... (j't1?lall ~,'III,i)j7/J.J /5;; i-J(b,,") '"/1;h',;!,,c,J 4I1ft'lf UNlIe< L J--p,iVN c/< vdr,J.. 0 tI4. !hI!r (!.ft/j) "",nJ:' II iGII 1/1} il 0/1h".J ;1 II(fA/J~t &'/lj.ilf,k'flJ t/ Ilt.icL..dl~p) t1 ( /J/I/rrsoCr/lM {'t'dt;I-1~r4:J I
]JlMIS()
t1ttrIf t;;t /rM/A/tP'L J
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either {F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (I.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible: single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (I.e., the question has a valid KIA but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (I.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.1.
- 7.
Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any 'U" ratings (e.g., how the Appendix B psychometric attributes are not being met).