ML13206A284
| ML13206A284 | |
| Person / Time | |
|---|---|
| Site: | Susquehanna  (NPF-014, NPF-022) |
| Issue date: | 03/13/2013 |
| From: | D'Antonio J Operations Branch I |
| To: | Public Service Enterprise Group |
| Jackson D | |
| Shared Package | |
| ML12347A266 | List: |
| References | |
| ES-401, ES-401-1, TAC U01868 | |
| Download: ML13206A284 (10) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401*9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOO (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- I Back-Q=
SRO B/M/N U/E/S Explanation Focus Oist.
Link units ward KIA Only 1
H 3
Y N
N S
2 F
2 Y
N B
S 3
H 3
X N
N B
E
- 1) Revised C&D distracters to increase plasuibility Changed to lO-sec TD to reflect 81% BUV scheme.
- 2) Does not appear to be a KIA match. The question is testing interlocks when transferring to the aux bus and away from offsite power. This question does not require any knowledge of electrical power supplies to off-site sources of power. Please explain your bases for the KIA match.
CE allowed licensee to retain Qsince revised Q appeared to be otherwise acceptable and selected KIA to match Q-K1.03 appeared to be a better match 4
F 2
Y N
B E
- 1)
- 2)
Changed to memory level not H Revised D be symmetrical to C, That way you have two div 1 and two div 2's to add balance to answer choices.
5 H
3 Y
N B
E List basis for KIA match along with KIA statement - not real obvious direct match. Kept Q and selected related KIA that better fit the Q with CE permission 6
H 2
X Y
N B
E C&O would be more plausible if the stem includes a LOCA condition such that the applicant s would then need to remember whether the low LO trip is bypassed for a LOCA condition. Q stem modified to add LOCA condition.
7 H
1-2 X
Y N
N U
Borderline LOO, distracters "C&O" are arguably not plausible -i.e, knowing that you must push both push buttons and that ECCS pumps are running. Consider revising ECCS pump arrangement to have Core Spray A and 0 combo. Q revised to raise LOD and increase plausibility.
8 H
3 X
Y N
B E
"0" answer is the longest choice and the only one that mentions the F013 valve. Added F013 to answer "A" choice to add balance.
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
0#
LOK LOD (F/H)
(1-5)
Stem Cues r/F Credo Partial Job-Minutia
- 1 Back 0= SRO B/M/N UlEtS Explanation Focus Dist.
Link units ward KIA Only 9
F 2
X Y
N B
E
- 1)
Typo in KIA "distribution" corrected
- 2)
Technical reference added to list the specific alarm.
- 3)
Revised stem to remove "automatic" which will make "8" plausible.
10 F
2 Y
N B
S 11 F
1-2 X
X Y
N N
U
- 1)
"A&B" distractors not plausible since knowing that the pumps will not trip on a loss of DC control power is very basic system knowledge.
- 2)
LOD borderline.
- 3)
Appears to be "F" rather than "H" Unless a loss of DC power could cause a loss of SID cooling then A&B not really plausilbe. Maybe as a suggestion to improving plausibility have RPV water level lower to 10"
- 4)
Also the correct answer "0" makes no mention of alternate DC which could cue the answer.
- 5)
Compare this to question 4. Especially if 4 B an 0 still mention RHR after 4 is revised. Seems like overlap.
Agree with other comments.
Rewritten based on NRC recommendations to resolve comments.
12 F
2 Y
N 8
S 13 F
2 X
Y N
N U
GFES 0 and 2 distractors not plausible. Replaced Q testing same KIA 14 H
3 X
Y N
8 E
- 1)
"8" auto returns to service doesn't seem plausible - what is the bases for plausibility. Revised "6" can be "manually placed in service if the isolation is bypassed
- 2)
Answer C, column (2), change "IF" to "if' Changes made 15 F
2 Y
N B
S 16 H
2 Y
N N
U Replace 0 overlaps with scenario #2, event #4 Resampled KIA due to overlap, new question on HPel isolation reset.
17 F
2 Y
N B
S 18 F
2 Y
N N
S
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
0#
LOK LOD (F/H)
(1-5)
Stem Cues TlF Credo Partial Job-Minutia
- 1 Back 0= SRO BIMIN UlElS Explanation Focus Dist.
Link units ward KiA Only 19 H
2 Y
N B
S 20 H
2 Y
N B
E Choice "A" editorial added comma in 40,000.
21 F
3 Y
N N
S 22 F
2 X
Y N
B U
B&D not credible with only one channel affected single failure criteria and shorting links installed. Q revised 23 H
2 X
Y N
B E
A&C Distracters not plausible with even minimal system knowledge of shutoff head.. I think this Q could be improved by including the core spray injection valves open or closed..
The set point for opening the valves is <420. The reference material did not say what the shutoff head was. If add injection valves in the answer then they need to differentiate between if valves are open and if below shutoff head. At least they need to know two things then.
Revised distractors 24 F
2 Y
N N
S 25 H
3 Y
N N
S 26 H
3 Y
N N
S 27 H
3 Y
N B
S 28 F
2 Y
N N
S 29 H
3 Y
N N
S 30 F
2 Y
N N
S 31 F
2 X
Y N
N U
- 1)
A&C not plausible that cld rate decreases or temp increases when cooling thru HTX increases.
- 2)
Additionally C is not consistent. It says loop temperature will rise. The other 3 answers all have cooldown rate.
Adjusted distractors to reflect expected effects of manipulations, ifthe system flowpath was actually a,.,
reflected in the distractor, to make plausible.
Corrected, all distractor now symmetric.
32 H
3 Y
N N
S
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues TfF Credo Partial Job-Minutia
- 1 Back-Q=
SRO BIM/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 33 F
2 Y
N N
S 34 H
3 Y
N N
E Revised liD" to make more plausible - restore AlE byplacing alternate SJAE on line.
35 F
3 Y
N N
E Changed to "F" rather than "HI' function ofcontinuous insert PS 36 H
1-2 X
Y N
N U
- 1)
A&B not plausible with FW temp increasing with a loss of steam.
- 2)
LOD borderline.
Rep/acedQ 37 F
1-2 x
Y N
B U
- 1)
A&B not plausible If you know the auto scram points which is very fundamental.
- 2)
Answer obvious LOD=1 Removed reference to automatic scram on MSL rad, added distractor on scram on MSIVposition with mode switch in Startup. Adjusted distractors to require knowledge ofMVP and MVP suction valve response.
38 F
2 Y
N B
S 39 H
2 Y
N B
E Revise "C" to make plausible make first part read same as "An and for second part make HPCI available and RCIC not available for ED Change made.
40 H
3 X
Y N
N U
GFES and 2 implausible distracters. Replaced Q testing same KIA 41 F
2 Y
N B
E Revised A&S to make more plausible revised to 9% decay heat.
42 H
2 X
Y N
N E
The second part of the Q is borderline minutia considering the big picture. Revised X'd part of Q 43 H
3 Y
N B
E Extra space before FSH in answer "on. Corrected.
44 F
3 Y
N B
S 45 F
2 Y
N B
S 46 H
2 Y
N N
S
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F CreeL Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N UtEtS Explanation Focus Dist.
Link units ward KIA Only 47 H
2 Y
N B
S 48 H
2 Y
N N
S 49 H
3 Y
N N
S 50 H
3 Y
N N
E In the explanation section for "An third line "ands" Corrected 51 F
2 Y
N B
S 52 H
2 Y
N B
S 53 F
2 Y
N B
S 54 F
2 X
Y N
B E
- 1)
"F" rather than "H" Changed
- 2)
"C" not plausible. Revised.
55 F
3 Y
N B
E
.oF" rather than "H" Revised 56 H
3 Y
N B
S 57 F
2 Y
N B
S 58 H
2 Y
N B
S 59 H
2 X
X Y
N B
U Distractors not plausible. Revise distracters or replace Q.
The answer is the only choice that exceeds dose limit cues answer. Consider revising "A". Consider revising "A" to read Zone 3 HV AC and may exceed dose limits Changed A distractor to exceed does limits with SGTS. If changed to read Zone 3 and exceed limits, believe that to be a 2nd correct answer.
60 F
2 Y
N N
S 61 F
3 Y
N B
E
.oF" rather than.oW Revised 62 H
3 Y
N B
S 63 F
3 Y
N N
E uFo rather than "H" Revised
a#
- 1.
LOK
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
(F/H)
(1-5)
Stem Cues TtF Credo Partial Job-Minutia
- /
Back-a=
SRO a/MIN utE/S Explanation Focus Dist.
Link units ward KIA Only 64 H
3 Y
N N
S 65 H
2 Y
N N
S 66 F
1-2 X
Y N
a E
"Au not credible consider revising to say "Acknowledge Alarm ONLY Revised 67 F
2 Y
N 8
S 68 H
2 Y
N N
S 69 H
2 Y
N 8
S 70 H
2 Y
N N
E KIA statement spelling error "potable" rather than "portable" Revised 71 H
3 X
Y N
N U
GET a and 2 implausible distracters. Replaced Q testing same KIA 72 H
3 Y
N 8
S 73 F
3 Y
N N
E Delete word 'today" it is distracting. Revised 74 H
3 Y
N a
S 75 H
3 Y
N 8
E Typo in answer "0" explanation delete "is" after word "were" Revised 76 H
3 X
Y Y?
B U
- 1)
This a can be answered with RO system level knowledge.
ROs should know must secure "C' EDG with cooling water and that you would not want to reset the "8" EOG lockout relay. Also that you would have no cooling to "E" EOG if you tried to substitute "En for "C* EOG. 2) The stem for "0" tripped on low L.O. may cue answer to RO Q #6. 3)
Why can't you consider substituting HE" for "0" EOG rather than for "8"?
77 H
3 Y
Y N
E CE allowed licensee to retain Q since Q appeared to be otherwise acceptable and selected XI A to match Q EA 2.02 78 H
3 X
Y Y
8 E
"C" distractor not plausible to flood Prim Containment with an injection source available. Revised "8" distractor.
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H) (1-5)
Stem Cues TfF Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/MIN UlEfS Explanation Focus Dist.
Link units ward KIA Only 79 H
3 Y
Y B
E Unsure why it is plausible to shut down Zone 1 ventilation when only Zone 3 is affected. Add "Containment Isolation" to the end of answerC 80 H
3 Y
Y B
E The correct answer is the only one making mention of MSCP, perhaps add a second?
81 H
3 Y
Y N
S 82 H
3 Y
Y N
E Remove reference to ADS solenoids, just state that the SRV are inoperable, let them determine if they are ADS valves. Possibly add in other TS requirement times (for regular SRV inop) as distracters.
Added missing word "be" in question. Specified applicant reference TS to be redacted.
Distractors C and D are the TS timesfor an SRV inop.
Failure changed to "pneumatic actuator" to eliminate cue ofADS status.
83 H
2 Y
Y?
N E
- 1)
The first part of the Q is RO system level knowledge and
- 2) if the answer to the second part of the Q is an operator immediate action in the ON then this would be RO level and nor SRO Jevel.
It is RO and pretty easy.. Alii need to know is the 6 inch setpoint for the recirc rundown. Since the first part has four unique answers I can pick "c" and I don't even need to know the answer to the second part 84 H
3 Y
Y N
E Updated explanation for "D" 85 H
3 Y
Y B
86 H
3 X
Y Y
N E
For "B" change from +45" to +38" to make more plausible and balanced with D. as D only has level lowering from +35" to +32" Adjusted "B&D levels to better reflect OB procedure.
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOO (F/H) (1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO BIMIN UtE/S Explanation Focus Oist.
Link units ward KIA Only 87 H
2 X
Y Y?
N U
Explain why this an SRO level knowledge Q. 1) It is pretty fundamental RO knowledge to know that level needs to be restored to at least 45" to aid in natural circulation 2) It could be argued that 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> TS are RO level knowledge. We believe this would be an ok RO Q but not SRO. Revised distracters to make more appropriate SRO level.
88 H
2 Y
Y?
N U
Explain why this an SRO level knowledge Q. It appears to be able to be answered with system level knowledge. It would seem to be pretty fundamental system knowledge to know you would expect the EOG should pick-up the bus and you don't need to start ESW.
If I know how the EDG is going to respond then I have narrowed it down to A or B. And if I know that ESW auto starts in 40 seconds I know A is wrong. That leaves B which has to be correct even though I have no Idea if that procedure directs those actions. So it definitely is not Knowledge of abnormal condition procedures, which is the KA. Revised Qand answers to make SRO level.
89 H
3 Y
Y B
S I assumed that both unit 1 systems were inop and I looked at the 90 H
2 Y
Y N
E TS. I would have picked Restore RHRSW valve HV-11215A OR RHRSW Pump 1 B OPERABLE within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> if that answer was available. The correct answer is based on the fact that the HV 11215A failure prevents aligning RHRSW Pump 2A to the RHR 1A HX, therefore the 8-hour Completion Time applies, otherwise It would be 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />.
Recommend the following revisions; Revise "B" to within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> vice 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br />. Revise "C' to read within 7 days vice 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> and revise "0" to read "Restore UHS or be in Mode 4 within 36 hours4.166667e-4 days <br />0.01 hours <br />5.952381e-5 weeks <br />1.3698e-5 months <br />."
Revised Qand distracters to address comments.
91 H
3 Y
Y N
E Revised stem condition ofRx power to increase to 23% which makes this a little more challenging. Proposed answer "e" does not match listed answer-Typo.
92 F
2 X
Y Y
B U
A, B, and C aren't really plausible. A) The rod is not exhibiting double notching. B) Surveillance steps are not re-performed as this would constitute preconditioning. C) If failing a surveillance test had no required actions, why would it be performed?" Revised distracters.
93 H
3 Y
Y N
S
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
0#
LOK LOD (F/H) (1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-0= SRO B/M/N UtE/S Explanation Focus Dist.
Link units ward KIA Only 94 H
3 Y
Y B
S 95 H
3 Y
Y B
S 96 F
3 Y
Y B
S 97 F
3 Y
Y B
S 98 H
3 X
Y Y
B U
The designated answer appears to be incorrect see TS bases on page TS/B3.8-32 section 3.8.1.18 it appears that once the knife switch is opened for 1C RHR that the EDG would be operable and 1C RHR would be inop.
Recommend that you revise the stem condition for CS 1A pump to 14.4 seconds and remove all TS reference for TS 3.5.2, 3.8.1 and 3.8.2 in the stem. The answers would then read as follows:
A.
Declare 1C RHR & 1A CS pumps inop ONLY B.
Declare A&C EDGs inop ONLY.
C.
Declare C EDG and 1C RHR pump inop ONLY D.
Declare A&C EDG and !C RHR & 1A CS pumps inop Recommended changes made.
99 F
2 Y
Y N
S 100 F
3 Y
Y N
E Added "only" after choices "A, B, and e"
Instructions
[Refer to Section 0 of ES-401. and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; Single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section 0.2.f.
- 7.
Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).