ML20024E750
| ML20024E750 | |
| Person / Time | |
|---|---|
| Site: | Ginna  |
| Issue date: | 07/23/2019 |
| From: | NRC Region 1 |
| To: | Exelon Generation Co |
| Shared Package | |
| ML18347A730 | List: |
| References | |
| EPID L-2019-OLL-0039 | |
| Download: ML20024E750 (10) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
a LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Distractors C1 and D2 indicate PRT is at a vacuum. Not 1
H 2
X B
u operationally valid. Changed pressures to indicate saturations temoeratures above 212 /10 osia) 2 H
3 N
E Changed "re-opening" to "MCB operation to re-open" in 2nd part of question Randomly reselected a new KIA and rewrote question.
3 F
2 B
E Changed "lists" to "states" in stem. Changed "of the loss of CCW to "Based on the plant conditions" at end of question statement 4
H 2
B s
5 H
3 B
E Removed "for operation of RHR pumps" from stem Removed statement in stem about Pzr pressure lowering, and changed stem question to "Which one of the following describes 6
H 2
B E
the plant response" Added NOTE for PT-429 nomenclature and removed wording from answer choices 7
H 2
M E
Swapped order of 2nd and 3rd stem bullets 8
F 2
B s
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOO) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
"Cues": The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F": The answer choices are a collection of unrelated true/false statements.
"Cred. Dist.": The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
"Partial": One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
"Job Link": The question is not linked to the job requirements (i.e.. the question has a valid KIA but, as written, is not operational in content).
"Minutia": The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e.* it is not required to be known from memory).
#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
"Backward": The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those Kl As that are designated "SRO-only." (KIA and license-level mismatches are unacceptable.)
- 6.
Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section 0.2.f.
- 7.
Based on the reviewer's judgment. is the question. as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only "Boron concentration significantly lower" cues candidate that A and Care not plausible.
9 F
2 X
X N
u Changed 30 min to 60 minutes in A 1 and B1. and removed 4th stem bullet about boron concentration.
Deleted 4th stem bullet 10 H
3 B
s 11 F
3 B
s H
2 X
M u
KIA mismatch - This question tests rules of EOP usage, not 12 AFW. New question written H
2 N
s Rewritten question is SAT Answer seems obvious given a transfer is stated to have occurred and alarm is in (A1 and C1 is not plausible) 13 H
2 X
B u
Changed stem to say a failure to O VAC occurred (removed transfer)
Changed 2nd stem bullet to "Secondary EO reports 'B' Inverter output voltage is O VAC and the Inverter Static Switch is in BYPASS" 14 F
2 B
s 15 H
2 M
E Will provide multiple references to choose from (0-6.9, Attachments 2 - 10).
16 F
2 B
E Change distractor D to "RCS pressure and Pressurizer Level" Added "ONLY" to end of answer choices A and B 17 H
3 B
E Reworded 3rn bullet to "'B' CCW Pump and 'B' SW Pump are OOS for repairs" A2 and C2 are subset for distractors B2 and 02, making C a B
u correct answer as well.
H 2
X New question written 18 H
3 M
s New question is SAT
ES-401 3
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only 19 H
3 B
s Change second bullet to say "automatic charging pump 20 H
2 B
E controller fails to output of 100%". Correct answer changed to'C' 21 F
2 B
s 22 F
2 N
s 23 F
3 M
E Added "Aux Building" prior to "radiation levels" in 1st part of stem Clarified 'A' plausibility statement that R-9 is an Area 24 F
2 N
E Radiation Monitor Added P-9 as a reference document A2 and C2 not plausible - continuing cooldown (adding 25 F
2 X
N u
positive reactivity) would not stop source range from rising.
Chanaed part 2 in stem to 'What action is reQuired" 200 part is SRO level (bases question) 26 F
3 X
N u
Chanaes second oart of auestion to be RO level Modified times to make more realistic to get to ES 0.2 27 H
3 N
E Changed offsite power is tripped to "is lost" Added stem bullet for RCS borated to CSD Chanaed answer choice 'A' to "12" 28 H
3 N
s 29 F
2 B
E Changed given PRZR level in stem to 100%
Deleted 5th bullet in stem Changed "piping elbow downstream of V-543" to "valve 30 H
2 N
E packing of V-543" in last stem bullet Revised 2nd question statement to "Following CNMT entry, the leakage is classified as leakage" 31 F
2 B
s 32 H
2 B
s
ES-401 4
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only B2 and D2 are not plausible due to single failure criterion 33 H
1 X
M u
Added stem bullet for 'C' SIP MOV failure to failed SI line Clarified 2nd part of question to specify which SI line fails Added "an RCS Hot Leg" prior to LOCA in 1st stem bullet Changed answer choices 'A(2)' and 'C(2)' to "prevent thermal 34 F
2 M
E shock to PRT spray nozzles" Added "in accordance with applicable plant procedures" to end of 1st part of stem question 35 F
3 N
E Changed 4th bullet to stable level 36 H
3 M
E Deleted "<" signs 37 H
3 N
s 38 F
3 M
s 39 H
2 N
E Removed first part of stem with regards" statement.
BOLDed "Both" in answer choice *c*
40 F
2 N
E BOLDed "None" in answer choice 'D' Deleted AOV-427 stem bullet 41 F
2 B
s 42 F
3 B
E Deleted stem information Changed COG level to LOW Distractor C not plausible.
Changed answer choice 'C' to "BA FCV fails to a higher 43 H
2 B
E setting during normal makeup" Changed answer choice 'D' to "EO places S/G blowdowns in service at 60 gpm" Problem with justification of distractor b stating "stable at 340 psig".
C1 and D1 not plausible (rising level) due to FRVs are in 44 H
1 X
N u
Manual as stated in stem, and pumps are lowering.
Changed answer choices 'C(1)' and 'D(1)' to "stable" Added "FIRST" prior to "automatic action" in 2nd part of stem auestion
ES-401 5
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only A1 and B1 not plausible - stem says no AC power 45 H
2 X
N u
Changed 1st stem bullet to "100/0 Electric plant lineup" Added bullet "loss of Offsite Ckt 767" Added stem bullet "Offsite Power Circuit 7T is out-of-service for maintenance" 46 H
2 B
s 47 H
3 M
E Removed statement about B EOG powering bus 16, as this informs question 57.
Revised question and answer formatting/wording 48 F
2 B
E Added 'WILL/WILL NOT" following "Buses" in question statement Changed "MCCs" to "MCC loads" in auestion statement 49 H
3 N
E Added bullet at end "'B' EOG becomes available" Changed 2nd stem bullet time to "one minute" 50 F
3 B
s 51 F
3 B
s 52 F
2 B
E Changed "valves" to "MOVs" throughout question and answer choices "Plant trips due to a SGTR" is not operationally accurate. B2 and 02 not credible.
Changed "Plant trips" to "Operating crew trips the plant" in 53 H
3 X
N u
2nd stem bullet Changed answer choices '8(2)' and '0(2)' to '1o allow operation of PRZR PORVs" Revised plausibility statements 'B' and 'D' Added bullet for 'A' EOG loaded onto Safeguards Buses 54 F
3 M
s Changed 2nd question statement to "the design function 55 H
3 B
E accomplished by a Cl signal" Deleted 4th stem bullet H
1 X
B u
SRVs are never isolable per ASME Code. New question 56 written F
3 B
s New auestion is SAT with the followina chanaes made:
ES-401 6
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Added from overpressure" following "plant protection" in question statement Changed "No effect" to 'The plant will not exceed the RCS Pressure Safety Limit" in answer choices 'A' and 'B' Added "provided Pressurizer Safety Valve V-434 operates as designed" to end of answer choice 'D' Changed "plant protection from overpressure" to the Ginna Accident Analysis" in question statement Added stem bullet "Pressurizer Safety Valve V-434 will operate as required" Revised answer choices 'A', 'B', and 'C' Revised plausibility statements 'A' and 'B' 57 H
2 X
M u
Question #47 stem tells me B EOG powers bus 16, so B or D are correct (a and c not plausible). Edited question 47.
B and D would only be plausible if candidate did not know that there was an A EOG.
Added stem bullet '"A' EOG is being prepared for post maintenance testing and is INOPERABLE and available Changed 'B' EOG stem bullet to make unavailable 58 H
1 X
N u
Added bullet "Preparations are being made to shutdown the plant iaw ITS 3.0.3" Changed last stem bullet to "Offsite Power was lost one minute ago Changed "aligned" to "in its normal alignment" in 3rd stem bullet for 'A' EOG status Chanaed "Fuel Pump" and "Offsite Power to lower case 59 F
2 B
s a1 and b1 not plausible Replaced answer choices 'A' and 'B' Deleted 1st part of question and answer choices 60 H
2 X
N u
Revised plausibility statements Revised question statement wording Revised answer choices so that each is a single action Revised olausibilitv statements 61 F
3 B
s F
2 X
N u
KIA Mismatch -,question does not test FHS/RCS. New question written 62 F
2 N
s New auestion Is SAT
ES-401 7
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only 63 H
2 N
E Changed 2nd stem bullet to "Both MSIVs spuriously CLOSE" Added will" prior to "stabilize" in stem question statement Added stem bullet "'A' Gire Water Pump was started prior to 64 H
2 N
E the release beginning" Added "'A' Gire Water Pump and" following "if' in 2nd auestion statement 65 F
3 B
s 66 F
2 B
s Choice 2 in stem is implausible, which leads to only choice as right answer.
67 F
3 N
E Changed item #2 following stem question to "RCS TAVG" and plausibilitv statements 68 F
2 B
s 69 F
2 B
s F
1 X
B u
Answer is obvious, new question written 70 New question is SAT with the following change:
F 2
N s
Added "in accordance with A-3, Containment Vessel Access Requirements" to end of last stem bullet 71 H
3 B
E B and D justifications edited for clarification 72 F
2 B
s 73 F
3 N
s H
3 X
N u
KIA mismatch, new question written 74 New question is SAT with the following change:
F 2
N s
Changed "Satellite" to "Cellular and revised plausibility statement 'D' 75 F
3 N
s
ES-401 8
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Deleted "1331" bullet Removed stem bullet regarding HCO actions 76 H
2 X
B E
Changed question statement to "If the given plant conditions remain unchanged, which ONE of the following are the reauired action(s) to be taken IMMEDIATELY?"
Deleted last stem bullet 77 H
3 X
N E
Added "prior to exiting the appropriate AP" to end of 2nd part of stem question KIA mismatch because Q does not address FR-S.1 entry H
3 X
B u
conditions. New question written.
78 H
3 N
s New question is SAT Changed "EMD" to "Electrical Maintenance" in last stem bullet 79 H
3 X
N E
Added To comply with ITS," to beginning of 2nd question statement KIA mismatch Provided 0-6.9, Attachments 2 -10, as reference provided Changed MVARS to "+280" from "+260" in 4th stem bullet Revised 1st part of question statement to "The Main Generator Voltage Regulator WILL / Will NOT trip to MANUAL after a time delay" 80 F
2 X
N u
Revised 1st part of answer choices to 'WILL" or 'WILL NOT" Revised 2nd part of question statement to "IF the Main Generator voltage regulator trips to MANUAL, THEN RG&E Energy Control Center must be notified within to meet a North American Electric Reliability Corporation (NERC) requirement" Changed correct answer to 'A' Revised olausibilitv statements KA mismatch because Q does not address loss of recirculation. New question written.
F 2
X B
u New question is still UNSAT - is not written at SRO level.
81 H
2 N
s Final question is SAT 82 H
3 X
N E
Added bullet "l&C reports stationary coil fuses for control rod G-3 are blown"
ES-401 9
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Changed 4th stem bullet to "Bank 'D' control rod G3 is indicating 216 steps on MRPI" Added "and its" prior to "Bases" in stem question statement Changed N-42 failure to "high" in stem bullet Swapped order of 6th and 7th stem bullets Added '1he" prior to "safety analysis" in 2nd question statement Revised plausibility statements to clarify difference between 12 steos (ITS limit\\ and 25 steps (safety analysis)
Deleted "In accordance with FR-S.1 and ER-CVCS.1" in stem question 83 H
2 X
B E
Added "in accordance with FR-S.1" to end of answer choices
'B' and 'C' Added "in accordance with ER-CVCS.1" to end of answer choices 'A' and 'D' 84 H
2 B
s 85 H
2 B
s 86 H
3 M
s No indications of a LOCA in the stem, so B2 and 02 are not plausible.
87 H
2 X
N u
Changed "ES-1.2" to "ES-0.1" in answer choices '8(2)' and
'D(2)' and revised plausibility statements 88 H
2 X
N E
Changed "FT-477" failure to "low" Deleted statement prior to stem questions Added "in accordance with ER-Fl RE.1" to end of answer choices 'A(1 )' and 'C(1 )'
Moved valve nomenclature from answer choices to NOTE 89 H
2 X
N E
Added "Close V-4002 and" to beginning of 2nd part of answer choices 'A' and 'B' Changed CST level in stem to "3 feet" from "4 feet" Added stem bullet for "TDAFW flow oscillating" Added "first" to 1st question statement Changed 2nd stem bullet to "IAC 'C' has tripped and can 90 H
2 N
E NOT be restarted" Revised 3rd stem bullet to "Secondary EO reports a large air leak on the Instrument Air Drver 'B' Prefilter housina"
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only 91 H
2 B
E Deleted AFW flow stem bullet 92 F
3 N
s 93 H
2 N
s 94 F
2 N
s KA mismatch as Q does not address conservative decision F
3 X
B u
making. New question written New question is SAT with the following changes:
Changed "2360" to "2380" in answer choice *o*
95 Revised question statement to be in accordance with OPG-OPERATIONS-EXPECTATIONS Replaced answer choices 'A' and 'B' F
2 N
s Re-arranged answer choices for psychometrics Revised plausibility statements Question is not SRO level - only requires general knowledge of tagging process 96 F
3 X
B u
Edited question to include testing of SRO approval level in the tagging process.
97 H
3 B
s 98 F
3 B
s 99 F
1 N
s Deleted "Approval of' in answer choice 'B' 100 F
3 B
E Changed answer choice *c* to "Authorization of emergency exposure controls in excess of 5 Rem TEDE" to match wordina in EP-AA-1000