ML17079A510
| ML17079A510 | |
| Person / Time | |
|---|---|
| Site: | LaSalle  |
| Issue date: | 11/16/2016 |
| From: | NRC/RGN-III |
| To: | Exelon Generation Co |
| Shared Package | |
| ML15274A408 | List: |
| References | |
| Download: ML17079A510 (9) | |
Text
ES-401 Written Examination Review Worksheet (LaSalle 2016 ILE)
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q =
K/A SRO Only 1
F 2
B S
2009 LORT Annual Exam 2
H 3
N E
NRC: Add reset L.O. relay to distractors B & C LIC: Comments incorporated.
3 H
3 N
S 2x2 4
H 2
X N
U E
NRC: Stem is asking if a manual scram is required but by LOA-TG-101 scram or insert rods is allowed, so scram is not actually required. Also, awkward wording concerning pressure response.
LIC: Pedigree enhanced to address concerns.
5 F
2 N
S 6
F 2
M N
S E
NRC: Need to see original question (M vs B).
LIC: Pedigree enhanced to correct source as New.
7 H
3 N
E 2x2 NRC: When would bearings only be plausible?
LIC: Pedigree enhanced to add information.
Instructions (Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
Partial: One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- / Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q =
K/A SRO Only 8
F 2
X N
U E
NRC: K/A deals with loss of SDC and RWCU.
Question is unrelated to loss of SDC.
LIC: Pedigree enhanced to address concerns.
9 H
2 B
E NRC: Is a reference provided during the exam?
LIC: No - Pedigree corrected.
10 F
2 X
M B
E NRC: Distractor A is implausible because SRV tail pipe level limit is unrelated to containment pressure.
2012 NRC Exam 11-1 question (RO Question 57) is essentially the same (Bank)
LIC: Pedigree enhanced to correct source as Bank.
Plausibility supported by validation results.
11 H
3 N
S 12 H
1 2
N U
E Reference Provided NRC: Question only involves reading a graph without any further knowledge.
LIC: LOD=2 supported by validation results.
13 H
2 N
E 2x2. NRC: Distractor analyses appear to be mismatched with distractors.
LIC: Pedigree enhanced to address concerns.
14 F
2 X
B E
NRC: Distractor B states why an action is acceptable. The question is NOT asking for a required action though. Add missing pedigree information (K/A, etc.)
LIC: Distractor B clarified and pedigree fixed.
15 H
2 N
S Reference Provided 16 H
4 M
B S
NRC: Need to see original question (M vs B).
LIC: Pedigree enhanced to correct source as Bank.
17 F?
H 2
N E?
S NRC: The question asks why an action is required for an ATWS, so no plant conditions seem to be needed.
Why is the question H vs F?
LIC: Question accepted as-is.
18 H
3 M
N E
U 2x2. NRC: Distractor B has a typo: all none essential personnel. 2012 NRC Exam 11-1 question Need to see original question (M vs B).
LIC: Pedigree enhanced to correct source as New.
Typo corrected in distractor The NRC and the Licensee agreed that there were NO answers in the Administered exam.
19 H
4 N
S 20 H
2 N
E 2x2. NRC: Occurred misspelled in stem. Common wording in answer/distractors should be in stem.
LIC: Misspelled word corrected.
21 H
3 N
E S
NRC: LOK listed as H but actually F. Stem should say which of following is potential cause of isolation.
LIC: Question accepted as-is.
22 H
2 X
M U
E 2x2. NRC: Typo ? in stem. Distractors B and C implausible since all TS 3.3.3.1 items have same modes of applicability. Need to see original question (M vs B). Reference provided in stem only?
LIC: Pedigree enhanced to address concerns.
Original question reviewed to ensure M vs. B.
23 H
2 X
N U
Reference Provided NRC: Answer D can be selected simply based on tolerances used on chart w/o other info. If any distractor was correct all higher valued distractors would have to logically be correct also.
LIC: New question provided - LGA-001 Table K only to be provided reference.
24 H
2 N
S?
E 2x2 NRC: Explain the plausibility of minimizing spread of contamination ONLY. When is it appropriate without regard to minimizing personnel exposure?
LIC: Pedigree enhanced to address concerns.
25 H
2 N
S 26 H
2 X
N E
Reference Provided NRC: Distractor D has grammar error (is unclear).
Distractor D is not plausible; why would applicant logically think that Sup. Chamber D/P would challenge PSPL curve? Clarify reasoning for plausibility.
LIC: Plausibility supported by validation results.
Distractor D grammar error corrected.
27 H
2 X
N E
NRC: Distractor D is not credible if the H2 recombiner lacks an auto-start feature. Clarify plausibility basis.
LIC: Pedigree enhanced to address concerns.
28 F
2 N
S E
NRC: Level typo - SRO?
LIC: Pedigree corrected.
29 F
2 B
S 30 F
1 X
N U
NRC: No distractors are credible; 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> have already elapsed so if you add this to the listed options you get 6/8/16/30 of which only 8 is a plausible value.
LIC: New question provided.
31 F
2 X
B U
E NRC: Distractors B and D are not plausible. Only Red or green colors are realistic options per Att. C info.
LIC: Pedigree enhanced to address concerns.
32 H
3 B
E 2012 NRC Exam 11-1. NRC: Distractor B analysis says injection valve opens after test valve closes; clarify.
LIC: Pedigree enhanced to address concerns.
33 H
2 N
S 34 H
3 N
S 35 H
2 X
M N
U E
2x2. NRC: Distractors C and D are not plausible.
LOA-RP-101 has steps to suspend half-scram testing, but nothing that says to complete it. Need to see original question (M vs B).
LIC: Pedigree enhanced to address concerns.
36 F
3 B
S 2001-01 NRC Exam 37 H
3 X
N U
NRC: Two correct answers. This question needs to specify in the stem that shorting links are installed (see question 38 comments also). Otherwise distractor D could be correct also.
LIC: Question stem corrected to specify no special testing and not initial core load.
38 F
2 X
X N
U E
2x2. NRC: Question is based on a maintenance procedure, too specific for a closed book exam.
Adding clarification to question 37 will cause question 38 to test same knowledge. K/A is broad enough to allow other testing options.
LIC: Pedigree enhanced to address concerns.
39 F
2 N
U S
NRC: Its not plausible that failure of a power supply inside APRM A would affect other APRMs, so C is the obvious choice.
LIC: Question accepted as-is.
40 H
3 N
S 2x2.
41 H
2 N
S 2x2.
42 F
3 X
N E
NRC: Distractor C is not plausible. The distractor provides an indication for 1VR04YA which per the distractor analysis is not displayed on the PCIS display.
LIC: Distractor C fixed.
43 H
3 N
E NRC: Distractor 2 pedigree has typos LIC: Pedigree enhanced to address typos.
44 F
3 N
S 45 H
2 N
E NRC: Distractor B is not plausible. Failure is on Unit
- 2. Why would applicant think that failure would start SBGT on only Unit 1?
LIC: Pedigree enhanced to address plausibility.
46 F
2 N
S 2x2.
47 H
3 B
E 2x2. NRC: Voltage/VARS relationship previously tested on question 20. Current question is too similar to question 20 and one of these should be changed.
LIC: Question accepted as-is.
48 H
3 N
S 2x2 49 F
2 X
N E
NRC: Distractor B is not credible. The lockout feature it discusses is non-existent on DC buses.
LIC: Pedigree enhanced to address plausibility.
50 H
3 M
S 2008 NRC Exam 2x2. NRC: Need to see original question (M vs B).
LIC: Original question reviewed to ensure M vs. B.
51 F
2 N
E 2x2. NRC: Change stem to station air compressors were running when Bus 141X tripped on overcurrent LIC: Comments incorporated.
2 F
2 X
N E
U NRC: Distractors are not plausible. Why wouldnt the applicant associate the 2SA01C station air compressor differently? Why would OSA01C be affected at all?
LIC: Pedigree enhanced to address plausibility.
53 H
3 N
E NRC: Stem contains a typo; missing the word on prior to 1H13-P601.
LIC: Correction incorporated.
54 H
2 M
S NRC Exam 2002-01 Cert Exam NRC: Need to see original question (M vs B).
LIC: Original question reviewed to ensure M vs. B.
55 F
1 2
X N
U E
NRC: Distractors A and B are not plausible; low pressure setpoints on an overpressure protection question.
LIC: Validation supports plausibility. Removed Low
& High from distractors.
56 F
2 X
N E
NRC: Stem should ask following scram from 100%
the following is indicated on full core displaywhich of following provides expected indications.
Distractor A' (0) is implausible given combinations of provided indications. A value of 3 might be a better choice.
LIC: Comments incorporated.
57 F
2 N
S 58 H
4 M
S NRC Exam 13-1.
59 H
2 N
S 60 H
3 X
B U
E NRC: Distractor D could be considered a correct answer also since conductivity could be rising from cooling loss, and 2nd part is a valid action of LOA-GC-101.
LIC: Pedigree enhanced and D was verified to be incorrect.
61 H
3 X
N E
NRC: Distractor C is not plausible. SJAE cond. leak improving SJAE function is not a reasonable misconception.
LIC: Comment incorporated.
62 F
3 X
N U
S NRC: Distractor C could also be considered correct.
LOP-FW-16 Reactor Level Err (pg. 18) for deviation of 3 of 4 Rx level transmitters discusses transfer to manual mode and manual control.
LIC: Question accepted as-is.
63 F
2 N
S 64 F
3 M
S 2x2. NRC: Need to see original question (M vs B).
LIC: Original question reviewed to ensure M vs. B.
65 F
2 X
N E
NRC: Distractor D is not plausible. An MSIV would not reasonably be associated with RWCU.
LIC: Pedigree enhanced to address plausibility.
66 F
2 N
S 67 F
2 N
S 2x2.
68 H
2 N
E NRC: Add missing pedigree information (K/A, etc.)
LIC: Pedigree enhanced to address concerns.
69 F
3 N
S 70 H
3 X
N U
E NRC: This is an SRO-level question. Refer to ES-401 Att. 2 Figure 1. Applying SRs IAW TS rules of usage and generic SR requirements is SRO-only.
LIC: Pedigree enhanced to address RO-Level appropriateness for this question at LAS.
71 F
2 B
S 08-1 NRC RO Exam 72 H
1 2
X N
U S
NRC: Question is too easy and has no plausible distractors. A simple knowledge of general locations is all thats required.
LIC: Validation supports plausibility.
73 F
2 N
S 74 H
3 N
S 75 F
2 B
S 4/75 = 5% Flawed Questions before post-exam comments were considered
Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus C
u e
s T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q =
K/A SRO Only 76/
1 H
3 2
X B
U E
NRC Exam 11-1.
NRC: 1) Distractor B could potentially be considered correct. No initial power level is specified to establish that a turbine trip would not have led to a Rx scram (30% set point?) 2) E vice U because Q on previous NRC exam.
LIC: Stem information enhanced to address concerns.
77/
2 H
3 X
N U
E 2x2.
NRC: Distractors B and C not plausible. Requires a DEHC operation from an already evacuated CR.
LIC: Distractors B and C changed to use DEHC.
78/
3 F
2 X
R N
E 2x2 NRC: 1) Distractor A is not plausible; spent fuel pit racks have linings containing neutron absorbers.
Positive reactivity additions are not likely misconceptions. 2) Not SRO Only, RO knowledge.
Difficult to write SRO Q for this KA.
LIC: Pedigree enhanced to address concerns.
79/
4 H
2 X
X R
N S
U 2X2 NRC: 1) C and D not plausible. Stem is asking for method of pressure control which directs applicant to A or B. 2) The term pressure is used in stem question and choices A and B (cueing). 3) Majority of required answer is RO knowledge (i.e., dont need to select/interpret procedure steps or the like).
LIC: Question accepted as-is.
80/
5 H
3 N
S 81/
6 H
F 3
1 X
X N
U Reference Provided NRC: 1) Distractors B and D are not plausible.
RPV water level channel C is not as far off as A so why would someone deem C inoperable but A operable. 2) Also, the K/A is for low RPV water level. 3) LOD=1. Providing reference for this Q requires very little analysis to determine correct answer.
LIC: Question replaced.
82/
7 H
1 N
U E
Reference Provided NRC: Providing LGA-004/006/009 in entirety for this question will affect the validity of other questions on this exam that are written to be closed book. With these full references, question is LOD-1.
LIC: Pedigree enhanced to address concerns.
83/
8 F
2 R
N S
2x2 NRC: Knowledge of Safety Limit TS, associated LCOs, and bases are RO knowledge and can answer the question.
LIC: Question accepted as-is.
84/
9 H
3 X
X N
U Reference Provided NRC: 1) Providing LGA-001 in entirety as a reference will affect validity of other exam questions similar to Ques 82. 2) Question doe s not test the high suppression pool level portion of the K/A.
LIC: Question replaced.
85/
10 H
2 R
N U
E Reference Provided NRC: 1) Capitalize annunciators. 2) Providing LGA-002 similar to prior comments. 3) Diagnosing the leak location and reactor scram are RO knowledge and can answer the question, the rest would be direct lookup/cues based on the LGA info.
LIC: License level OK. Reference concerns and clarifications are addressed in pedigree enhancements.
86/
11 H
2 N
E 2x2. Reference Provided NRC: Stem should just say TS 3.6.4.3 requires LIC: Change incorporated.
87/
12 H
1 2
N U
E NRC: Applicant can answer this question if all they know is RPV is flooded if level is unknown (ie, can eliminate distractors).
LIC: Validation supports plausibility. Also, LGA-005 is reference in B for balance.
88/
13 H
3 N
S 89/
14 H
1 X
R N
U 2x2.
NRC: 1) LOD=1. Question has an answer and two distractors that are identical. Part one is the same in all. 2) RO knowledge (ie, big picture knowledge of whether to continue to operate, or trip RCIC)
LIC: Question replaced.
90/
15 H
2 N
S Reference Provided 91/
16 H
2 R
N U
E NRC: 1) Distractor A is not plausible. If the applicant thinks the rod would stop at 12, why would they think the RWM is inop? 2) RO knowledge.
LIC: Distractors enhanced to address the concerns.
92/
17 H
3 N
S 2x2 Reference Provided 93/
18 H
3 R
N S
NRC: Q pedigree identifies this as RO level question.
LIC: Pedigree corrected to correct license level 94/
19 F
2 N
S U
The NRC and the Licensee agreed that there were 2 correct answers in the Administered exam because the stem was unclear.
95/
20 H
1 2
N U
E Reference Provided NRC: LOD=1. Items 2, 5, 6, and 8 are in all choices. And if A RHR taken out of service, I would surmise that B RHR was a protected train (Choices B and C) and eliminate A and D. I would then pick B because it has the B/C RHR Rm Fan vice D which has the MDRFP.
LIC: Review complete. Needed password to unlock Paragon list reference.
BOLDED = 30 question Initial review 5/25 = 20% flawed 96/
21 H
2 N
S Reference Provided 97/
22 H
3 N
S NRC: LOK=F? Only way to determine answer is by memory.
LIC: LOK determined to be high 98/
23 H
2 X
B E
NRC: Distractor D is not plausible; why would an applicant think they need to bypass an alarm that is functioning properly?
LIC: New D distractor provided.
99/
24 F
2 N
S 10 0/
25 H
2 R
B U
NRC: Question is not SRO only, it can likely be answered using systems knowledge without detailed procedure knowledge.
LIC: