ML17079A510
| ML17079A510 | |
| Person / Time | |
|---|---|
| Site: | LaSalle  |
| Issue date: | 11/16/2016 |
| From: | NRC/RGN-III |
| To: | Exelon Generation Co |
| Shared Package | |
| ML15274A408 | List: |
| References | |
| Download: ML17079A510 (9) | |
Text
ES-401 Written Examination Review Worksheet (LaSalle 2016 ILE) Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 1 F 2 B S 2009 LORT Annual Exam NRC: Add reset L.O. relay to distractors B & C 2 H 3 N E LIC: Comments incorporated.
3 H 3 N S 2x2 NRC: Stem is asking if a manual scram is required but by LOA-TG-101 scram or insert rods is allowed, U
4 H 2 X N so scram is not actually required. Also, awkward E
wording concerning pressure response.
LIC: Pedigree enhanced to address concerns.
5 F 2 N S M S NRC: Need to see original question (M vs B).
6 F 2 N E LIC: Pedigree enhanced to correct source as New.
2x2 NRC: When would bearings only be plausible?
7 H 3 N E LIC: Pedigree enhanced to add information.
Instructions (Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)
- 3. Check the appropriate box if a psychometric flaw is identified:
- Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).
- T/F: The answer choices are a collection of unrelated true/false statements.
- Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- Partial: One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- # / Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable)
- 6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only NRC: K/A deals with loss of SDC and RWCU.
U 8 F 2 X N Question is unrelated to loss of SDC.
E LIC: Pedigree enhanced to address concerns.
NRC: Is a reference provided during the exam?
9 H 2 B E LIC: No - Pedigree corrected.
NRC: Distractor A is implausible because SRV tail pipe level limit is unrelated to containment pressure.
M 2012 NRC Exam 11-1 question (RO Question 57) is 10 F 2 X E B essentially the same (Bank)
LIC: Pedigree enhanced to correct source as Bank.
Plausibility supported by validation results.
11 H 3 N S Reference Provided 1 U NRC: Question only involves reading a graph without 12 H N 2 E any further knowledge.
LIC: LOD=2 supported by validation results.
2x2. NRC: Distractor analyses appear to be 13 H 2 N E mismatched with distractors.
LIC: Pedigree enhanced to address concerns.
NRC: Distractor B states why an action is acceptable. The question is NOT asking for a 14 F 2 X B E required action though. Add missing pedigree information (K/A, etc.)
LIC: Distractor B clarified and pedigree fixed.
15 H 2 N S Reference Provided M NRC: Need to see original question (M vs B).
16 H 4 S B LIC: Pedigree enhanced to correct source as Bank.
NRC: The question asks why an action is required for F? E? an ATWS, so no plant conditions seem to be needed.
17 2 N H S Why is the question H vs F?
LIC: Question accepted as-is.
2x2. NRC: Distractor B has a typo: all none essential personnel. 2012 NRC Exam 11-1 question Need to see original question (M vs B).
M E LIC: Pedigree enhanced to correct source as New.
18 H 3 N U Typo corrected in distractor The NRC and the Licensee agreed that there were NO answers in the Administered exam.
19 H 4 N S 2x2. NRC: Occurred misspelled in stem. Common 20 H 2 N E wording in answer/distractors should be in stem.
LIC: Misspelled word corrected.
NRC: LOK listed as H but actually F. Stem should E
21 H 3 N say which of following is potential cause of isolation.
S LIC: Question accepted as-is.
2x2. NRC: Typo ? in stem. Distractors B and C implausible since all TS 3.3.3.1 items have same U modes of applicability. Need to see original question 22 H 2 X M E (M vs B). Reference provided in stem only?
LIC: Pedigree enhanced to address concerns.
Original question reviewed to ensure M vs. B.
Reference Provided NRC: Answer D can be selected simply based on tolerances used on chart w/o other info. If any 23 H 2 X N U distractor was correct all higher valued distractors would have to logically be correct also.
LIC: New question provided - LGA-001 Table K only to be provided reference.
2x2 NRC: Explain the plausibility of minimizing spread of contamination ONLY. When is it S?
24 H 2 N appropriate without regard to minimizing personnel E
exposure?
LIC: Pedigree enhanced to address concerns.
25 H 2 N S Reference Provided NRC: Distractor D has grammar error (is unclear).
Distractor D is not plausible; why would applicant logically think that Sup. Chamber D/P would 26 H 2 X N E challenge PSPL curve? Clarify reasoning for plausibility.
LIC: Plausibility supported by validation results.
Distractor D grammar error corrected.
NRC: Distractor D is not credible if the H2 recombiner lacks an auto-start feature. Clarify 27 H 2 X N E plausibility basis.
LIC: Pedigree enhanced to address concerns.
S NRC: Level typo - SRO?
28 F 2 N E LIC: Pedigree corrected.
29 F 2 B S NRC: No distractors are credible; 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> have already elapsed so if you add this to the listed options 30 F 1 X N U you get 6/8/16/30 of which only 8 is a plausible value.
LIC: New question provided.
NRC: Distractors B and D are not plausible. Only U Red or green colors are realistic options per Att. C 31 F 2 X B E info.
LIC: Pedigree enhanced to address concerns.
2012 NRC Exam 11-1. NRC: Distractor B analysis says injection valve opens after test valve closes; 32 H 3 B E clarify.
LIC: Pedigree enhanced to address concerns.
33 H 2 N S 34 H 3 N S 2x2. NRC: Distractors C and D are not plausible.
LOA-RP-101 has steps to suspend half-scram M U 35 H 2 X testing, but nothing that says to complete it. Need to N E see original question (M vs B).
LIC: Pedigree enhanced to address concerns.
36 F 3 B S 2001-01 NRC Exam NRC: Two correct answers. This question needs to specify in the stem that shorting links are installed (see question 38 comments also). Otherwise 37 H 3 X N U distractor D could be correct also.
LIC: Question stem corrected to specify no special testing and not initial core load.
2x2. NRC: Question is based on a maintenance procedure, too specific for a closed book exam.
U Adding clarification to question 37 will cause question 38 F 2 X X N E 38 to test same knowledge. K/A is broad enough to allow other testing options.
LIC: Pedigree enhanced to address concerns.
NRC: Its not plausible that failure of a power supply U inside APRM A would affect other APRMs, so C is 39 F 2 N S the obvious choice.
LIC: Question accepted as-is.
40 H 3 N S 2x2.
41 H 2 N S 2x2.
NRC: Distractor C is not plausible. The distractor provides an indication for 1VR04YA which per the 42 F 3 X N E distractor analysis is not displayed on the PCIS display.
LIC: Distractor C fixed.
NRC: Distractor 2 pedigree has typos 43 H 3 N E LIC: Pedigree enhanced to address typos.
44 F 3 N S NRC: Distractor B is not plausible. Failure is on Unit
- 2. Why would applicant think that failure would start 45 H 2 N E SBGT on only Unit 1?
LIC: Pedigree enhanced to address plausibility.
46 F 2 N S 2x2.
2x2. NRC: Voltage/VARS relationship previously 47 H 3 B E tested on question 20. Current question is too similar to question 20 and one of these should be changed.
LIC: Question accepted as-is.
48 H 3 N S 2x2 NRC: Distractor B is not credible. The lockout 49 F 2 X N E feature it discusses is non-existent on DC buses.
LIC: Pedigree enhanced to address plausibility.
2008 NRC Exam 50 H 3 M S 2x2. NRC: Need to see original question (M vs B).
LIC: Original question reviewed to ensure M vs. B.
2x2. NRC: Change stem to station air compressors were running when Bus 141X tripped 51 F 2 N E on overcurrent LIC: Comments incorporated.
NRC: Distractors are not plausible. Why wouldnt the applicant associate the 2SA01C station air E
2 F 2 X N compressor differently? Why would OSA01C be U
affected at all?
LIC: Pedigree enhanced to address plausibility.
NRC: Stem contains a typo; missing the word on 53 H 3 N E prior to 1H13-P601.
LIC: Correction incorporated.
NRC Exam 2002-01 Cert Exam 54 H 2 M S NRC: Need to see original question (M vs B).
LIC: Original question reviewed to ensure M vs. B.
NRC: Distractors A and B are not plausible; low pressure setpoints on an overpressure protection 1 U 55 F X N question.
2 E LIC: Validation supports plausibility. Removed Low
& High from distractors.
NRC: Stem should ask following scram from 100%
the following is indicated on full core displaywhich of following provides expected indications.
56 F 2 X N E Distractor A' (0) is implausible given combinations of provided indications. A value of 3 might be a better choice.
LIC: Comments incorporated.
57 F 2 N S 58 H 4 M S NRC Exam 13-1.
59 H 2 N S NRC: Distractor D could be considered a correct answer also since conductivity could be rising from U cooling loss, and 2nd part is a valid action of LOA-GC-60 H 3 X B E 101.
LIC: Pedigree enhanced and D was verified to be incorrect.
NRC: Distractor C is not plausible. SJAE cond. leak improving SJAE function is not a reasonable 61 H 3 X N E misconception.
LIC: Comment incorporated.
NRC: Distractor C could also be considered correct.
LOP-FW-16 Reactor Level Err (pg. 18) for deviation U
62 F 3 X N of 3 of 4 Rx level transmitters discusses transfer to S
manual mode and manual control.
LIC: Question accepted as-is.
63 F 2 N S 2x2. NRC: Need to see original question (M vs B).
64 F 3 M S LIC: Original question reviewed to ensure M vs. B.
NRC: Distractor D is not plausible. An MSIV would 65 F 2 X N E not reasonably be associated with RWCU.
LIC: Pedigree enhanced to address plausibility.
66 F 2 N S 67 F 2 N S 2x2.
NRC: Add missing pedigree information (K/A, etc.)
68 H 2 N E LIC: Pedigree enhanced to address concerns.
69 F 3 N S NRC: This is an SRO-level question. Refer to ES-401 Att. 2 Figure 1. Applying SRs IAW TS rules of usage U
70 H 3 X N and generic SR requirements is SRO-only.
E LIC: Pedigree enhanced to address RO-Level appropriateness for this question at LAS.
71 F 2 B S 08-1 NRC RO Exam NRC: Question is too easy and has no plausible 1 U distractors. A simple knowledge of general locations 72 H X N 2 S is all thats required.
LIC: Validation supports plausibility.
73 F 2 N S 74 H 3 N S 75 F 2 B S 4/75 = 5% Flawed Questions before post-exam comments were considered
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
- 6. Source 7. Status C
Q LOK LOD Stem u Cred. Job- Q= SRO 8. Explanation T/F Partial Minutia # / Units Backward (B/ M / N) (U /E /S)
Focus e Dist Link K/A Only (F/H) (1-5) s NRC Exam 11-1.
NRC: 1) Distractor B could potentially be considered correct. No initial power level is 76/ 3 U specified to establish that a turbine trip would not H B 1 2 X E have led to a Rx scram (30% set point?) 2) E vice U because Q on previous NRC exam.
LIC: Stem information enhanced to address concerns.
2x2.
77/ U NRC: Distractors B and C not plausible. Requires H 3 X N 2 E a DEHC operation from an already evacuated CR.
LIC: Distractors B and C changed to use DEHC.
2x2 NRC: 1) Distractor A is not plausible; spent fuel pit racks have linings containing neutron absorbers.
78/
F 2 X R N E Positive reactivity additions are not likely 3
misconceptions. 2) Not SRO Only, RO knowledge.
Difficult to write SRO Q for this KA.
LIC: Pedigree enhanced to address concerns.
2X2 NRC: 1) C and D not plausible. Stem is asking for method of pressure control which directs applicant 79/ S to A or B. 2) The term pressure is used in stem H 2 X X R N 4 U question and choices A and B (cueing). 3) Majority of required answer is RO knowledge (i.e., dont need to select/interpret procedure steps or the like).
LIC: Question accepted as-is.
80/
H 3 N S 5
Reference Provided NRC: 1) Distractors B and D are not plausible.
RPV water level channel C is not as far off as A so why would someone deem C inoperable but A 81/ H 3 X X N U operable. 2) Also, the K/A is for low RPV water 6 F 1 level. 3) LOD=1. Providing reference for this Q requires very little analysis to determine correct answer.
LIC: Question replaced.
Reference Provided NRC: Providing LGA-004/006/009 in entirety for this 82/ U question will affect the validity of other questions on H 1 N 7 E this exam that are written to be closed book. With these full references, question is LOD-1.
LIC: Pedigree enhanced to address concerns.
2x2 NRC: Knowledge of Safety Limit TS, associated 83/
F 2 R N S LCOs, and bases are RO knowledge and can 8
answer the question.
LIC: Question accepted as-is.
Reference Provided NRC: 1) Providing LGA-001 in entirety as a 84/
H 3 X X N U reference will affect validity of other exam questions 9
similar to Ques 82. 2) Question doe s not test the high suppression pool level portion of the K/A.
LIC: Question replaced.
Reference Provided NRC: 1) Capitalize annunciators. 2) Providing LGA-002 similar to prior comments. 3) Diagnosing the leak location and reactor scram are RO knowledge 85/ U H 2 R N and can answer the question, the rest would be 10 E direct lookup/cues based on the LGA info.
LIC: License level OK. Reference concerns and clarifications are addressed in pedigree enhancements.
2x2. Reference Provided 86/ NRC: Stem should just say TS 3.6.4.3 requires H 2 N E 11 ____.
LIC: Change incorporated.
NRC: Applicant can answer this question if all they know is RPV is flooded if level is unknown (ie, can 87/ 1 U H N eliminate distractors).
12 2 E LIC: Validation supports plausibility. Also, LGA-005 is reference in B for balance.
88/
H 3 N S 13 2x2.
NRC: 1) LOD=1. Question has an answer and two 89/ distractors that are identical. Part one is the same H 1 X R N U 14 in all. 2) RO knowledge (ie, big picture knowledge of whether to continue to operate, or trip RCIC)
LIC: Question replaced.
90/
H 2 N S Reference Provided 15 NRC: 1) Distractor A is not plausible. If the 91/ U applicant thinks the rod would stop at 12, why would H 2 R N 16 E they think the RWM is inop? 2) RO knowledge.
LIC: Distractors enhanced to address the concerns.
92/
H 3 N S 2x2 Reference Provided 17 NRC: Q pedigree identifies this as RO level 93/
H 3 R N S question.
18 LIC: Pedigree corrected to correct license level The NRC and the Licensee agreed that there were 94/ S F 2 N 2 correct answers in the Administered exam 19 U because the stem was unclear.
Reference Provided NRC: LOD=1. Items 2, 5, 6, and 8 are in all choices. And if A RHR taken out of service, I would surmise that B RHR was a protected train (Choices 95/ 1 U H N B and C) and eliminate A and D. I would then pick 20 2 E B because it has the B/C RHR Rm Fan vice D which has the MDRFP.
LIC: Review complete. Needed password to unlock Paragon list reference.
96/
H 2 N S Reference Provided 21 NRC: LOK=F? Only way to determine answer is by 97/
H 3 N S memory.
22 LIC: LOK determined to be high NRC: Distractor D is not plausible; why would an 98/ applicant think they need to bypass an alarm that is H 2 X B E 23 functioning properly?
LIC: New D distractor provided.
99/
F 2 N S 24 NRC: Question is not SRO only, it can likely be 10 answered using systems knowledge without 0/ H 2 R B U detailed procedure knowledge.
25 LIC:
BOLDED = 30 question Initial review 5/25 = 20% flawed