ML22173A042
| ML22173A042 | |
| Person / Time | |
|---|---|
| Site: | Braidwood  |
| Issue date: | 06/27/2018 |
| From: | Randy Baker NRC/RGN-III/DRS/OLB |
| To: | Exelon Generation Co |
| Baker R | |
| Shared Package | |
| ML17164A319 | List: |
| References | |
| Download: ML22173A042 (14) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: BRAIDWOOD STATION UNITS 1 and 2 Exam Date: May 29 - June 8, 2018 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist>: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only. (K/A and license-level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 3
M S
NRC: Need to view the original question to determine if it was significantly modified.
Also, does the tag description of the SSPS train A slave relay K626 include (CNMT ISOL PHASE B)? (Is this a cue?).
Response: The original question has been significantly modified. Additionally, the component label contains the indicated nomenclature. Question is SAT as written.
2 F
2 N
S Question is SAT as written.
3 H
3 N
E S NRC: The K/A refers to CFR 41.7, Design Features (not 41.6, Reactivity Control Design Features), which is a better match for this question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
4 H
3 B
E S 1 out of 2 twice; 2011 NRC Exam NRC: The K/A refers to CFR 41.5, Operating Characteristics (not 41.7, Design Features), which is a better match for this question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
5 H
2 B
S 2011 NRC Exam. Question is SAT as written.
ES-401 2
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 2 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 6
H 2
B E S 2011 NRC Exam.
NRC: The K/A refers to CFR 41.2 - 41.9, but 41.5, Operating Characteristics (versus 41.7, Design Features), is a better match for this question.
Response
Licensee Agreed - Analysis revised. Question is SAT.
7 H
3 B
S 2013 NRC Exam. Question is SAT as written.
8 H
2 N
S Question is SAT as written.
9 H
2 N
E S 1 out of 2 twice NRC: The K/A refers to CFR 41.7, Design Features (not 41.10, Procedures), which is a better match for this question.
Also, pressure and temperature are not specifically monitored during the surveillance because the pressurizer PORVs are isolated by their block valves before testing, so the PRT sees no energy addition. (Not discriminating for competency?)
Response: Licensee Agreed - Analysis revised. Requires specific System interrelational knowledge to answer. Question is SAT.
10 H
2 B
S 2009 NRC Exam.
NRC: Fix the typo in K/A wording (as they apply).
Response: Typo fixed. Question is SAT.
11 H
2 B
S 2013 NRC Exam. Question is SAT as written.
12 H
2 B
S NRC: LOD = 1. It is well known that a loss of DC bus results in a reactor trip, thereby eliminating two distractors.
Response: The loss of a DC Bus will not results in a trip at all power levels.
Question is SAT as written.
13 H
2 N
S 1 out of 2 twice NRC: Distractors C and D are not plausible because the coincidence remains unchanged after tripping the bistables.
Response: Since there are for four pressure channels, the distractors are plausible and incorrect. Question is SAT as written.
14 F
2 M
E S 2011 NRC Exam. (Modified)
NRC: Need to view the original question to determine if it was significantly modified.
Also, the K/A refers to CFR 41.5, Operating Characteristics (not 41.10, Procedures),
which is a better match for this question (memory question).
Response: Question was significantly modified. Licensee Agreed - Analysis revised.
Question is SAT.
ES-401 3
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 3 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 15 F
2 N
S NRC: The K/A refers to CFR 41.5, Operating Characteristics (not 41.7, Design Features), although 41.7 seems to be a better match for this question.
Response: Question is SAT as written.
16 F
2 N
S NRC: The K/A refers to CFR 41.5, Operating Characteristics (not 41.11, RMS),
although 41.11 seems to be a better match for this question. Also, is it required to know which panel that the first indication of a SGTL would appear on (Objective)?
Response: Learning Objective provided. Question is SAT as written.
17 H
3 N
S NRC: The K/A refers to CFR 41.5, Operating Characteristics (not 41.7, Design Features), and 41.5 seems to be a better match for this question.
Response: Question is SAT as written.
18 H
2 N
E S 1 out of 2 twice NRC: Distractors A and B are not plausible because 1BwEP-1, LOSS OF PRIMARY OR SECONDARY COOLANT is not used for a faulted S/G, which is described in these distractors. 1BwEP-2, FAULTED SG ISOLATION is used to address a faulted S/G, which is described in all distractors, so distractors C and D are LOD=1. Finally, The K/A refers to CFR 41.5, Operating Characteristics (not 41.4, Secondary Systems), and 41.7 seems to be a better match for this question.
Response: Although distractors A and B plausible as written, changed both of the distractors to address the EOP flowpath. Question is now SAT.
19 F
2 B
S Question is SAT as written.
20 F
3 B
E S NRC: The K/A refers to CFR 41.5, Operating Characteristics (not 41.7, Design Features), and 41.5 seems to be a better match for this question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
21 F
2 N
E S NRC: The K/A refers to CFR 41.5, Operating Characteristics (not 41.7, Design Features), and 41.5 seems to be a better match for this question. Also need to justify why this is considered an RO-level question, since Tech Spec 3.8.4 condition B, which appears in all answer explanations, is below the line.
Response: Licensee correcting the analysis. Question is SAT.
22 F
2 N
S 1 out of 2 twice NRC: The K/A refers to CFR 41.7, Design Features (not 41.8, Emergency Systems),
although 41.8 seems to be a better match for this question.
Response: Question is SAT as written.
23 F
3 B
E?
2011 NRC Exam.
NRC: The K/A refers to CFR 41.7, Design Features (not 41.11, RMS), although 41.11 seems to be a better match for this question.
ES-401 4
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 4 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Question is SAT as written.
24 F
2 B
E S 2016 NRC Exam; Q-72.
NRC: The K/A refers to CFR 41.10, Procedures (not 41.7, Design Features), and 41.10 is a better match for this question, since the K/A specifies using control room reference material. This question was written for the 2011 NRC exam using a different K/A (same as the 2016 K/A) for the ability to use radiation monitoring systems (G-2.3.5) which is a match. How does this bank question match the K/A for this exam?
Response: Use of Rad Monitor System meets control room Reference K/A match.
Licensee Agreed - Analysis revised. Question is SAT.
25 F H 3
N E S NRC: There are several conditions in the stem to consider, making this a LOK = H question, not F Response: Licensee agreed. Question is SAT.
26 F
3 N
E S NRC: The K/A refers to CFR 41.7, Design Features (not 41.8, Emergency Systems),
and 41.7 seems to be a better match for this question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
27 H
3 B
S 2006 NRC Exam.
NRC: Fix the typo. in the pedigrees K/A -...malfunction fo the.
Response: Typo fixed. Question is SAT as written.
28 F
2 N
E S NRC: A dropped fuel assembly is clearly a Fuel Handling Emergency, and a visible crack in the containment wall is clearly a Loss of Containment Integrity, so the answer can be easily determined.
Also, the K/A refers to CFR 41.7, Design Features (not 41.10, Procedures), although 41.10 is a better match for this question. There is potentially conflicting information in the stem: No other issues have been noted after all walk downs were completed, and 1BwOA REFUEL-2, REFUELING CAVITY OR SPENT FUEL POOL LEVEL LOSS.
Therefore, distractors B & D become implausible.
Response: Pen 66 location makes the third option plausible, and Distractors B & D plausible. Licensee Agreed on K/A - Analysis revised. Question is SAT.
29 F
2 N
S 1 out of 2 twice NRC: The K/A refers to CFR 41.7, Design Features (not 41.6, Reactivity Control Design Features); though both of them match for this question.
Response: Question is SAT as written.
30 H
2 M
E S 1 out of 2 twice NRC: Need to view the original question to determine if it was significantly modified.
Also, bullets in the stem imply the conditions given are not sequential, but they are.
ES-401 5
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 5 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Question was significantly modified. Changed Stem to separate ICs from event conditions. Question is SAT.
31 F
4 N
E S NRC: The K/A refers to CFR 41.7, Design Features (not 41.10, Procedures),
although 41.10 is a better match for this question - explain how the K/A is met.
Also, the provided reference does not support that the use of post-accident neutron flux monitors instead of the source range detectors is allowed. Is the information given in the bases? If so, is this still RO level knowledge? (Required from memory?)
Response: Licensee Agreed - Analysis revised. Acceptable to use post-accident neutron flux monitors and data requested is RO knowledge. Question Is SAT.
32 H
2 N
S 1 out of 2 twice. Question is SAT as written.
33 H
3 M
S 2011 NRC Exam. (Modified)
NRC: Need to view the original question to determine if it was significantly modified.
This question has not been significantly modified: although one condition in the stem has been modified (operator attempted action) the distractors and the correct answer appear to remain the same.
Response: Per required procedural response, question is significantly modified.
Question is SAT as written.
34 F
2 N
S NRC: Distractor D (and possibly A) is/are not plausible. Two or three SG safeties are plausible choices, but one or four SG safeties are not, as that assumes an extreme misunderstanding of the setpoints.
Response: Need to know system component knowledge to answer successfully.
Question is SAT as written.
35 H
2 N
S Question is SAT as written.
36 F
2 N
E S 1 out of 2 twice NRC: Distractors B & D are not plausible, as a GDT release would be monitored.
Also, the K/A refers to CFR 41.5, Operating Characteristics (not 41.11, RMS),
although 41.11 seems to be a better match for this question.
Response: The question asks which specific Rad monitor is monitoring the release.
Licensee Agreed - Analysis revised. Question is SAT.
37 F
3 B
S 2014 NRC Exam; Q-9. Question is SAT as written.
38 H
3 B
S 2009 NRC Exam. Question is SAT as written.
39 H
3 N
S Question is SAT as written.
40 H
3 B
S 2014 NRC Exam. Question is SAT as written.
41 F
3 N
E S 1 out of 2 twice.
ES-401 6
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 6 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only NRC: The K/A refers to CFR 41.8 & 10, Emergency Systems & Procedures (not 41.7, Design Features), and 41.8 is a better match for this question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
42 H
2 B
S 2009 NRC Exam. Question is SAT as written.
43 H
2 N
S NRC: The K/A refers to CFR 41.7, Design Features (not 41.5, Operating Characteristics); though both of them match for this question.
Response: Question is SAT as written.
44 H
4 N
S NRC: This appears to be an SRO-level question because it involves implementation of mitigation actions within an AOP RNO action and aligning the RWST to a charging line break.
Also, the K/A refers to CFR 43.5, SRO Procedure Selection (not 41.10, Procedures);
though 43.5 matches best for this RO-level question. K/A mismatch?
Response: Question requires the applicant to deduce the leak location and use system to select a mitigating action. Question is SAT as written.
45 H
3 B
E S NRC: Although recognition of plant status is required to properly locate the correct answer from Table A of the procedure in use, the Table is directly provided to the applicant, separate from the procedure. This appears to make the question, as written, potentially a direct look-up.
The given CFR link for the question is 41(b)(4), which references secondary coolant and auxiliary systems; RH is neither. It would appears that 41(b)(10) for use of select procedures to mitigate the event. NOTE: K/A refers to 41(b)(8 & 10).
Response: Do not provide a reference. Question is SAT.
46 F
2 N
S NRC: The K/A refers to CFR 41.5 & 10, Operating Characteristics & Procedures (not 41.7, Design Features), and both 41.5 and 41.7 are better matches for this question.
Response: Question is SAT as written.
47 F
2 N
E S NRC: LOD = 1. This is a very basic principle which should be known from memory and is read to the crew each time EOP-3 is executed in the simulator. It does not discriminate between a competent and less than competent operator.
Response: Changed distractor reasons to fill Steam line. Question is SAT.
48 F H 2
X N
U S NRC: The stem is unclear. The information ultimately sought is not comparing and contrasting both units NDT at 2 power levels, given several conditions, and not from memory.
Also, the K/A refers to CFR 41.8 & 10, Emergency Systems & Procedures (not 41.7, Design Features), although 41.7 matches better for this question.
Response: The Stem was restructured to list ICs for different Units to relate to the same distractors. Question is SAT.
ES-401 7
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 7 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 49 F
2 N
E S NRC: Contrary to the K/A match reasoning, the question does not require knowledge of the overall mitigation strategy, but only recalling the first action of 1BwOA SEC-1, Attachment D - questionable RO required knowledge.
Also, the K/A refers to CFR 41.10, Procedures (not 41.7, Design Features), and CFR 41.10 matches best for this question.
Response: Reword stem to require selection of correct answer based on the mitigating strategy. Question is SAT.
50 F H 2
?
B E
NRC: Given several equipment conditions, and complex plant conditions, this is not a memory-level question.
Also, the K/A refers to CFR 43.5, SRO Procedure Selection (not 41.10, Procedures);
though neither matches for this RO-level question.
Response: Agreed. Chose ___ K/A.
51 F H 3
?
N E S NRC: Given several equipment conditions, and complex plant conditions, this is not a memory-level question.
Also, the K/A refers to CFR 43.5, SRO Procedure Selection (not 41.7, Design Features); though 41.7 matches better for this RO-level question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
52 H
3 B
S 2011 NRC Exam. Question is SAT as written.
53 F
3
?
N E S NRC: The K/A refers to CFR 41.4 & 8, Secondary & Emergency Systems (not 41.7, Design Features), and 41.4 matches better for this question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
54 F
2
?
M E S 2013 NRC Exam. (Modified)
NRC: Question was significantly modified.
Also, the K/A refers to CFR 41.10, Procedures (not 41.7, Operating Characteristics),
which is a better match for this question.
Response: Licensee Agreed - Analysis revised. Question is SAT.
55 H
3 B
S 2013 NRC Exam.
56 F
3
?
B S
NRC: The K/A refers to CFR 41.10, Procedures (not 41.7, Operating Characteristics), although 41.7 is a better match for this question.
Response: Question is SAT as written.
57 H
3
?
B S
2009 NRC Exam.
NRC: The K/A refers to CFR 41.7, Design Features (not 41.6, Reactivity Control Design Features); though both of them match for this question.
Response: Question is SAT as written.
ES-401 8
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 8 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 58 H
3
?
N S
1 out of 2 twice NRC: The K/A refers to CFR 43.5, SRO Procedure Selection (not 41.6, Reactivity Control Design Features); although 41.6 matches better for this RO-level question.
Response: Question is SAT as written.
59 H
2
?
B S
NRC: The K/A refers to CFR 41.7, Design Features (not 41.2, Core Design Features); although 41.2 matches better for this question.
Response: Question is SAT as written.
60 F
2
?
B S
2011 NRC Exam.
NRC: The K/A refers to CFR 41.5 & 10, Operating Characteristics & Procedures (not 43.7, Fuel Handling Facilities and Procedures), although 43.7 matches best for this RO-level question.
Response: Question is SAT as written.
61 H
2
?
N E S NRC: Question requires the applicant to determine which procedure to use for mitigation of the loss of subcooling once the condition is identified by the applicant.
The given CFR link for the question is 41(b)(7), when it appears the 41(b)(10) may be a better match. NOTE: K/A refers to 41(b)(8/10).
Response: Licensee Agreed - Analysis revised. Question is SAT.
62 H
3 B
E S 2009 NRC Exam.
NRC: The given CFR link for the question is 41(b)(5), referencing facility operating characteristics during steady state and transient conditions, which predominately refers to at power operations. With the conditions given in the stem (Mode 4) the question may be a better tie to 41(b)(10). NOTE: K/A refers to 41(b)(5).
Response: Licensee Agreed - Analysis revised. Question is SAT.
63 F
3 M
S 2014 NRC Exam; Q-65. (Modified)
NRC: Question was significantly modified.
Question only requires the applicant to determine the source of the containment flooding; question does not address adherence to appropriate procedures or operation within the limitations in the facility's license/amendments. The given CFR link for the question is 41(b)(10), when it appears the 41(b)(8/9) may be a better match. NOTE: K/A refers to 43(b)(5), and there is not a 41(b) link to the K/A.
Response: Determining which system is responsible is procedurally driven, and driven by RO Knowledge. Question is SAT as written.
64 F
2 M B E S 2014 NRC Exam; Q-92. (Modified)
NRC: Question as written simply requires the applicant to determine which containment system is operated during performance of 1BwFR-Z.3, not determine how any of the components are related to a high containment radiation condition, and
ES-401 9
Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 9 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only therefore does not match the K/A. (Originally this was an SRO-Only question prior to attempted modification.)
Also, this question has not been significantly modified as no conditions in the stem (other than the specific question asked itself) have been changed and, although the distractors have been changed, the correct answer remains essentially the same.
Response: Procedure determines which system/components mitigates. Question is SAT.
65 H
3 M
E S 2011 Cert Exam.
NRC: Need to view the original question to determine if it was significantly modified.
Although the statement concerning a racked out breaker indicating the same as open for logic circuits, the explanation that racking out RTA is equivalent of opening the breaker, should be clarified.
Response: Update analysis. Question is significantly modified. Question is SAT.
66 F
2 X
B U S 2009 NRC Exam.
NRC: Question requires the applicant to recall how to correct an erroneous entry on a hand written document, per the appropriate procedure step. The stem should include the additional action procedurally required to correct the erroneous entry to ensure the correct answer is a complete answer.
Response: Added rest of answer into the stem. Question is SAT.
67 H
2 B
S 2013 NRC Exam.
NRC: The given CFR link for the question is 41(b)(7), referencing failure modes of safety systems, which may be a better match to the K/A statement. NOTE: K/A refers to 41(b)(10).
Response: Question is SAT as written.
68 H
3 M
S GFE Exam Bank NRC: Need to view the original question to determine if it was significantly modified.
Is the applicant also provided the procedure to perform the calculation given? (Is the calculation formula required to be memorized, or must the applicant simply know how to use it?) The given CFR link for the question is 41(b)(1), and is not a clear match to the K/A link; the K/A references the ability to use procedures. NOTE: K/A refers to 41(b)(10).
Response: Question is SAT as written.
69 F
3 B
S 2011 NRC Exam. Question is SAT as written.
70 H
3 B
S 1 out of 2 twice. Question is SAT as written.
71 F
2 B
S 2009 NRC Exam. Question is SAT as written.
ES-401 10 Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 10 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 72 H
3 B
S NRC: Question requires the applicant to determine when equipment line-ups require the containment release to be secured. The given CFR link for the question is 41(b)(13), referencing procedures for handling and disposal of effluents, when the K/A refers to 41(b)(11).
Response: Question is SAT as written.
73 F
3
?
B E
2013 NRC Exam.
NRC: Question requires the applicant to determine which evolution will result in the lowest increase in radiation levels in the Auxiliary Building. The given CFR link for the question is 41(b)(10), when it appears that 41(b)(12) may be a better match based on radiation safety considerations. NOTE: K/A refers to 41(b)(12).
Response: Licensee Agreed - Analysis revised. Question is SAT.
74 F
2 N
S 75 H
3
?
B E S 2009 NRC Exam.
NRC: The answer explanation/distractor analysis provided duplicates that of the original question, but identifies the wrong K/A statement (g 2.4.45), which is not a match for the question. Question does not require the applicant to prioritize or interpret the significance of any annunciator or alarm. However the question does match the required K/A. (G 2.4.46 is the required K/A per ES-401-3, Tier 3 outline.)
The given tie to the K/A link is 41(b)(10); but a better match to the question may be 41(b)(7).
Response: Corrected the K/A written. Question is SAT.
76 H
2
?
Y M
E S 2009 NRC Exam. (Modified)
NRC: Question was significantly modified.
The stem initially assumes no operator actions to support the first part of the question. To support the procedure selection second part of the question, need to separate the two parts. (i.e., Following crew identification of the component failure, the US will enter...)
Response: Reworded the question in the stem to: The US will use to... Question is SAT.
77 F H?
2 N Y B
E S 2013 Cert Exam.
NRC: RO applicants must be cognizant of the P&Ls of the Ops, i.e. BwOP RH-11, which is the only consideration for crediting 1A RH train. Stem may need to address SRO-job related items (TS bases requirements) to ensure the question meets SRO-Only criteria.
Also, this appears to be higher cog based on recall of a limit and then need to know what to do; was the bank question memory?
Response: Reworded stem question to link to SRO-Only.
ES-401 11 Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 11 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 78 F
2
?
Y M
E S 2013 NRC Exam. (Modified)
NRC: Question was significantly modified.
Question requires the applicant to select the correct procedure/action based on current plant conditions and is therefore SRO-Only. However, since the selection is made based on understanding the impact of current conditions on containment design limits, a tie to 41(b)(9) may be a better match for a K/A link. The given tie to the K/A link is 41(b)4; neither is a clear match to the K/A link. NOTE: K/A refers to 41(b)(5), which refers to the primary system, not secondary and auxiliary systems.
Response: Licensee Agreed - Analysis revised. Question is SAT.
79 H
3
?
Y N
E S NRC: The construction of the distractors, using the identifiers ONLY and BOTH, along with the unique conjunction (and) in Distractor D, potentially cue the applicant towards the correct answer.
Response: Reworded the distractors to list procedure names in the stem. Question is SAT.
80 F
2 Y
N E S NRC: Why are the actions referenced in the distractors considered... actions that require immediate operation of system components and controls, without reference to procedures.? These are actions from an OA and it is not clear why reference to them is SRO-Only.
Response: Reworded 3rd bullet to a question. Question is SAT.
81 F
2 Y
N S
NRC: Correct the grammar/punctuation in the stem question (...SYSTEM OPERATION, the US will direct the NSO to delay performing Step 6, To startup Cnmt Mini-Purge Supply Fan, until/because...
Response: Corrected. Question is SAT.
82 H
2 Y
N E S NRC: Question simply asks the applicant what the Emergency Classification level will be. Better to state in the question: The Shift Manager will declare a/an...
Response: Reworded stem question. Question is SAT.
83 F
2 Y
N S
NRC: The RO applicants should be cognizant of Rx Pwr limits for secondary chemistry action levels. SRO should direct crew to reduce power appropriately for SRO-Only.
Response: Not required RO knowledge from memory. Question is SAT as written.
84 F
2 Y
B S
Question is SAT as written.
85 H
3 Y
B S
2011 NRC Exam. Question is SAT as written.
86 F
2 Y
N E S 1 out of 2 twice; Please provide a copy of 1BwFR-S.1 to validate bases of priority.
NRC: The question stem does not require the applicant to provide a reason why the selection of a procedure is made as part of the correct answer. (Providing the reason
ES-401 12 Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 12 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only for selecting a particular answer, potentially turns the distractors into a collection of true/false statements?)
Response: Modified stem wording. Question is SAT.
87 H
2 Y
N S
Question is SAT as written.
88 H
3 Y
N S
Question is SAT as written.
89 H
3
?
Y B
S 2016 NRC Exam; Q-93(not 91).
NRC: K/A listed in the Sample Plan, W/E05EA2.05, is actually W/E05EA2.1.
Although the question is taken from the 2016 NRC Exam, the distractors have been modified is a way that makes the stem less focused and inconsistent with grammar.
Need to consider modifying Distractor D and possibly removing the word ONLY from Distractors B & C.
Response: Modified distractors to match the K/A. Question is SAT as written.
90 H
3 Y
N E
NRC: Question stem states Rx Pwr is 20% and lowering, but the Correct Answer A states 25%; which is it? While Answer A appears to be most correct, the analysis does not give a completely accurate explanation of how to apply LCO 3.0.4. Entry into other Modes of applicability are permitted if the changes in Mode are part of a shutdown of the Unit. Distractor C may also be considered a correct answer.
(Possibly >1 or NO Correct Answer?)
Response: Corrected the distractor analysis.
91 H
2 X
Y N
U S 1 out of 2 twice NRC: Since BwAP 1110-1A4 requires an hourly fire watch OR automatic detection instrumentation available, AND Restore the CO2 system OR Initiate an Issue, there is not a complete correct answer given. (This assumes the bullet in the stem (zone 1D-
- 72) is accurate and automatic detection is not available, otherwise there is not a correct answer.)
Response: Added conditions to distractor C & D to allow only one complete correct answer. Question is SAT.
92 H
3
?
Y N
S NRC: Are there any reportability requirement to the NRC that are required within 15 minutes? Why is distractor A plausible? - Yes!
Also, although the reportability may be due to not meeting TS requirements for operability, the question involves correctly performing reportability requirements, which is a better K/A tie to 41(b)(10) & 43(b)(5), verses simply knowing operating limits in TSs and the bases.
(Provide the entire Reportability Manual?) - Yes.
Response: Question is SAT as written.
93 F
2 Y
B S
Question is SAT as written.
ES-401 13 Form ES-401-9 Facility: FACILITY NAME Exam Date: ACTUAL EXAM DATES Page 13 of 14 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 94 F
2
?
Y B
E S 2011 NRC Exam.
NRC: Although the question does require the applicant to know limitations in the facility license pertaining to fire protection, the question also requires the applicant to know and correctly implement the procedure requirements for staff manning and may be tied to 43(b)(5), which matches the K/A link.
Response: Licensee Agreed - Analysis revised. Question is SAT.
95 H
3 Y
B S
2009 NRC Exam. Question is SAT as written.
96 F
3
?
Y B
S 2016 NRC Exam; Q-76.
NRC: Question requires applicant knowledge of the TS bases (3.4.16) and is a tie to 43(b)(2). The given tie to 41(b)12 is not a good K/A match (should refer to 55.43 for Tier 3 SRO questions).
Response: Question is SAT as written.
97 F
3 Y
B S
2014 Cert Exam. Question is SAT as written.
98 F H?
3
?
Y B
E S NRC: Question stem requires the applicant to determine what actions are required, per the ODCM, for the given abnormal condition, referencing 43(b)(1/2). However, this decision is based on a knowledge of the radiation hazards presented by the abnormal condition and may be a better tie to 43(b)(4).
Response: Licensee Agreed - Analysis revised. Question is SAT.
99 F
2
?
Y B
E S 2014 NRC Exam; Q-87.
NRC: Question stem does not require the applicant to select any portion of a procedure based upon an assessment of plant conditions and is therefore not tied to 43(b)(5) for SRO-Only. However, a tie may be made to 43(b)(2) for knowledge of TS bases (3.1.1). NOTE: K/A refers to 43(b)(5).
Response: Licensee Agreed - Analysis revised. Question is SAT.
100 H
3 Y
B S
2014 Cert Exam. Question is SAT as written.
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: BRAIDWOOD STATION UNITS 1 and 2 Exam Date: May 29 - June 8, 2018 Page 14 of 14 RO TOTALS:
B= 35 F= 32 E= 38 Additional Notes:
M= 7 H= 43 U= 2 N= 33 SRO TOTALS:
B= 12 F= 11 E= 12 Additional Notes:
M= 2 H= 14 U= 1 N= 11
GENERAL COMMENT
S:
- 1. Question numbers highlighted as xx are sample test items the facility submitted for pre-review.
- 2.
Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
- 3.
Questions from the previous 2 NRC Exams: _4_ (RO4)/ _2_ (SRO2)
- 5.
The 10CFR55.41/43 distribution is: RO / SRO 41.1 =
41.8 =
43.1 =
41.2 =
41.9 =
43.2 =
41.3 =
41.10 =
43.3 =
41.4 =
41.11 =
43.4 =
41.5 =
41.12 =
43.5 =
41.6 =
41.13 =
43.6 =
41.7 =
41.14 =
43.7 =
- 6.
The answer distribution is:
/
SRO A = (%)
/
(%)
B = (%)
/
(%)
C = (%)
/
(%)
D = (%)
/
(%)
- 7.
There are ____ (RO)/ ____ (SRO) questions with references/attachments provided.