ML22179A249
| ML22179A249 | |
| Person / Time | |
|---|---|
| Site: | Prairie Island  |
| Issue date: | 10/16/2018 |
| From: | Randy Baker NRC/RGN-III/DRS/OLB |
| To: | Nuclear Management Co |
| Baker R | |
| Shared Package | |
| ML17165A317 | List: |
| References | |
| Download: ML22179A249 (9) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist>: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only. (K/A and license-level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only General note: Majority of the questions to not provide justification for why the correct answer is correct. This is required in support of the distractor analysis.
1 H
3 N
S NRC: Why would Distractor B be considered plausible?
Response: Plausible if applicant thinks Train A SI only removes power to the UV coil.
Question is SAT.
2 F
2 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
3 H
3 N
S NRC: 1 out-of 2 twice. [Reference Provided-1E-1 pg. 3]
Response: Providing the reference requires the applicant to work through the procedure. Question is SAT as written.
4 H
3 B
S NRC: Question is SAT as written.
5 H
2 N
S NRC: {Correct K/A IR=4.0}
Response: Agreed - Corrected K/A IR. Question is SAT as written.
ES-401 2
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 2 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 6
F 2
N B
U S
NRC: [2014 ILT Audit Exam] How does this question relate to a PZR PCS malfunction? (K/A match)?
Response: Agreed -Modified Distractor A to read... if PRZR press is less than 2235. Question is now SAT.
7 H
2 B
S NRC: [2014 ILT Exam] Removing Boron will add negative reactivity?!
Response: Modified question to remove first part and reorder the distractors.
Question is SAT.
8 H
3 N
E S
NRC: {Correct K/A IR=3.1} [Reference Provided-Steam Tables]
Response: Agreed - Question is now SAT.
9 H
2 N
S NRC: Question is SAT as written.
10 H
3 B
S NRC: Question is SAT as written.
11 H
2 M
S NRC: {Original Bank question provided} [2016 ILT Exam Significantly Modified]
Question is SAT as written.
12 F
2 N
E S
NRC: {Correct K/A IR=3.7}
Response: Agreed - Question is now SAT.
13 F
2 N
S NRC: [K/A IR=2.2*; Justification: Important at facility because of loss of remote operating capability (from Control Room) for Safeguards equipment.]
Question is SAT as written.
14 F
3 N
S NRC: Question is SAT as written.
15 H
2 M
S NRC: {Need to see the original Bank question to ensure modified.}
Response: Agreed - Question is Modified and SAT.
16 H
2 B
S NRC: [2017 LOR Exam] Question is SAT as written.
17 F
2 N
S NRC: Question is SAT as written.
18 F
2 B
S NRC: [2016 ILT Audit Exam] Question is SAT as written.
19 F
3 B
S NRC: [2014 ILT Audit Exam] {Add K/A IR=3.6}
Response: Agreed - Question is SAT as written.
20 F
3 N
S NRC: Question is SAT as written.
21 H
2 N
E S
NRC: [Reference Provided-Fig. C1-20]
Response: Removed the reference. Question is now SAT.
22 F
2 N
S NRC: Question is SAT as written.
ES-401 3
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 3 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 23 H
3 B
S NRC: [2017 LOR Exam] {Correct K/A IR=3.1} [Reference Provided-pg. 16 of 2C1.3 AOP1 and Fig. C1-10B]
Response: References provided to permit calculation. Agreed - Question is SAT.
24 H
2 N
S NRC: [Reference Provided-pg. 10 of 1FR-C-1]
Response: Providing the reference requires the applicant to work through the procedure. Agreed - Question is SAT.
25 F
2 N
S NRC: Question is SAT as written.
26 H
3 B
S NRC: [2014 ILT Exam (Similar)] [Reference Provided-1E-0 pg. 5]
Response: The reference is provided to preclude reference to minimum AWF flow requirements. Agreed - Question is SAT.
27 F
3 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
28 F
2 N
E S
NRC: {Correct K/A IR=3.9}
Response: Agreed - Question is now SAT.
29 H
3 N
S NRC: Question is SAT as written.
30 H
3 B
S NRC: 1 out-of 2 twice. [2013 LOR; 2017 LOR Re-Exam] Question is SAT as written.
31 H
3 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
32 F
2 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
33 H
2 N
E S
NRC: 1 out-of 2 twice. {Correct K/A IR=2.9}
Response: Agreed - Question is now SAT.
34 H
2 N
E S
NRC: [Reference Provided-1C-14 AOP 1 pg. 5]
Response: Removed The reference (Immediate action.) Question is now SAT.
35 H
2 N
S NRC: Question is SAT as written.
36 H
3 B
E S
NRC: 1 out-of 2 twice. [2014 ILT Audit Exam] {Correct K/A IR=3.6}
Response: Agreed - Question is now SAT.
37 F
2 N
E S
NRC: 1 out-of 2 twice. {Correct K/A IR=3.9*}
Response: Agreed - Question is now SAT.
38 H
2 N
S NRC: Question is SAT as written.
39 H
3 M
S NRC: 1 out-of 2 twice {Original Bank question provided} [2016 ILT Exam Significantly Modified] [Reference Provided-NF-86186-3,-4]
ES-401 4
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 4 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Applicants required to work through the logic diagrams. Agreed -
Question is SAT.
40 H
3 M
E S
NRC: {Original Bank question provided.} {Correct K/A IR=3.9*} [2016 ILT Exam Significantly Modified]
Response: Agreed - Question is now SAT.
41 H
3 M
E S
NRC: {Original Bank question provided.} [2016 ILT Exam Significantly Modified]
[Reference Provided-1E-0 Attachment L pg. 4]
Response: Remove the reference. Agreed - Question is now SAT.
42 H
2 N
E S
NRC: {Correct K/A IR=3.2*}
Response: Agreed - Question is now SAT.
43 F
2 N
S NRC: Question is SAT as written.
44 F
2 X
X N
U S
NRC: Which comes first the reactor or the turbine trip? Is there only 1 correct answer? Is it actually B? (Does the turbine trip cause the Reactor trip at power?)
Response: Agreed - The stem was reworded to ask what the result will be of a loss of Both MFWP, therefore only one answer is correct. Question is now SAT.
45 H
2 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
46 H
2 N
S NRC: [Reference Provided-B20.2 pg. 18]
Response: Providing the reference requires the applicant to determine the normal line-up. Agreed - Question is SAT.
47 H
3 N
E S
NRC: 1 out-of 2 twice. {Correct K/A IR=2.8*}
Response: Agreed - Question is now SAT.
48 F
2 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
49 F
2 N
E S
NRC: 1 out-of 2 twice. {Correct K/A IR=2.7*}
Response: Agreed - Question is now SAT.
50 H
2 N
E S
NRC: 1 out-of 2 twice. [Reference Provided-1C20.7 pg. 12-15]
Response: Remove reference. (Validate Stem ICs.) Agreed - Question is now SAT.
51 H
2 M
B E
S NRC: This question does not appear to be modified.
Response: Agreed - This is a B question. Question is now SAT.
52 H
2 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
53 H
3 M
S NRC: {Original Bank question provided.} [2014 ILT Exam Significantly Modified]
Question is SAT as written.
ES-401 5
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 5 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 54 H
2 N
E S
NRC: 1 out-of 2 twice. {Correct K/A IR=2.8*}
Response: Agreed - Question is now SAT.
55 H
3 B
S NRC: [2015 LOR Exam] [Reference Provided-XH-1-242]
Response: Providing the reference requires the applicant to work through the logic diagram. Agreed - Question is SAT.
56 H
3 N
S NRC: [Reference Provided-C47012-0507]
Response: Reference provided to support plausibility of distractors. Agreed -
Question is SAT.
57 F
2 N
S NRC: Question is SAT as written.
58 H
3 N
S NRC: Question is SAT as written.
59 F
3 N
E S
NRC: {Correct K/A IR=2.9}
Response: Agreed - Question is now SAT.
60 F
2 N
S NRC: Question is SAT as written.
61 H
F 2
N E
S NRC: This appears to be a simple limit value memory question not Higher Cog.
Is Distractor D (30 Hg) plausible for the facilitys physical location?
Response: Agreed -changed Distractor D to 25. Question is now SAT.
62 H
2 M
S NRC: {Original Bank question provided.} [2016 ILT Exam Significantly Modified]
Question is SAT as written.
63 F
3 N
S NRC: [Reference Provided-SP1783.1A pg. 6]
Response: Providing the reference requires the applicant to determine which annunciators are expected. Agreed - Question is SAT.
64 H
4 M
S NRC: 1 out-of 2 twice. {Original Bank question provided.} [2017 LOR Exam]
[Reference Provided-PINGP 0045, C25 Fig. 1]
Response: Matches K/A! Agreed - Question is SAT.
65 F
2 N
S NRC: Question is SAT as written.
66 H
3 M
S NRC: 1 out-of 2 twice. {Original Bank question provided.} [2014 ILT Exam Significantly Modified] [Reference Provided-Fig. C1-8, TS 3.1.4 pg. 1]
Response: Reference provided to allow calculation for K/A. Agreed - Question is SAT.
67 H
3 B
M E
S NRC: 1 out-of 2 twice. {Original Bank question provided.} [2016 ILT Exam Significantly Modified] This question is modified; why is it listed simply as a B question?
ES-401 6
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 6 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Response: Agreed - Question is now SAT.
68 F
2 X
N U
S NRC: Asking the question in the per TS 3.6.6 Bases... puts this into the SRO-Only arena. Is there another RO knowledge document that also discusses why this is done?
Response: Agreed - the purpose is referenced to the procedural requirements.
Question is SAT.
69 H
3 X
N U
S NRC: 1 out-of 2 twice. Again, is there somewhere else that defines what qualified backup equipment is, other than the TS bases? This could also be SRO-Only. {Fix the grammar in the stem.}
Response: Agreed - The stem was modified to add load test of portable charger and provided the installation procedure. Question is SAT.
70 H
2 N
S NRC: Question is SAT as written.
71 F
2 N
N U
S NRC: How does knowing the general location of an annunciator, demonstrate knowledge of the process used to track an inoperable annunciator? (K/A match)?
Response: Stem was modified to instruct applicant to follow required procedure.
Question is SAT.
72 H
3 N
S NRC: 1 out-of 2 twice. [Reference Provided-RWP 180051 Tasks 101/203]
Response: Applicant must decide which RWP is appropriate and then implement requirements to answer the question. Agreed - Question is SAT.
73 F
3 B
S NRC: [2016 ILT Exam] Question is SAT as written.
74 F
2 N
S NRC: 1 out-of 2 twice. Question is SAT as written.
75 F
3 N
S NRC: {Fix the grammar in the stem. Remove the first in.} Question is SAT as written.
76 H
3 N
Y N
U S
NRC: How does performing the next Step in 1ES-1.1 (which would be performed regardless) relate to verifying the existence of Natural Circulation for this question?
(K/A Match)? [Reference Provided-1ES-1.1] Why is the reference required for an SRO to answer this question?
Response: Changed distractor to include action if natural circulation is not verified, requiring the use of the reference to support the correct answer. Now matches the K/A. Question is SAT.
77 H
3 Y
N E
S NRC: 1 out-of 2 twice. [Reference Provided-TSs-3.5.1 and 3.5.4] Why are the TSs provided as references for an SRO to answer this question?
Response: Agreed - Removed the SRs from the references. Question is now SAT.
78 H
2 Y
N S
NRC: Question is SAT as written.
79 H
2 Y
B E
NRC: {Correct K/A Statement for 2.2.40 vs 2.2.22}
ES-401 7
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 7 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S
Response: Agreed - Corrected. Question is now SAT.
80 H
2 Y
N S
NRC: 1 out of 2 twice. Question is SAT as written.
81 H
3 N
Y M
U S
NRC: How does selecting which contingency procedure to enter relate to the ability to identify post-accident instrumentation for this question? (K/A Match)? [Reference Provided-1E-1 pg. 9-12]
Response: The applicant must diagnose the ICs, using the EOP reference to select the proper transition point. Agreed. Changed K/A to random selection - 2.4.4. Re-worked the question to match the K/A. Agreed - Question is SAT.
82 H
2 Y
N S
NRC: Question is SAT as written.
83 H
2 Y
N S
NRC: [Reference Provided-PINGP 1576]
Response: Reference does not answer any additional questions and requires the applicant to review all possibilities. Agreed - Question is SAT.
84 H
3 Y
N S
NRC: [Reference Provided-F5 Appendix A Zone 50]
Response: Reference provided to better match K/A. Agreed - Question is SAT.
85 H
3 Y
N S
NRC: 1 out of 2 twice. Question is SAT as written.
86 H
2 N
E S
NRC: Perhaps it would be better to reword the stem to state that The Unit Supervisor will direct transition into... [Reference Provided-1ES-1.2]
Response: Removed the bases section of the reference procedure. Agreed -
Question is now SAT.
87 H
3 Y
N E
S NRC: 1 out-of 2 twice. [Reference Provided-PINGP 577 and F3-2.1 pg. 6-F 17]
Response: Remove the F3 reference. Agreed - Question is now SAT.
88 H
2 Y
N S
NRC: [Reference Provided-1ES-1.2]
Response: See Question 86. (Already Provided) Question is SAT as written.
89 H
3 Y
N S
NRC: [Reference Provided-1FR-H.1 pg. 3-10]
Response: Providing the reference requires the applicant to work through the procedure. Agreed - Question is SAT.
90 H
3 Y
N E
S NRC: {Correct K/A IR=4.3} [Reference Provided-1ECA-0.0 pg. 6-7]
Response: Providing the reference requires the applicant to demonstrate how to implement the NOTE. Agreed - Question is now SAT.
91 H
2 Y
N S
NRC: Question is SAT as written.
92 H
2 Y
N E
S NRC: [Reference Provided-C20.3 AOP1 Fig. 5 and TS 3.8.1]
Response: Remove the SRs. Fig. OK. Agreed - Question is now SAT.
ES-401 8
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 8 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 93 H
3 Y
B S
NRC: 1 out-of 2 twice. [2016 ILT Exam] [Reference Provided-H4 Table 2.2]
Response: Reference requires applicant to diagnose and implement corrective actions. Agreed - Question is SAT.
94 H
2 Y
M E
S NRC: {Correct K/A IR=4.4} [Reference Provided-C35 AOP6]
Response: Removed the system diagram. Agreed - Question is now SAT.
95 H
2 Y
N S
NRC: [Reference Provided-PINGP 1125 pg. 20-21]
Response: Reference provided to match the K/A. Question is SAT as written.
96 H
3 Y
N S
NRC: [Reference Provided-FP-OP-RSK-01 pg. 18]
Response: Table provided as reference required for calculating answer. Window dressing (ICs) provided for Operational validity. Agreed - Question is SAT.
97 H
2 Y
N S
NRC: 1 out-of 2 twice. Question is SAT as written.
98 H
2 Y
B S
NRC: [2016 ILT Audit Exam] Question is SAT as written.
99 H
3 Y
N E
S NRC: 1 out-of 2 twice. [Reference Provided-C34 AOP1]
Response: Limit the procedure provided to the first 20 pages. Agreed - Question is now SAT.
100 H
2 Y
B S
NRC: [2014 ILT Audit Exam] Question is SAT as written.
RO TOTALS:
B=14 F=30 E=21 Additional Notes: H = 38-45; N 8; B 56 M=10 H=45 U=5 N=51 SRO TOTALS:
B=4 F=0 E=8 Additional Notes: H 13; N 3; B 18 M=2 H=25 U=2 N=19
GENERAL COMMENT
S:
- 1. Question numbers highlighted as xx are sample test items the facility submitted for pre-review.
- 2.
Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
- 3.
Questions from the previous 2 NRC Exams: _3 (8)_ (RO)/ _1 (2)_ (SRO)
[Max (without OLB concurrence) is 4 RO and 2 SRO.]
ES-401 9
Form ES-401-9 Facility: PRAIRIE ISLAND NUCLEAR GENERATING PLANT UNITS 1 and 2 Exam Date: September 10 - 21, 2018 Page 9 of 9
- 5.
The 10CFR55.41/43 distribution is: RO / SRO 41.1 =
41.8 =
43.1 =
41.2 =
41.9 =
43.2 =
41.3 =
41.10 =
43.3 =
41.4 =
41.11 =
43.4 =
41.5 =
41.12 =
43.5 =
41.6 =
41.13 =
43.6 =
41.7 =
41.14 =
43.7 =
- 6.
The answer distribution is:
/
SRO A = 0 (17%)
/
0 (24%)
B = 0 (22%)
/
0 (28%)
C = 0 (28%)
/
0 (20%)
D = 0 (33%)
/
0 (28%)
- 7.
There are _13_ (RO)/ _16_ (SRO) questions with references/attachments provided.