ML22131A204
| ML22131A204 | |
| Person / Time | |
|---|---|
| Site: | Davis Besse  |
| Issue date: | 02/24/2022 |
| From: | Chuck Zoia NRC/RGN-III/DRS/OLB |
| To: | Energy Harbor Corp, FirstEnergy Nuclear Operating Co |
| Zoia C | |
| Shared Package | |
| ML20140A262 | List: |
| References | |
| L-22-061 | |
| Download: ML22131A204 (18) | |
Text
ES-401 Facility: Davis-Besse Year: 2022 Written Examination Review Worksheet Form ES-401-9 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist>: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only. (K/A and license-level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 18 61 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 2
M S
2X2 ANO 2014 NRC Exam Q1; 2X2 2
F 2
M B
E S
NRC: Bank Question - To be considered a significantly modified question, at least one pertinent condition in the stem and at least one distractor must be changed from the original bank question.
Response
Question Source changed to Bank NRC: Question is now SAT.
3 H
3 B
S 4
H 3
B S
ANO 2014 NRC Exam Q4 5
F 2
B S
NRC Exam DB 2013 Q5
ES-401 Facility:
/Year:
2 Form ES-401-9 Page 2 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 6
H 2
M B
E S
2x2 NRC: Bank Question - To be considered a significantly modified question, at least one pertinent condition in the stem and at least one distractor must be changed from the original bank question.
Response
Question Source changed to Bank NRC: Question is now SAT.
7 H
2 B
S DB 2013 NRC exam Q8; 2X2 8
H 3
M E
S EARLY SUBMITTAL NRC:
- 1)
Spell out ARTS and any other non-universal abbreviations in the question or pedigree to enhance understanding. For example, also spell out SFRCS.
- 2)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate
- 3)
The pedigree needs to clearly focus on why the incorrect answer is wrong and why the correct answer is the only right answer.
- 4)
Typo - if they fail too automatically open
- 5)
Add per DB-OP-02000 RPS, SFAS, SFRCS TRIP, OR SG TUBE RUPTURE to the question stem.
- 6)
What is the proposed Level Of Difficulty for this question?
- 7)
The pedigree has to indicate the TIER being tested (ES-401-5).
Response: Comments incorporated NRC: Question is now SAT.
9 H
3 X
N E
S EARLY SUBMITTAL NRC:
- 1)
Spell out SFRCS and any other non-universal abbreviations in the question or pedigree to enhance understanding.
- 2)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate
- 3)
The pedigree needs to clearly focus on why the incorrect answer is wrong and why the correct answer is the only right answer.
- 4)
How do we know that BOTH MSIVs are open as required?
- 5)
What is the proposed Level Of Difficulty for this question?
ES-401 Facility:
/Year:
3 Form ES-401-9 Page 3 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6)
Should refer to ICS Cabinet 1 - ICS Cabinet C5761 is much more specific than other distractors and may be a cue.
- 8)
The pedigree has to indicate the TIER being tested (ES-401-5).
Response: Comments incorporated NRC: Question is now SAT.
10 H
3 B
S NRC Exam DB 2011 Q10; 2X2 11 H
3 X
M E
S TMI 2010 NRC Exam Q45 NRC: Closing the PORV Block Valve to prevent loss of inventory is not plausible, and the distractor unnecessarily justifies itself, unlike the other distractors.
Response
Changed distractor A. to, Limit Feedwater Flow to 300 gpm if restoring Steam Generator (SG) level to a single dry SG.
Post Walkthrough validation comments Updated distractor D to make different than distractor C, which includes opening the PORV.
NRC: Question is now SAT.
12 H
2 X
X N
U S
~2X2 NRC: 1) Both Trains of Auxiliary Feedwater will be Shutdown is not plausible, and with 4 unique choices for part 2, a 2X2 question is not needed. 2) competition should be completion in the stem. 3) The question needs to specifically address load reduction, not just mention it in the stem.
Response
- 1)
Removed part 1 of the question.
- 2)
Fixed typo
- 3)
The term for Load Reduction at DB is Load Shedding. Which is already in the question.
NRC: Question is now SAT.
13 H
2 X
M B
E S
NRC: 1) A MDFP is not plausible in a LOOP situation. 2) This question could assist in answering Q#12 as originally proposed - suggest refocusing Q#12. 3)
Bank Question - This question is not significantly modified.
Response
- 1) Changed distractor A.
- 2) Q12 is not like Q13. Q12 is a Station Blackout Event with DC Load shedding
ES-401 Facility:
/Year:
4 Form ES-401-9 Page 4 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only in progress. Q13 is a LOOP with all systems responding normally. Q13 will not assist in answering Q12.
- 3) Question Source changed to Bank NRC: Question is now SAT.
14 F
3 M
B E
S NRC: 1) Bank Question - This question is not significantly modified. 2) for HX11B is included in all distractors and is not needed. Common items can be brought up into the stem for clarity and to eliminate excess words.
Response
- 1) Question Source changed to Bank
- 2) HX11B brought up into the stem NRC: Question is now SAT.
15 F
H 3
X B
N E
S EARLY SUBMITTAL NRC:
- 1)
Spell out SFAS and any other non-universal abbreviations in the question or pedigree to enhance understanding.
- 2)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate.
- 3)
What is the proposed Level of Difficulty for this question?
- 4)
The pedigree has to indicate the TIER being tested (ES-401-5).
- 5)
The question tests the reason for system lineups that are maintained, versus automatic alignments within the nuclear service water that occur from the actuation of the ESFAS as the K/A requires.
Response: The original question was replaced to address the K/A concern.
NRC: The new question is SAT.
16 F
3 X
N E
S
~2X2 NRC: With 4 unique answers in part 2 of the answer choices, the answer can be determined on that basis alone. If both parts of the question are needed, there should only be 2 choices for part 2 distractors.
Response
Rewrote question to eliminate 2x2 question.
Walkthrough comments Updated stem and question as discussed.
NRC: The new question is SAT.
17 H
3 X
N E
EARLY SUBMITTAL
ES-401 Facility:
/Year:
5 Form ES-401-9 Page 5 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S
NRC:
- 1)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate.
- 2)
The pedigree needs to clearly focus on why the incorrect answer is wrong and why the correct answer is the only right answer - Example: Distractor C.
- 3)
What is the proposed Level of Difficulty for this question?
- 4)
The pedigree has to indicate the TIER being tested (ES-401-5).
- 5)
The question tests a specific response for a degraded grid without requiring the knowledge related to the definitions of specific electrical terms, which only addresses the K/A partially. To more fully addresses the K/A, the distractors could be revised to elicit the knowledge of the listed terms in order to correctly answer the question.
Response: The original question was modified to address the K/A concern.
NRC: The new question stem should say, Which of the following actions should be taken to maintain Grid stability, and why? since the distractors also explain why.
Response
Added, and why to the question.
NRC: The question is now SAT.
18 H
3 N
S 19 H
3 N
S 20 H
2 X
N E
S NRC: Omit the unnecessary statements which reduce plausibility of distractors:
- 1)
Distractor A - A hold of up to 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> is allowed during a reactor startup.
- 2)
Distractor D - High startup rates can lead to fuel damage.
Response
Omitted the unnecessary statements from distractors A and D.
Walkthrough comments
- 1.
Updated Justifications as discussed
- 2.
Evaluated overlap with Scenario 5. Both have a NI 3 failure however the plant conditions are different.
- a.
(Q20 - approach to criticality - taking 1/M data)
- b.
(Scenario 5 - The plant is critical at 10E-8 and pulling rods to the POAH)
ES-401 Facility:
/Year:
6 Form ES-401-9 Page 6 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only NRC: The question is now SAT.
21 F
3 B
S DB 2013 NRC Exam Q21 22 F
2 X
N E
S NRC: Leaving the Control Room without tripping the reactor is implausible.
The question could be vastly improved by asking for those actions needed in addition to tripping the reactor and bring tripping the reactor up into the stem of the question.
Response
Revised question as suggested.
NRC: The question is now SAT.
23 H
2 M
B E
S 2011 Crystal River NRC Exam Q53 NRC: This question is not significantly modified - Bank question should be indicated in the pedigree.
Response
Question Source changed to Bank NRC: The question is now SAT.
24 H
2 X
N E
S 2X2 NRC: This is effectively a 2X2 question where Raise SG 1 Level to can be brought up into the stem for clarity (since it appears in all distractors).
Response
Revised question as suggested.
NRC: The question is now SAT.
25 H
2 X
N E
S NRC: Affirm in the pedigree that DB-OP-02014, Alarm Panel 14 Annunciator 14-3-D Immediate Actions are required to be known from memory
Response
Immediate Actions for ARPs are not required to be known from memory.
Revised question to ask what the required action is, since ROs are responsible for knowing their actions when ICS stations are in Hand. Updated question and distractors as discussed NRC: The question is now SAT.
26 H
2 X
N E
S NRC: For distractor consistency, either omit DH63 and DH64 from distractor A OR include similar details in the other distractors.
Response
Omitted DH63 and DH64 from distractor A
ES-401 Facility:
/Year:
7 Form ES-401-9 Page 7 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only NRC: The question is now SAT.
27 F
2 X
B E
S 2018 DB Initial License NRC Exam Q26 NRC: Distractor plausibility would be greatly enhanced if they were less detailed. For example, distractor A could be reduced to to prevent RCP Seal damage and distractor B, similarly, could be to shift SG Level Control signal to the 124-inch setpoint, for example. Also, the beginning of the sentence in the stem should now be Immediate tripping the Reactor Coolant Pumps is required ___________ to better align with these choices.
Response
Revised distractors as suggested NRC: The question is now SAT.
28 H
3 B
S 2011 DB NRC Exam Q29; 2X2 29 F
3 X
X N
E S
NRC:
- 1) The word automatically appears in 2 of the 4 distractors, and one of those is the correct answer!
- 2) Explain the necessity/relevance for limiting the correct answer to automatic design features or interlocks.
Response
- 1) Removed the word Automatically from B and C.
- 2) Removed the word Automatically from the stem.
NRC: The question is now SAT.
30 F
2 N
S 31 F
2 N
S 32 H
2 B
S 33 H
2 N
S 2X2 34 F
2 M
B E
S NRC: Bank Question - This question is not significantly modified.
Response
Question Source changed to Bank NRC: The question is now SAT.
35 H
3 M
S Crystal River 2011 NRC Exam Q16; 2X2 36 F
2 B
S DB 2013 NRC Exam Q35
ES-401 Facility:
/Year:
8 Form ES-401-9 Page 8 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 37 H
3 N
S 38 F
3 M
B E
S NRC: Bank Question - This question is not significantly modified.
Response
Question Source changed to Bank NRC: The question is now SAT.
39 F
H 2
X X
X N
E S
EARLY SUBMITTAL NRC:
- 1)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate. DONE
- 2)
Include the ACTUAL long term core cooling requirement in the pedigree for distractor D, or explain why 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> is incorrect. N/A
- 3)
What is the proposed Level of Difficulty for this question?
- 4)
The pedigree has to indicate the TIER being tested (ES-401-5). DONE
- 5)
The question tests the candidates knowledge of the specific design criteria met for the Emergency Core Cooling System from SFAS actuation, versus knowledge of the effect that a loss or malfunction of the SFAS will have on the fuel, which is the reverse of what the K/A asks.
DONE Response: Revised the question to address issues.
NRC: While the original issues with the question were addressed in rewriting it as noted above, the newly proposed H question now has 2 implausible distractors, B and D, which normally is considered UNSAT.
Response
Revised distractors, B and D.
NRC: The question is now SAT.
40 H
3 N
E S
EARLY SUBMITTAL; 2X2 NRC:
- 1)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate.
- 2)
What is the proposed Level Of Difficulty for this question?
- 3)
The pedigree has to indicate the TIER being tested (ES-401-5).
Response: Revised the pedigree to address issues.
NRC: The question is now SAT.
41 F
3 B
S DB 2015 NRC Exam Q42
ES-401 Facility:
/Year:
9 Form ES-401-9 Page 9 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only NRC: NOTE - this question was entirely different from the NRC exam question referenced in the Question Sources document that you provided as a reference. The actual question from the NRC was correct and this question is SAT. Updated Question Source folder to have the correct question 42 F
2 X
N U
S NRC: A SGTL, as opposed to a SGTR, is NOT a LOCA, since a LOCA, by definition, implies that normal makeup pump capacity is exceeded, and therefore a reactor trip would be required. A leak is less severe, requiring only a controlled S/D, as the proposed question states. The K/A, on the other hand, asks for Flow paths of steam during a LOCA, which is different.
Response
- 1)
Changed to SGTR > MU capacity, and updated justification
- 2)
Changed distractor C since this could be a correct answer.
NRC: The question is now SAT.
43 F
2 N
S 44 H
3 X
M S
2X2 45 H
3 M
B E
S NRC: Bank Question - This question is not significantly modified.
Response
Question Source changed to Bank NRC: The question is now SAT.
46 H
3 B
S 47 F
2 M
S DB 2016 NRC Exam Q49 48 H
3 N
S 49 F
3 X
M N
U S
TMI 2 - 2014 NRC Exam Q43 NRC: The question requires knowledge which is below the line and not considered RO knowledge (See ES-401, Page 20 of 52).
Response
Wrote New Question.
NRC: The new question is SAT.
50 H
2 B
S DB 2016 NRC Exam Q71 51 F
2 B
S DB 2020 NRC Exam Q53 52 H
2 N
S
ES-401 Facility:
/Year:
10 Form ES-401-9 Page 10 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 53 F
2 X
N E
S NRC:
- 1) As given, this is a simple memory question which could be improved by removing SFRCS has not Actuated from the stem.
- 2) In the pedigree, lowing should be lowering
Response
- 1) a. Left SFRCS has not Actuated in the stem. SFRCS has not actuated is needed, since by procedure the reactor is tripped and SFRCS is actuated at 75 psig, which would cause the valves to close prior to supply air reaching 60psig.
- 1) b. Removed the word instrument from the question.
- 2) Fixed typo in the pedigree NRC: The question is now SAT.
54 F
2 X
M E
S EARLY SUBMITTAL NRC:
- 1)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate.
- 2)
Spell out SAM and any other non-universal abbreviations in the question and/or pedigree to enhance understanding.
- 3)
What is the proposed Level of Difficulty for this question?
- 4)
The pedigree has to indicate the TIER being tested (ES-401-5).
- 5)
The answer choices seem to be a collection of true/false statements, testing simple rote memory (see App. B, page B-24).
Response: Comments incorporated Post Walkthrough validation comments Removed MU2 from the stem. Which was also in question 55.
NRC: Question is now SAT.
55 F
2 N
E S
NRC: Does following a SFAS Level 2 actuation need to be repeated?
Response
Removed the first following a SFAS Level 2 actuation from the question.
NRC: Question is now SAT.
56 H
3 N
S 57 H
2 M
B E
S NRC: Bank Question - This question is not significantly modified.
Response
ES-401 Facility:
/Year:
11 Form ES-401-9 Page 11 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Question Source changed to Bank NRC: Question is now SAT.
58 F
3 B
S DB 2020 NRC Exam Q58; 2X2 59 F
2 X
X N
U S
NRC: The question asks what ensures a negative pressure in Containment when the System is operating, and distractors C and D state that neither criteria are met and are implausible choices.
Response
Changed Distractors C and D.
NRC: Question is now SAT.
60 H
3 N
S 2X2 61 H
3 B
S DB 2020 NRC Exam Q43; 2X2 Post Walkthrough validation comments Changed stem to All other Turbine Bypass Valves respond to Steam Generator Pressure as expected to prevent confusion with distractor A.
NRC: Question is SAT.
62 F
2 N
E S
NRC: As written, this question has minimal discriminatory value - hydrogens explosiveness is common knowledge, and the distractors given offer weak alternative choices. To correct this, the applicant could be required to recall the explosive limit and/or other DB-OP-06210 information pertaining to hydrogen purity deterioration.
Response
Wrote New Question.
Post Walkthrough validation comments
- 1)
Added seal oil vacuum s/d for reason why purity was lowering
- 2)
Changed distractor C. to make it more wrong. It could be argued as correct since no action has been taken yet to restore H2 purity.
NRC: Question is now SAT.
63 H
3 M
B E
S NRC: Bank Question - This question is not significantly modified.
Response
Question Source changed to Bank NRC: Question is now SAT.
64 F
2 M
B E
S NRC: Bank Question - This question is not significantly modified.
Response
ES-401 Facility:
/Year:
12 Form ES-401-9 Page 12 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Question Source changed to Bank NRC: Question is now SAT.
65 H
2 N
S 66 F
2 N
S 67 F
2 X
N U
S NRC: Distractors B and D are not credible, and omit should from Distractor C, if used as a distractor. Suggest bringing should be reviewed into the stem, and having the applicant consider various options for reviewing the Night Orders
Response
Modified Distractors B and D NRC: Question is now SAT.
68 F
2 X
X B
E S
TMI 2014 NRC exam Q68 NRC:
- 1)
Distractor plausibility cannot generically be attributed to stating that permission from both is required prior to bypassing any installed Equipment Bypasses. This is, in effect, saying it is because the correct answer is correct.
- 2)
The question asks for conditions when fuel stored in the Spent Fuel Pool (SFP) is being loaded into the Reactor Vessel, but 2 distractors address SFP actions (B & C).
Response
- 1)
Updated plausibility
- 2)
Changed stem as requested Walkthrough comments Updated distractor D. as discussed NRC: Question is now SAT.
69 F
2 X
B E
S DB 2011 NRC Exam Q68 NRC:
- 1)
Address plausibility of each of the incorrect responses alone.
- 2)
Consider eliminating Setpoints, regrouping and renumbering. Then, only the plausibility of Changes to equipment position and Changing the purpose of the procedure would be required as incorrect choices.
Response
ES-401 Facility:
/Year:
13 Form ES-401-9 Page 13 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 1) Addressed plausibility of each of the incorrect responses alone.
- 2) Left as-is since validation was completed on this question.
Walkthrough comments Updated distractors and justifications as discussed NRC: Question is SAT.
70 F
2 B
S DB 2016 NRC Exam Q69; 2X2 71 H
3 B
S DB 2011 NRC Exam Q70; 2X2; Embedded Reference 72 F
2 B
S DB 2015 NRC Exam Q72; 2X2 73 F
2 X
B U
S 2X2 NRC: The procedures offered as distractors are not plausible for being required to be implemented. Consider addressing other K/A aspects instead of procedure selection
Response
Selected a different bank question.
NRC: Question is now SAT.
74 H
2 N
E S
NRC: Performing a Crew Update and performing immediate actions from memory appears to be an obvious choice, while choice B is implausible during this event. Consider a less universally obvious correct answer and a more plausible distractor for distractor B.
Response
Revised as suggested. Changed Answer and distractor B Walkthrough comments Updated distractors and justifications as discussed NRC: Question is now SAT.
75 F
2 B
E/U?
S NRC: Affirm within the pedigree that the notification information is required to be known from memory by RO-level applicants.
Response
RO Knowledge, lesson plan OPSGOPI603 added to pedigree Walkthrough comments
- 1.
Updated distractors and justifications as discussed
- 2.
Added specific objective number to justification NRC: Question is now SAT.
ES-401 Facility:
/Year:
14 Form ES-401-9 Page 14 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 76 H
3 X
X N
E S
EARLY SUBMITTAL; 2X2 NRC:
- 1)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate.
- 2)
Spell out MUP and any other non-universal abbreviations in the question and/or pedigree to enhance understanding.
- 3)
What is the proposed Level Of Difficulty for this question?
- 4)
The pedigree has to indicate the TIER being tested (ES-401-5).
- 5)
A SGTR is a special case of an RCS Leak, which is different from a SBLOCA in several important ways. Thus, the question does not meet the K/A.
- 6)
The question does not require the selection of the correct procedure to mitigate the event. Instead, it requires the applicant to recognize that a reactor trip is required when Pressurizer Level becomes less than 100 inches, a Reactor Trip Setpoint, while utilizing DB-OP-02000. This is RO-level knowledge.
- 7)
Explain why a loss of all Makeup Pumps could be a possibility.
Response: Question was revised to address concerns.
NRC: Revised question is SAT 77 H
2 N
S 78 H
3 N
S 79 F
2 X
B E
S EARLY SUBMITTAL NRC:
- 1)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate.
- 2)
What is the proposed Level Of Difficulty for this question?
- 3)
The pedigree has to indicate the TIER being tested (ES-401-5).
- 4)
Reducing off-site dosage is basic SGTR strategy and not specific SRO-only knowledge. ES401 Attachment 2 Item II. A. Conditions and Limitations in the Facility License cites several examples that clearly differ from basic RO-level knowledge within this category.
Response: Question was revised to address concerns.
NRC: Revised question is SAT 80 F
3 M
S DB NRC Exam 2016 Q13; 2X2
ES-401 Facility:
/Year:
15 Form ES-401-9 Page 15 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 81 H
3 M
S ANO U2 NRC Exam 2011 Q78; Reference Provided 82 H
2 N
S 83 H
2 N
S 84 H
3 N
S 85 H
3 B
E S
DB NRC Exam 2011 Q80 NRC:
- 1)
Supplemental Action should be Supplemental Actions in the question stem.
- 2)
A SGTR is a special case of a LOCA, but the K/A and the question appropriately focuses on RCS cooldown. The pedigree should address similarities, differences, and reasons for acceptability, not just say they are the same thing
Response
- 1) Changed as requested
- 2) Fixed pedigree NRC: Question is now SAT.
86 H
3 X
N E
S NRC:
- 1)
Omit Continued operation of RCP 2-2 at 100% power is permitted as long as Reactor Coolant System Leakage remains within Technical Specification Limits from Distractor A - this belongs in the pedigree.
- 2)
What action is required to be performed next for Distractor A?
Response
- 1)
Omitted Continued operation of RCP 2-2 at 100% power is permitted as long as Reactor Coolant System Leakage remains within Technical Specification Limits from distractor, placed in pedigree, as requested.
- 2)
Added Evaluate Reactor Coolant System Leakage to distractor A NRC: Question is now SAT.
87 H
3 N
E S
NRC: Fix typo(s) in the pedigree, such as doe (due) in Distractor C explanation.
Response
Fixed typos
ES-401 Facility:
/Year:
16 Form ES-401-9 Page 16 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only NRC: Question is now SAT.
88 H
2 N
E S
NRC: This question is minimally discriminating - given the high containment pressure, it is easy to determine that a large LOCA is occurring. Consider using a different accident and more subtle indications. This would result in a question that would be more challenging to diagnose.
Response
Revised question as suggested.
NRC: Question is now SAT.
89 H
2 X
N U
S 2X2 NRC: It is not plausible that Shift Manager approval is required before attempting to manually start Containment Spray Pump 1 that failed to autostart or for all Control Room Actions. Since this is a 2X2 question, this results in 2 implausible distractors.
Response
Changed Shift Manager to Command SRO.
Walkthrough comments Changed Q89 question and justification as discussed.
NRC: Question is now SAT.
90 H
2 X
N E
S 2X2; Reference Provided NRC: Unusual Event (Distractor A) is implausible for this situation. Another incorrect Site Area Emergency might be more discriminating under these conditions.
Response
Revised Distractor A as requested NRC: Question is now SAT.
91 H
2 X
X N
E S
EARLY SUBMITTAL NRC:
- 1)
Please provide a copy of the question files without hidden text in them so they can be added to ADAMS when appropriate. DONE
- 2)
Spell out SBODG and any other non-universal abbreviations in the question and/or pedigree to enhance understanding. DONE
- 3)
What is the proposed Level Of Difficulty for this question? DONE
- 4)
The pedigree has to indicate the TIER being tested (ES-401-5). DONE
- 5)
The first part of Distractor A differs from the others because it includes to minimize inventory losses from the RCS in addition to saying
ES-401 Facility:
/Year:
17 Form ES-401-9 Page 17 of 18 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only something similar in the second part. This may provide a cue and should be deleted. DONE
- 6)
The second part of the distractors (since there are 4 unique choices) can be used to answer the question/eliminate choices with RO knowledge.
Response
Part A must be answered first, which is SRO knowledge, since the 2nd part relates to the first part - it is not independent. Left as-is.
Walkthrough comments Clarified Q91 question and justification as discussed.
NRC: Question is now SAT.
92 H
3 N
S 93 F
2 X
N U
S NRC: The note requiring reviewing LCO 3.1.1 is above the line and is therefore RO required knowledge (see ES-401, page 20 of 52).
Response
Wrote a new question Walkthrough comments Clarified New Q93 question and justification as discussed.
NRC: Question is now SAT.
94 F
3 N
S 95 H
2 B
S DB 2013 NRC Exam Q94 96 F
3 B
S DB 2021 SRO Written Exam Q22; 2X2 97 F
3 B
S DB 2016 NRC Exam Q96 98 F
2 B
S DB 2016 NRC Exam Q98; 2X2 99 H
3 B
S DB 2016 NRC Exam Q78 100 H
3 N
S 2X2; Reference Provided
ES-401 Facility:
/Year:
18 Form ES-401-9 Page 18 of 18 RO TOTALS:
B= 33 F= 34 E= 34 Additional Notes:
M= 7 H= 41 U= 6 N= 35 SRO TOTALS:
B= 7 F= 7 E= 8 Additional Notes:
M= 2 H= 18 U= 2 N= 16
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N