ML20153A606
| ML20153A606 | |
| Person / Time | |
|---|---|
| Site: | Quad Cities  |
| Issue date: | 04/10/2020 |
| From: | NRC/RGN-III/DRS/OLB |
| To: | Exelon Generation Co |
| Roach G | |
| Shared Package | |
| ML19121A179 | List: |
| References | |
| Download: ML20153A606 (22) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: March 2020
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
LOK LOD Q# (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Reference Provided - Actual vs. Indicated Fuel Zone Level NRC: The level of difficulty of this question is not sufficiently high. Specifically, this question can be answered by applying the stem information to the provided chart without requiring any specific systems or procedural knowledge on the part of the U applicant. Additionally, this question does not address the listed K/A and only relates to it in a superficial manner (e.g. recirc pumps being tripped in the stem).
1 H 2 B Response: Revised to add an additional bullet in the stem including a second RWL S instrument (Medium Range RWL). The candidate will be required determine which indicator is used and if it must be corrected utilizing the provided chart. The candidate must analyze which indication to use with the given information that Recirc pumps are off as not all RPV water level detectors are usable in this condition.
NRC: The question is satisfactory with changes made.
E Quad Cities 2012 NRC Exam H 2 B NRC: The distractor justifications do not match the order of the provided answer 2
choices for A and C.
S Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
- Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).
- T/F: The answer choices are a collection of unrelated true/false statements.
- Cred. Dist>: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SR-only. (K/A and license-level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 22
ES-401 2 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
Free Look Question NRC: Correct answer is associated with the battery, while all three distractors are associated with the battery charger. This cues the applicant to the correct answer as to which would NOT give a battery charger alarm. Additionally, NUREG 1021, App B, E SECTION 2.e warns against using negative questions. This question follows the protocol for using a negatively stated stem in that the word NOT is capitalized. Just a 3 H 2 N warning to consider. Is there a way to reword the stem to positively ask the question?
S Response: Question revised to remove the negatively stated stem and eliminate cueing.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer choices. Additionally, for distractor C what is the basis for using a value of 460 psig? TS 3.3.1.1, Table 3.3.1.1-1, Function #11 lists a value of 475 psig. This E appears to be a more plausible alternative.
4 H 2 B S Response: Revise distractor revised from 460 to 475 psig. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: This question is written at a fundamental level of knowledge (versus the listed higher LOK). Specifically, answering this question only requires the applicant to E recall information that is contained in QCOP 0600-21, Attachment A, section 4.c.
5 F 3 B Response: Updated pedigree information to Memory level question.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer choices for B and D.
E Reference Provided - QOA 0010-05 6 F 2 B S Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
Free Look Question E
NRC: Distractors B and D are not credible in that the question asks for which valve is 7 H 2 B located inside the drywell. Distractors B and D include the name of MO 1-3703 as S the outboard return valve. As a result, these choices can be quickly disregarded.
Consider writing a question focusing on what components/systems are affected by Page 2 of 22
ES-401 3 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 isolating portions of the RBCCW system based on addressing a system leak.
Additionally, the question as written is a memory only style question.
Response: Question replaced to test the KA more directly and require analyzing the conditions to determine the location of the leak.
NRC: The question is satisfactory with changes made.
NRC: This question is written at a fundamental level of knowledge (versus the listed higher LOK). Specifically, answering this question only requires the applicant to recall information that is contained in QOA 4700-06 step D.2 and section E.
E Additionally, the stem should be specific in asking that [t]he required action per QOA F 3 N 4700-06 is to 8
S Response: Updated pedigree information to Memory level question, Procedure number added to stem.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices for C and D.
9 H 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: This question is written at a fundamental level of knowledge (versus the listed higher LOK). Specifically, answering this question only requires the applicant to recall information that is contained in QCFHP 0110-02. Additionally, the distractor E justifications do not match the order of the provided answer choices for B and D.
10 F 2 B S Response: Updated pedigree information to Memory level question. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: Distractor C may potentially be considered as a second correct answer due to an applicant being able to make unstated assumptions that are not contradicted by stem. Specifically, since the stem does not provide any information regarding RPV level, an applicant could potentially assume that RPV level is less than the value that would make C a correct answer.
11 F 3 B S Response: Statement in stem Steam leak inside containment indicates that an extreme low reactor water level would not be plausible. NRC discussion determined the question is Sat. Original version of question will be provided.
NRC: Question is satisfactory as originally written. Question is a bank question.
H 2 B S Quad Cities 2012 NRC Exam 12 NRC: None.
Page 3 of 22
ES-401 4 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 E NRC: The distractor justifications do not match the order of the provided answer choices.
13 H 3 B S Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: Distractor D is not plausible. There appears to be no basis for why an applicant would have the misconception that recirculation pumps require actions to protect them during EOP implementation. There are actual EQ components such as containment isolation MOVs inside the drywell which would be plausible. Additionally, E the distractor analysis for Distractor B appears to contain a typo. Specifically, TS 3.6.1.5 lists a drywell temperature limit of 150F, while the distractor analysis lists a 14 F 2 N temperature value of 180F.
S Response: Distractor D revised to state Recirc Pump discharge valve motors which are a qualified component the drywell but do not apply to this case.
Distractor B revised to state maximum normal operating temperature vice LCO limit.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
15 H 3 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Reference Provided - RPV Level Criteria from the EOPs 16 H 3 B S NRC: None NRC: The combination of distractor options presented creates a cue that potentially allows an applicant to logically eliminate Distractors C and D without applying systems or procedural knowledge. Specifically, the stem does not ask for what minimum action or set of actions is necessary. Therefore, since Answer A is comprised of a combination of the actions contained in Distractors C and D, an U
applicant could reason that selecting A would constitute a correct answer even if part of it was incorrect, thus preventing the need to even consider Distractor C or 17 H 2 B D. This flaw also has the effect of lowering the level of difficulty of this question unacceptably. The stem should be modified to ask [w]hat MINIMUM action(s) will S
permit (or words to that effect).
Response: Minimum added to stem question.
NRC: The question is satisfactory with changes made.
F 3 B S Quad Cities 2011 NRC Exam 18 NRC: None Page 4 of 22
ES-401 5 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: The distractor justifications do not match the order of the provided answer E choices for A and C.
19 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The stem of this question should be specific in asking that [w]hat is the reason for this action per QCOA 6000-02 and QCOA 6000-03 (or words to that effect).
E 20 F 2 N Response: Question revised to include What is the reason for this action? and the S applicable procedure names.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
21 F 2 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
22 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Free Look Question Reference Provided - Steam Flow vs. Feed Flow Indications NRC: Explain in greater detail how the various level inputs affect DFLWC.
Specifically, a LT has failed upscale which will initially cause DFWLC to see a high RPV level and close FRVs which will result in an actual low RPV level. How do the E unaffected level channels override the failed LT and cause the FRVs to come back open? Explain the logic.
Response: Reference provided clarification added. Answer explanation improved to 23 H 3 B include detailed effect of failure.
NRC: There still appears to be a disconnect in the stem between the information provided regarding the failed level transmitter/single element mode and what is S ultimately being asked (which trace shows level rising? Are you asking which trace shows how the system would respond in single element if 1 input to DFLWC failed upscale? If so the answer justification should describe why feed flow initially goes below steam flow (presumably that the failed level instrument shows level increasing so the feed reg valve starts to close.) The inflection point then occurs as feed flow increases because other level instruments inputting to DFWLC see level lowering due to the initial FRV response. The full shape of the correct answer should be Page 5 of 22
ES-401 6 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 described. If the question is solely asking which curve shows level increasing, then Distractor C may potentially be considered as a second correct answer due to an applicant being able to make unstated assumptions that are not contradicted by stem.
Specifically, based upon the distractor analysis provided for Distractor D, it appears that the combined effects of both feed flow and CRD flow represented in the chart trace provided for Distractor C would result in RPV level rising under this circumstance as well, thereby making Distractor C potentially correct based upon what the stem is asking for. Due to the previous submittal of this question as a free look question, this issue is treated as an enhancement rather than being unsatisfactory in the final assessment of submittal quality.
Response: The stem states that DFWLC is in SINGLE-ELEMENT which drives Feed directly from RWL, therefore if feed rate is constant then RWL must be steady at the setpoint. Discussion with NRC, question is Sat as left.
NRC: NRC agrees that with DFLWC in SINGLE-ELEMENT at the start of the event the there is only one correct trace. The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
24 H 3 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
25 H 3 N S NRC: None 26 H 2 N S NRC: None NRC: The distractor justifications do not match the order of the provided answer E choices.
27 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer choices. Additionally, the information provided in the distractor analysis for Distractor 1 appears to conflict with the information provided in the answer E explanation (e.g. Bus 28 versus Bus 29).
28 H 3 B S Response: Distractor 1 statement updated to the correct normal feeds for MCC 28/29-5. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
E NRC: The distractor justifications do not match the order of the provided answer 29 H 2 B choices.
S Response: Distractor justifications and answer choices aligned in correct order.
Page 6 of 22
ES-401 7 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: The question is satisfactory with changes made.
Free Look Question NRC: Spelling errors in stem for words transferred and Essential. Change bullet (1)
E to say, With NO additional operator action 30 H 4 N S Response: Question revised for simplicity, and correct spelling errors.
NRC: The question is satisfactory with changes made.
NRC: The structure of this question creates a backwards logic flaw (e.g. the question requires reverse logic or application compared to the job requirements).
Specifically, this question asks the examinee for information that is normally received (e.g. annunciators), which is contrary to NUREG-1021 Appendix B, section G.
U Additionally, answer options (2) and (4) are not plausible as there appears to be no basis for why an applicant would have the misconception that these annunciators would be illuminated given the conditions provided in the stem. This, in turn, results 31 H 2 N in all three distractors not being credible since each contains at least one of these implausible options. Finally, having three implausible distractors in this question reduces the questions level of difficulty to an unacceptably low level.
S Response: Annunciator choice 4 was removed and stem revised to remove backward logic and stem updated to state Core Spray Pump Room temperature is 110 degrees.
NRC: The question is satisfactory with changes made.
NRC: In the first half of the question, it would be preferable to ask [w]hat is the expected status of Additionally, the distractor justifications do not match the order of the provided answer choices. Furthermore, the information contained in the E distractor justifications appears to be incorrect when compared to the answer option H 3 B content.
32 S Response: Distractor statements revised for clarity and accuracy. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
33 H 3 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E
choices.
34 F 3 N S Response: Distractor justifications and answer choices aligned in correct order.
Page 7 of 22
ES-401 8 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: The question is satisfactory with changes made.
35 H 2 B S NRC: None Previous 2 Exams: Quad Cities 2016 NRC Exam NRC: The distractor justifications do not match the order of the provided answer E choices.
36 H 2 B S Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
37 H 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: Distractor B may potentially be considered as a second correct answer due to an applicant being able to make unstated assumptions that are not contradicted by stem. Specifically, Distractor B does not specify when the scram is reset; because U of this, an applicant could assume that SRMs are fully inserted into the core prior to the scram being reset. In such a scenario, it would appear that the scram reset F 3 B would have no effect on the insertion of the SRMs, thereby making Distractor B a 38 second correct answer.
S Response: Question revised to address answer choices with resetting the scram to include reset while SRMs are driving in.
NRC: The question is satisfactory with changes made.
39 F 3 N S NRC: None NRC: Specify in the stem which nuclear instrumentation procedure(s) contain the required actions (i.e. per QCXX-XXXX, what action is required). Additionally, the distractor justifications do not match the order of the provided answer choices. Bullet E dots dont match font and contain multiple dots for bullets 2 and 3.
40 H 3 B S Response: Stem revised to include procedure names. Formatting bullets fixed.
Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
41 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Page 8 of 22
ES-401 9 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 Free Look Question NRC: Spelling errors in justification for Distractor 3 (speed and valve). Correct E alignment of Trip Throttle Valve position indication so that all four answer choices F 3 N look like answer A.
42 S Response: Spelling and formatting errors fixed.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
43 H 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
44 H 4 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Free Look Question NRC: Question does not address the KA statement. KA is associated with the air supply for the safety/relief valves. This question is focused on reactor, TBV, and tailpipe temperature response for operation of the S/RVs. Recommend a question concerning the pneumatic supply to the Target Rock S/RV. Additionally, question pedigree does not indicate the source of this question.
E Response: Question source, New, added to pedigree. K/A match is from operating the keylock switches which actuate the valves. No controls are utilized directly with 45 H 3 N the air system.
NRC: KA was changed to A1.04 as indicated in the ES-401-4. Update to correct KA S in question summary. Potential cueing with three answers having RPV = 300 psig.
Recommend Distractor A have RPV pressure = 280 psig this would indicate the TBVs remained unchanged (2 VALVES FULL OPEN) with additional steam flow through SRV.
Response: KA updated to A1.04. Distractor A updated to 280 pisg.
NRC: The question is satisfactory with changes made.
NRC: The question pedigree does not indicate the source of this question (i.e. bank).
E Additionally, since the stem indicates that RPV level is currently at 26 inches, the first H 3 N half of Distractor C is essentially saying the same thing as the first half of Answer 46 D. This question may potentially be constructed in a more effective manner by S modifying it into a balanced two-part question format. Finally, the distractor justifications do not match the order of the provided answer choices.
Page 9 of 22
ES-401 10 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 Response: Question pedigree updated to include question source as New.
Question stem and answer choices revised to a more fluid question format. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: This question is written at a fundamental level of knowledge (versus the listed higher LOK). Specifically, answering this question only requires the applicant to E recall information related to SBGT system response to a start demand signal.
47 F 2 B S Response: Question pedigree updated to include cognitive level as Memory.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
48 H 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: Capitalize the word not in the stem.
E Distractor justifications do not match the order of the provided answer choices.
49 H 2 N Response: Question revised with NOT and procedure name included in stem.
S Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
50 F 2 B S NRC: None NRC: The distractor justifications do not match the order of the provided answer E choices.
51 H 3 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: Distractor D is not plausible; since the 1A IAC is presented in the stem as being in a standby condition, with its associated control switch in pull-to-lock (e.g.
E not running), there appears to be no basis for why an applicant would have the misconception that another air compressor must be started in order to replace it.
F 2 B Additionally, this question is written at a fundamental level of knowledge (versus the 52 listed higher LOK). Specifically, answering this question only requires the applicant S to recall the power supplies associated with the Instrument Air Compressors. Finally, the distractor analysis for Distractor 1 appears to contain a typo. Specifically, the 1/2B IAC appears to be incorrectly referred to as the 1B IAC.
Page 10 of 22
ES-401 11 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 Response: Question pedigree revised to Memory level and stem revised to directly address the KA statement.
NRC: The question is satisfactory with changes made.
NRC: This question does not match the listed K/A; specifically, this question tests knowledge associated with the Service Water System, whereas the associated K/A is for System 400000, which is the Component Cooling Water System (i.e. RBCCW).
Additionally, this question is listed as being a modified bank question, however, the original bank question is not provided. NUREG-1021 ES-201, section C.1.h states
[i]n its examination submittal to the NRC, the facility licensee (or its contractor) shall provide the following information for each test item proposed for use as part of the written examination and/or the operating test [f]or those items that were derived E from changing existing/current bank items in any way, note the changes that were made or submit a copy of the item from which it originated. Furthermore, the 53 H 2 B distractor justifications do not match the order of the provided answer choices.
Finally, the distractor analysis for Distractor 2 appears to contradict itself by stating S both that Service Water Pump auto-start features do, and do not, exist.
Response: Service Water is included with Component Cooling Water and there is no specific K/A for service water. Reworded distractor 2 explanation to remove contradiction. Discussion with NRC, the question as left will be Sat. The original question will be provided. Distractor justifications corrected and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: This question is written at a fundamental level of knowledge (versus the listed higher LOK). Specifically, answering this question only requires the applicant to recall the actions of QCAN 901(2)-5 D-3. Additionally, the distractor justifications do not match the order of the provided answer choices. Finally, the stem of this question should be specific in asking that [w]hat action is required to continue control rod E insertion per QCAN 901(2)-5 D-3 (or words to that effect).
54 F 3 B S Response: Pedigree updated to MEMORY level. Stem revised to What action is required to continue control rod insertion per QCAN 901(2)-5 D-3? Annunciator name previously stated in the stem. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
55 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
56 H 3 B U Reference Provided - M-77, Sheet 1 Page 11 of 22
ES-401 12 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: This question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
Specifically, this question requires an applicant to recall, without the aid of references, which instrument channels are located on a given instrument rack.
S Recommend an attached reference of electrical drawings. Additionally, this question is written at a higher cognitive level of knowledge (versus the listed fundamental LOK). Specifically, answering this question requires the applicant to analyze RPV level instrument failures and predict the resultant effects on both indicated RPV level and RPS (refer to NUREG-1021 Appendix B, section C.1.d). Finally, the distractor justifications do not match the order of the provided answer choices.
Response: M-print that shows instrument and rack relationships will be provided to allow candidates to confirm instrument connections. Corrected pedigree to state Higher order. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
57 F 4 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
58 H 4 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Reference Provided - MSIV and MSL Drain control board position indications NRC: Update the question pedigree to reflect the reference handouts that are provided in conjunction with this question. Fix the KA number to K4.10 as reflected in the 401-4/401-1. The wording is correct, the number needs to be changed from 09 to E 10. Additionally, the distractor justifications do not match the order of the provided answer choices.
59 H 2 N S Response: Question revised with correct KA code and pedigree with reference provided statement. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The stem of this question should be specific in asking that what action is E REQUIRED per QOA 5650-02 (or words to that effect). Additionally, the distractor justifications do not match the order of the provided answer choices.
60 H 3 N S Response: Stem updated to include procedure name and wording. Distractor justifications and answer choices aligned in correct order.
Page 12 of 22
ES-401 13 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: The question is satisfactory with changes made.
NRC: Distractor D is not plausible; there appears to be no basis for why an applicant would have the misconception that a gland exhaust issue related to the Main Turbine would affect the Main Generator in this manner. Additionally, Distractor C contains a typo; the word will is misspelled. Finally, it would be preferable to E
word the stem question in a manner similar to [w]hich of the following describes the expected effects, if any, of continued operation of the Main Turbine?
61 H 2 N Response: Question revised to include NRC suggestion for removing generator S
stator ground as an answer choice and replaced with another turbine related distractor.
NRC: The question is satisfactory with changes made.
Quad Cities 2009 NRC Exam NRC: The distractor justifications do not match the order of the provided answer E choices.
62 H 2 B S Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: This question does not match the listed K/A; specifically, NUREG-1021, ES-401, section D.2.a discusses that when testing A2 K/A statements, testing both aspects of the K/A statement should be attempted. If that is not possible, then the scope of the question should be limited to the (b) portion of the A2 K/A statement U since it requires the highest cognitive level. However, this question only tests the (a) portion of the applicable K/A statement. Additionally, Distractor D is not plausible; 63 H 2 N there appears to be no basis for why an applicant would have the misconception that S reactor power would be unaffected by changes in feedwater temperature.
Response: Question rewritten to directly address the selected K/A.
NRC: The question is satisfactory with changes made.
NRC: The stem of this question should be specific in asking that requiring the NSO to _______ per QOA 3300-02 (or words to that effect). Additionally, the E distractor justifications do not match the order of the provided answer choices.
64 H 2 B Response: Question revised to include recommended change and procedure name.
S Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: Since QOA 912-5 C-1 directs the method described in Answer C, the stem of E
this question needs to be specific in asking [w]hich of the following actions must be 65 H 3 B taken FIRST per QOA 912-5 C-1 (or words to that effect). The stem must be S specific in asking for the first action directed by QOA 912-5 C-1 because the procedure goes on to subsequently direct the action described in Distractor B at a Page 13 of 22
ES-401 14 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 later point. Finally, the distractor justifications do not match the order of the provided answer choices.
Response: Question revised to ask more specifically actions from the procedure.
Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: None 66 F 2 B S NRC: The distractor justifications do not match the order of the provided answer E choices.
67 F 2 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
68 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The stem contains grammatical errors; the semicolon after TEST should instead be a comma and the wording thereafter should instead read the Test E Director determines that the changes are needed 69 F 2 B S Response: Grammar and wording corrected.
NRC: The question is satisfactory with changes made.
NRC: The stem of this question should be specific in asking [w]hich of the following meets the requirements of the Clearance and Tagging program per OP-AA-109-101 E (or words to that effect). Additionally, the distractor justifications do not match the order of the provided answer choices.
70 F 2 N Response: Procedure number/name added to stem. Distractor justifications and S answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The stem of this question should be specific in asking for what action is E required per MA-AA-716-004.
71 F 2 B Response: Procedure number/name added to stem.
S NRC: The question is satisfactory with changes made.
Page 14 of 22
ES-401 15 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 Quad Cities 2012 NRC Exam NRC: Distractors C and D may potentially be considered as second correct answers due to an applicant being able to make unstated assumptions that are not contradicted by stem. Specifically, the offsite release rates provided in Distractors C and D both exceed the QGA-400 entry criteria threshold, thereby making these E
distractors potentially correct answer options. The stem should instead ask the applicant to select the minimum EAL/offsite release rate condition that would require 72 F 2 B entry into QGA-400. Due to the previous usage of this question on an NRC license examination, this question is being categorized as an enhancement rather than S
being unsatisfactory.
Response: LOWEST clarification added to the stem.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
73 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Free Look Question 74 F 2 N S NRC: None NRC: The distractor justifications do not match the order of the provided answer choices. Additionally, there is a typo in the distractor analysis for Distractor 3.
E Specifically, the word with is misspelled in the last sentence.
75 F 3 B Response: Distractor justifications and answer choices aligned in correct order.
S Spelling error corrected.
NRC: The question is satisfactory with changes made.
Reference Provided - TS 3.8.4 and 3.8.5 NRC: The correct answer justification is inaccurate. For the operating unit (Unit 1),
you should have entered TS 3.8.4, Condition E due to the opposite unit 125 VDC E electrical subsystem inoperable. This is also a 7-day LCO, so the answer is correct, 76 H 3 N but the justification needs to be reworded.
S Response: Question revised to reflect correct Tech Spec conditions.
NRC: The question is satisfactory with changes made.
Free Look Question E NRC: The distractor justifications do not match the order of the provided answer 77 H 3 N choices.
S Response: Distractor justifications and answer choices aligned in correct order.
Page 15 of 22
ES-401 16 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: The question is satisfactory with changes made.
NRC: This question does not match the listed K/A; specifically, this question does not test the applicants ability to determine/interpret component cooling water temperature. While the question does involve a loss of cooling water condition, the stem indicates that QCOA 3800-03 is in effect, thus not requiring any determination/interpretation of cooling water temperature conditions on the part of the applicant. Additionally, although the stem lists the name of QCOA 3800-03 as Loss U of TBCCW, the actual title of this procedure is Total Failure of the TBCCW System.
78 H 2 B Finally, the distractor justifications do not match the order of the provided answer S choices.
Response: Inability to write a technically accurate question to the SRO level for this K/A. AA2.03 reselected by NRC and the question as is will meet this new KA statement. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
Reference Provided - T.S. 3.1.8, Conditions A&B Quad Cities 2014 NRC Exam NRC: This question does not test at the SRO license level. Specifically, the stem conditions indicate that two control rods are drifting in. The correct answer (option C) is to [s]cram the reactor. QCOA 0300-11, Control Rod Drift, contains the following immediate action: [if] two or more Control Rods start to drift AND all RPS scram solenoid lights are lit, THEN scram the Reactor. NUREG-1021 ES-401, E Attachment 2, section E, provides guidance that questions that can be answered solely by knowing immediate operator actions are RO-level questions. Due to the 79 H 2 B previous usage of this question on an NRC license examination, this question is S being categorized as an enhancement rather than being unsatisfactory.
Additionally, there is a typo in the distractor analysis for Distractor 1; specifically, the word directed is misspelled.
Response: Question replaced with a scram discharge volume tech spec call.
Reference provided is clip form the tech spec page embedded in the stem.
NRC: The question is satisfactory with changes made.
Free Look Question NRC: The question as written is a memory only style question requiring only recall of TS Bases information. Should be coded as Fundamental.
E Response: Cognitive level adjusted to Memory 80 F 2 N NRC: (Note: this question was re-reviewed due to a distractor having been subsequently modified by the facility.) The distractor justifications do not match the S order of the provided answer choices.
Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
Page 16 of 22
ES-401 17 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: There is a typo in the distractor analysis for Distractor 1; specifically, the word E scram is misspelled in the last sentence.
81 H 3 N Response: Spelling error corrected.
S NRC: The question is satisfactory with changes made.
Reference Provided - EAL pages: QC 2-9, 2-11, 2-13 NRC: The word following is misspelled in the stem. The distractor justifications do E not match the order of the provided answer choices.
82 H 3 N Response: Spelling error corrected. Distractor justifications and answer choices S aligned in correct order.
NRC: The question is satisfactory with changes made.
Reference Provided - QGA 200 Redacted NRC: The question pedigree indicates that QGA 200 (Primary Containment Control) is to be provided as a reference in its entirety. Since a considerable amount of this procedures content would be considered to be closed-reference knowledge (i.e.
entry criteria, overall mitigative strategies, etc.), it is not acceptable to provide this entire reference as a handout. When taken in conjunction with the current reference, U this question becomes largely a direct-lookup, with only minimal knowledge being required on the part of the applicant to answer it (refer to NUREG-1021 ES-401, 83 H 2 N section D.2.g). As a result, the level of difficulty of this question is unacceptably low.
Additionally, the distractor justifications do not match the order of the provided S answer choices.
Response: Provided reference is redacted with the information that would make the question direct look up or from memory removed. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
Reference Provided - QGA 400 section embedded in stem Quad Cities 2003 NRC Exam NRC: This question does not test at the SRO license level. Specifically, the stem includes an embedded reference that provides the criteria for transitioning to QGA 500-1. The only knowledge required on the part of the applicant is to evaluate which E of the answer choices constitutes a primary system that is discharging outside of both the primary and secondary containments, which is systems-level knowledge.
H 2 B NUREG-1021 ES-401, Attachment 2, section E, provides guidance that questions 84 that can be answered solely by knowing systems knowledge (i.e. flowpaths, S component locations, etc.) are RO-level questions. Due to the previous usage of this question on an NRC license examination, this question is being categorized as an enhancement rather than being unsatisfactory. Additionally, the distractor justifications do not match the order of the provided answer choices.
Response: Discussed with NRC, question marked as sat. Distractor justifications and answer choices aligned in correct order.
Page 17 of 22
ES-401 18 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 NRC: The NRC agrees with the licensee that systems knowledge in conjunction with procedural transition decision making is required to answer this question, therefore, it is at the SRO level. The question is satisfactory with changes made.
Reference Provided - SAMG-2 H2/O2 Drywell and Torus Sections Embedded NRC: The distractor justifications do not match the order of the provided answer E choices.
85 H 2 B S Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
H 2 B S Reference Provided - QGA 200, QGA 500-1 Redacted 86 NRC: None 87 H 3 N S NRC: None NRC: This question does not match the listed K/A; specifically, NUREG-1021, ES-401, section D.2.a discusses that when testing A2 K/A statements, testing both aspects of the K/A statement should be attempted. If that is not possible, then the scope of the question should be limited to the (b) portion of the A2 K/A statement since it requires the highest cognitive level. However, this question only tests the (a) portion of the applicable K/A statement. Additionally, this question does not test at the SRO license level. The only knowledge required on the part of the applicant to identify the correct answer is CS/ECCS systems knowledge. NUREG-1021 ES-401, Attachment 2, section E, provides guidance that questions that can be answered 88 H 2 N S solely by knowing systems knowledge (i.e. flowpaths, how the system works, etc.)
are RO-level questions.
Response: The US must determine the correct course of action based on plant conditions, as it is plausible to allow a cavitating ECCS pump to continue to operate, that determination can only be made by the SRO US. Discussion with NRC, Question marked Sat.
NRC: NRC agrees that K/A is addressed in question stem. Question is satisfactory as originally written.
Reference Provided - TS 3.4.3 NRC: Distractors B and C are not credible due to the actions and associated completion times described in each distractor not being plausible. An applicant can U use the provided reference (TS 3.4.3) to eliminate these distractors since the completion time (24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />) listed in Distractor B and the action (be in Mode 2) listed H 2 N in Distractor C do not exist in TS 3.4.3. Therefore, there appears to be no basis for 89 why an applicant would have the misconception that either of these distractors was correct. Additionally, the information contained in the last two sentences of the S distractor analysis for Distractor 1 does not align with the content of the distractor itself. Finally, the distractor justifications do not match the order of the provided answer choices.
Page 18 of 22
ES-401 19 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 Response: Question revised with answer choices that require knowledge of the bases for the Safety and Relief valve Tech specs. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: Distractor A is not plausible; there appears to be no basis for why an applicant would have the misconception that a Technical Specification (TS 3.8.2) which has the word Shutdown in its title would be applicable when the stem E indicates that the reactor remains at power.
90 H 3 N S Response: Revised the distractors for plausibility. Including ONLY emphasis to avoid subsets.
NRC: The question is satisfactory with changes made.
Quad Cities 2012 NRC Exam NRC: This question, as currently written, overlaps excessively with Simulator E Scenario #1, Events 3 & 4. Additionally, the distractor justifications do not match the order of the provided answer choices.
91 H 2 B Response: Question replaced with a bank question that does not overlap with the S Operating Exam and meets the selected KA statement directly.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
92 H 3 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
93 H 3 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Quad Cities 2004 NRC Exam NRC: Distractors A and C may potentially be considered as second correct U answers due to an applicant being able to make unstated assumptions that are not contradicted by stem. TS SR 3.4.9.1 requires verifying that RCS heat-up and F 2 B cooldown rates are 100°F in any 1-hour period Distractors A and C both 94 present rates of 100°F in a 1-hour period. Therefore, Distractors A and C both S appear to represent potential additional correct answers for this question. With no concept of operations procedure which dictates that a conservative cooldown rate of 80F/ hr be followed, the operators are to remain within license limits. In addition, this question does not test at the SRO license level. The only knowledge required on the Page 19 of 22
ES-401 20 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 part of the applicant is RPV cooldown rate limits, which is systems-level knowledge.
NUREG-1021 ES-401, Attachment 2, section E, provides guidance that questions that can be answered solely by knowing systems knowledge are RO-level questions.
Finally, the distractor justifications do not match the order of the provided answer choices.
Response: NRC Reselected KA and question replaced with a bank question that directly addresses the KA at the SRO level. PREVIOUS NRC EXAM.
NRC: The question is satisfactory with changes made.
Reference Provided - Core Map embedded in stem.
Quad Cities 2010 NRC Exam NRC: Update the question pedigree to reflect the reference/graphic that is embedded E
in the stem of this question. Additionally, the distractor justifications do not match the order of the provided answer choices.
95 H 2 B Response: Question pedigree updated to include reference provided. Distractor S
justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
96 H 2 N Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
Reference Provided - WC-AA-101, Pages 1-3 NRC: The distractor justifications do not match the order of the provided answer E choices.
97 H 3 N S Response: Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
NRC: The distractor justifications do not match the order of the provided answer E choices.
98 F 2 B Response: Distractor justifications and answer choices aligned in correct order.
S NRC: The question is satisfactory with changes made.
U Reference Provided - QCOA 0010-12, Attachment D H 2 N NRC: The level of difficulty of this question is unacceptably low. Specifically, this 99 question can be answered by direct look up in the reference provided. Additionally, S the distractor justifications do not match the order of the provided answer choices.
Page 20 of 22
ES-401 21 Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: MARCH 2020 Response: Question revised to address the direct look up aspect and a decision the SRO will have to make for direction of the fire actions. Distractor justifications and answer choices aligned in correct order.
NRC: The question is satisfactory with changes made.
Free Look Question 100 H 2 B S LaSalle 2008 NRC Exam NRC: None.
Page 21 of 22
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: QUAD CITIES NUCLEAR POWER STATION Exam Date: March 2020 B= 50 F= 33 E= 58 RO TOTALS: M= 0 H= 42 U= 6 Additional Notes: 8% of RO Questions assessed as unsatisfactory.
N= 25 (15 with H Cog)
B= 10 F= 3 E= 16 Additional Notes: 20% of SRO Questions assessed as SRO TOTALS: M= 0 H= 22 U= 5 unsatisfactory.
N= 15 (14 with H Cog)
GENERAL COMMENT
S:
The number of questions on the RO/SRO section containing provided reference materials exceeds the FAQ 401.53 guideline of ~5% (4 questions) on the RO exam and ~ 20% (5-6 questions) on the SRO exam. Chief Examiner assessed the number of RO/SRO questions with attached references as not adversely affecting the LOD of the exam or cueing other questions.
- 4. As a general comment, question pedigree information does not list the facility learning objectives that are associated with each question. NUREG-1021 ES-401, section D.2.g states [i]f the facility licensee has a learning objective applicable to the question, it should also be referenced.
Page 22 of 22