ML18044A173
| ML18044A173 | |
| Person / Time | |
|---|---|
| Site: | Byron  |
| Issue date: | 12/14/2017 |
| From: | Jesse Seymour, Bielby M NRC/RGN-III/DRS/OLB |
| To: | Exelon Generation Co |
| Shared Package | |
| ML17152A381 | List: |
| References | |
| Download: ML18044A173 (12) | |
Text
ES-401 Facility: Byron / Year: 2017 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 F 2 B S LORT Exam bank (1596633) question 2 H 2 N S 2X2 Question NRC: Justify why RCP restart criteria must be known from memory.
3 H 3 N E Response: Per OTM ROs are trained and expected to know RCP restart criteria, as well as trip criteria.
4 F 2 N S NRC: The 2nd part of the answer is not addressed in the explanations for either 5 H 3 N E distractors A or B. Impact? Plausibility?
Response: Provided plausible explanation of second part of answer.
2X2 Question 6 H 3 N E NRC: Saying the control power fuses are not yet installed implies that this condition is important to answering the question correctly (i.e. they are not but they should be).
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
$ Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
$ Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).
$ T/F: The answer choices are a collection of unrelated true/false statements.
$ Cred. Dist>: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
$ Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
$ Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
$ Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
$ #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
$ Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only. (K/A and license-level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 12
ES-401 Facility: Byron / Year: 2017 2 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Delete yet from the stem.
Response: Comment incorporated.
7 H 3 B S Braidwood 2014 RO NRC Exam Question NRC: No action required is not plausible - the valve has a status light indicating it is 8 F 2 N E in the wrong position.
Response: As asked, no action is a plausible choice.
9 F 2 B S Braidwood 2013 RO NRC Exam Question 2X2 Question NRC: None, all criteria met is not plausible - it is common to start pumps as a U typical RNO action, and since this is a 2X2 question, that eliminates 2 answers 10 H 2 X N S Response: The stem identifies various possible criteria for starting the AF pump. The applicant has to know the criteria. As asked, the question makes None, all criteria met a possible (ie, plausible) answer.
11 F 2 N S 2X2 Question NRC: No TS LCOs apply is not plausible - TS 3.8.7 applies to inverters, not Instrument Bus 112, and is an easy choice. No RO-level question analysis done.
12 F 2 X N E Response: No TS LCOs apply is plausible based on the way question is asked.
ROs are expected to know TS LCOs above the line per NRC analysis.
2X2 Question 13 H 3 N S 14 H 2 N S 15 F 3 N S 16 H 3 N S NRC: The transitions to SI Termination or Post-LOCA Cooldown & Depressurization are not plausible from 1BEP ES-1.3 (which is given in the modified question) - the transfer to Cold Leg Recirculation must be performed without delay!
U Response: Plausibility is described in the Answer Key. 1-BEP ES-1.3 directs 17 H 2 X M transition to 1BEP-1 which directs transition to 1BEP ES-1.1 or 1BEP ES-1.2; S
however, both are incorrect because the actions of 1BCA-1.1 will mitigate both of these events. The transition to 1 BEP ES-1.1 or 1BEP ES-1.2 (ie, Choices A and D),
from 1BEP ES-1.3 are correct transitions and; therefore, plausible distracters.
However, the transition to 1BCA-1.1 from 1BEP ES-1.3 provides the correct Page 2 of 12
ES-401 Facility: Byron / Year: 2017 3 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only mitigating action, and is the correct answer.
Bank Question - SOURCE?
NRC: Go to 1BEP-1 is not plausible with FR-H.1 starting, a high priority procedure.
18 F 2 X B E Response: This is BYR exam bank question. Plausibility provided with distractor explanation.
2X2 Question NRC: A rod inserting clearly causes delta flux to be more negative. In addition, an U operator would not simply realign the rod, so the correct answer is easily selected.
19 H 2 N Response: As described in Answer Key, rod insertion does not always cause delta S
flux to be more negative. A rod drop all the way into core would cause positive delta I due to Tave dropping. Restoring a rod to bank position is a plausible action. NOTE:
deleted to < 50% from C.2 and D.2.
2X2 Question 20 F 2 N S NRC: Should reword the stem to say Which of the following conditions require Emergency Boration per 1BOA PRI-2: to test the knowledge of the entry conditions 21 F 2 N E versus simply recognizing them.
Response: Reworded stem to remove entry 2012 NRC Exam Bank Question NRC: Suggest rewording distractor C to: VERIFY/BLOCK the Source Range Hi Flux 22 H 2 B E trip. The P-6 reference and action should be removed.
Response: Incorporated comment and re-ordered choice format based on length of statement.
2X2 Question NRC: The NI failing low is a given state of inoperability, so entering TS is a must.
U Since this is a 2X2 question, 2 distractors are thereby eliminated. Also, being told that 23 H 2 X X N the NI is failed does not show the ability to assess operability per the K/A.
S Response: Pictures of the indicators make this an operationally valid question. The applicant must determine the state of operability, and whether or not the TS applies, as well as the result (ie, loss of control power). Question addresses the KA.
2X2 Question H U 24 2 X N NRC: Choosing between loss of MW and condenser damage is obvious (production F S over safety?). While addressing condenser leakage may be tied to loss of MW, a steam dump interlock would not. The 2X2 format then reduces the question to what is Page 3 of 12
ES-401 Facility: Byron / Year: 2017 4 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only the C-9 setpoint.
Response: 1) The question addresses the KA for loss of steam dump capability for a loss of condenser vacuum. Choosing between loss of MW and condenser damage (ie, production over safety) is an editorial comment about the KA. 2) Added the word EARLIEST to question portion 1 to prevent a potential subset.
2X2 Question U NRC: It is not plausible that the NSO could NOT manually actuate the Plant Wide 25 F 3 X N Fire alarm also. The 2X2 format eliminates distractors A & C.
S Response: The distracters A.2 and B.2 describe a condition when the PWFA system is auto initiated (ie, not manually initiated by the NSO) which is a plausible condition.
26 H 3 N S 2X2 Question 27 H 3 N S 2X2 Question 28 F 3 N S NRC: While there are many reasons that seal injection flow might plausibly lower, in reality, an SI is a very common scenario situation where it would be known to rise.
U 29 F 2 N Response: No question comment provided (or understood?). As written the question S
is asking if and why seal flow increases, or decreases, based on given conditions which meets KA.
30 H 3 N S 2X2 Question NRC: No TS LCO is not plausible when the applicable LCOAR is given in the stem.
Also, if above the line information is to be known, do we need to define what 3.5.2 31 H 2 X N E and 3.5.3 is in the distractors?
Response: Plausibility provided with the distractor explanation. Sufficient explanation is provided for the applicant to answer the question 32 H 3 N S 2X2 Question 33 H 3 N S 34 H 2 B S 2013 Cert Exam Question 35 H 3 B S 2016 Cert Exam Question 36 F 2 B S ILT Exam Bank Question (412277) 37 F 2 N S Page 4 of 12
ES-401 Facility: Byron / Year: 2017 5 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 38 H 2 N S 39 F 3 B S Braidwood 2013 RO NRC Exam Question NRC: No procedure entry required is not plausible after a PZR Pressure Channel 40 H 2 X N E fails LOW. An AOP entry, at minimum, would be required.
Response: A plausible procedure entry was provided for distractor A.2.
41 F 2 N S 42 F 2 N S 2X2 Question NRC: The incorrect distractor explanation (A) does not explain why B is correct.
43 H 3 N E Response: B. explanation is explained in the general paragraph above the choices in the key.
2X2 Question 44 H 3 B S Braidwood 2014 RO NRC Exam Question 45 H 3 N S 46 F 2 N S 2X2 Question NRC: The 2nd part of the question could be considered to be a cue that isolating the S/G is required. The question should be changed so that it can be answered entirely 47 H N E and the completed answer is aligned. For example, parts 1 and 2 could be combined for each distractor with the stem asking Which of the following is correct Response: Stem and distractors rewritten to remove potential cue.
2X2 Question 48 H 2 B S ILT Exam Bank Question (419336)
NRC: Q#48 eliminates A as a plausible distractor, since it was a 6.9KV bus that 49 H 3 X N E caused RCP 1D to trip.
Response: Rewrote to remove potential cue to question from Q48.
50 H 3 N S 2X2 Question U
51 F 2 X M Modified Bank Question 2016 CERT EXAM Q59 (1144551)
S NRC: It is not plausible that if starting air was 0 psig (normally 250 psig) that the Page 5 of 12
ES-401 Facility: Byron / Year: 2017 6 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only engine would be able to be rolled. Due to the 2X2 question format - UNSAT.
Response: Based on previous use of this question, and validation results, there is a potential misconception that the extra volume of air used for the pneumatic protection system, would be sufficient to start the EDG. As a result, the distractors are plausible.
2013 Byron NRC Exam Bank 2X2 Question 52 F 2 X B E NRC: It is not plausible that a release should be continued with its radiation monitor failed. Considered an Enhancement due to being a previous NRC Exam question Response: Reworded distractors to address comments.
53 F 3 M S Modified LORT Bank question (1512249)
NRC: The question could be enhanced by having an incorrect local action replace one of the incorrect automatic actions. All distractors can currently be eliminated by 54 F 2 N E simply knowing that there are very few automatic IA/SA responses.
Response: Replaced B with local action.
NRC: If the bus naming convention is that 1X1 applies to Train A and 1X2 applies to Train B, and X is related to bus voltage, then this question is too easy.
U 55 F 2 N Response: The question meets the KA. Knowing the naming convention is part of S
knowing the power supply, ie, ESF bus designation, component and bus designation for voltage (ie, 480 or 4160), train A or B.
56 F 2 N S 2X2 Question 57 H 3 N S 2X2 Question 2X2 Question NRC: Saying that P-6 remains LIT cues the applicant that something is not normal, 58 H 2 X N E which is the main point of this question. Change remains to is or provide the panel indications in a photo, which is all the information provided in this situation.
Response: Replaced remains to is in stem, and changed setpoint to 3.2E-5%.
2X2 Question H NRC: This is F question - it involves recognizing what is being displayed and basic 59 2 N E F familiarity with how to find thermocouple locations; not analysis.
Response: Although operationally valid question, changed H to F.
U NRC: There may be some facts to support plausibility of the smaller values given as 60 F 2 N distractors, and some may even confuse 23 feet with 24 feet, but this question, as S asked, is too easy.
Page 6 of 12
ES-401 Facility: Byron / Year: 2017 7 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Response: The distractor numbers are significant numbers and believable, question is safety significant and meets KA.
2X2 Question NRC: 1) Distractors B and C dont make logical sense; If the transfer to AUTO worked, why must the NSO go to MANUAL? If not, why leave it in AUTO? 2) Also, if 61 H 2 X X N U the readings did support leaving 1FW510 in auto, would that be required? If not, change must to something else.
Response: Replaced Question.
2X2 Question NRC: For distractors B and D to be plausible, one must assume MTC is negative, 62 H 3 N E which cannot be determined from the information provided.
Response: Added statement to stem that MTC is negative.
NRC: Distractor D provides a clue that P-8 might be an issue to consider, a key 63 H 2 X N E factor for answering this question correctly. Need to replace it.
Response: Replaced distractor D.
Moved if any from the end of question to the middle to maintain consistency of 64 H 2 N S placement within statement throughout exam.
65 F 3 N S NRC: Other listed tasks might be plausible, but this answer could not be called 66 F 1 N U incorrect.
Response: Replaced Question NRC: Is this a new question? It is not specified.
67 F 2 N E Response: Question is New.
Placed the AND statement from end of question portion 1 to a separate line to 68 F 2 N S maintain consistency of placement within statement throughout exam.
69 H 2 N S Drawing M-52 Sheet 1 is to be provided for this question.
NRC: The Shift Manager would be more plausible than the Station Surveillance 70 F 2 N E Coordinator. Synchoniztion is misspelled in Explanation.
Response: SM might be considered a second correct answer. Leave as is.
NRC: Saying applicable limits versus Exelon Administrative Dose Limits would be 71 H 2 N E more discriminating, since the applicants should consider them to be applicable.
Response: There are many applicable limits, want to discriminate that question is Page 7 of 12
ES-401 Facility: Byron / Year: 2017 8 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only asking for Exelon Administrative Dose Limits.
72 F 3 B S 2013 Byron NRC Exam Bank Modified 2013 CERT Exam Bank question #73 (1146450) 73 H 2 M S 1BOA PRI-10 Table A is to be provided for this question.
NRC: This question is LOD=1. An SI (ALERT - FA1) obviously must be reported via NARS within 15 minutes from its declaration. How often has this been practiced in the simulator and in the plant during drills? No analysis needed.
H 1 U 74 N Response: Applicants are novice operators at this stage of training and question F 2 S being tested meets the KA criteria. Analysis is required to answer the question, and information is not tested in the simulator or during drills in a regular time frame because applicants are in training and not LORT.
75 H 2 N S 76 H 2 N S 2X2 Question 77 H 2 N S 2X2 Question 2X2 Question 78 H 2 N S Changed will to must in question portion 1 to match actions.
NRC: This question requires recall of an extremely detailed EOP RNO that is not required to be performed from memory. Alternatively, the only analysis that could possibly be performed would be recognition of the associated noun name for the H U 79 2 X N N proper RNO valve, which would then be system knowledge.
F S Response: SRO applicants are expected to know systems and procedure mitigation strategy. The question is not asking for detailed actions, but rather big picture mitigating strategy actions.
2X2 Question NRC: Entry into ES-0.1 is implausible since its use requires assumptions for U conditions which are not provided in the stem. Due to the 2X2 format, 2 distractors 80 H 3 X N are affected.
S Response: Although incorrect, the plausibility of C. and D. are explained and justified.
The reviewer provided no suggestions of why they were not plausible or how they could be more plausible, yet incorrect.
1BOSR 8.9.1-1 is provided as a reference for this question.
81 H 3 N E NRC: Do the SROs have to know the actions independent of the applicable TS?
Response: Yes.
Page 8 of 12
ES-401 Facility: Byron / Year: 2017 9 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only BCB-1 Figure 36 is provided as a reference for this question.
NRC: Selecting the BCB figure from among the few references available for the U SRO exam is a trivial task. If it was simply inserted in the question stem, this task 82 H 2 N N could be expected of ROs.
S Response: This is an SRO task and not performed by ROs. The applicant needs to select the correct curve and correctly apply the information by plotting the number on a log scale.
2X2 Question EP-AA-1002 Addendum 3 is provided as a reference for this question.
83 H 2 N E NRC: Providing all 51 pages of this procedure seems excessive, and creates the possibility that information could be unintentionally provided for other questions.
Response: Providing EP-AA-1002 Addendum 3, Sheet BY 2-1. Page checked to verify no answers to other questions are provided by the handout.
NRC: This question requires recall of a detailed procedure that is not required to be performed from memory. The correct answer would likely be recognized only by U remembering those controls that can be activated from the Fire Panel, which would H
84 3 X N S make this an F question in that case. Finally, what is OAS?
F Response: This question requires memory and knowledge of panels and annunciators, comprehension and analysis of situation, and use of procedures. SRO applicants are expected to have the knowledge to direct these actions.
2X2 Question NRC: With PZR level off-scale low, it would not be plausible to have it stabilize, U eliminating 2 distractors. Also, why would the applicant be required to perform this 85 H 3 X X N procedure step from memory?
S Response: Stabilize means stable increasing or decreasing trend. Applicant using analysis and comprehension, and knowledge of mitigating strategy, to answer the question.
86 F 2 N S 2X2 Question 87 H 3 N S TS 3.6.6 is provided as a reference for this question.
88 H 3 N S 89 H 2 M S U 2X2 Question 90 F 3 X N NRC: It is not plausible to believe that lowering activity would be chosen over S
lowering temperature when starting the 1D Reactor Containment Fan COOLER, Page 9 of 12
ES-401 Facility: Byron / Year: 2017 10 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only eliminating 2 distractors.
Response: Plausibility and justification of A.2 and C.2 was provided with the question submission. The system can be used with Containment Spray to reduce activity during certain situations.
NRC: In spite of the BAR reference, it is not plausible to choose BOA PRI-1 based in information in the stem, thereby eliminating 2 distractors.
U 91 H 2 N Response: This is operationally valid SRO question as written. The SRO must use S
comprehension and analysis of systems and given parameters to deduce use of BAR and subsequent procedures to direct actions.
92 H 2 N S 93 H 3 N S H
94 3 N S F
95 H 2 B S Bank Question from BWD 2014 NRC Exam Q 94 (ID: SG1027-N14-94) 96 F 2 N S 97 F 2 N S 2X2 Question NRC: Verifying 1PR011, Containment Atmosphere Rad Monitor, is operable, is not a U plausible TS action, thereby eliminating 2 distractors.
98 H 2 N S Response: Plausibility of verifying 1PRO11 operable is explained in distractor justification. The Reviewer did not offer a better distractor or identify why it was implausible.
2X2 Question Question 92 of the 2013 Byron NRC exam -MODIFIED! SEE ATTACHED U NRC: Beginning Reportability Manual evaluation, is not a plausible action right after 99 F 2 M fire occurs, thereby eliminating 2 distractors. This was not a direct NRC exam S
question Response: Plausibility was justified with question submittal. Reviewer did not identify why it was not plausible.
MODIFIED - ILT Exam Bank (1144505)
U 100 H 2 M EP-AA-1002 Addendum 3 is provided as a reference for this question.
S NRC: UE and Alert are implausible for this situation, especially when the Page 10 of 12
ES-401 Facility: Byron / Year: 2017 11 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only applicants can very rapidly eliminate them with the provided references. Also, Providing all 51 pages of this procedure seems excessive, and creates the possibility that information could be unintentionally provided for other questions.
Response: UE and Alert are part of what is being tested. Justification of plausibility was provided with each classification. All procedure pages were reviewed to ensure no answers were provided for other questions.
Page 11 of 12
ES-401 Facility: Byron / Year: 2017 12 Form ES-401-9 B= 13 F= 35 E= 22 RO TOTALS: M= 4 H= 40 U= 2 Additional Notes:
N= 58 B= 1 F= 8 E= 2 SRO TOTALS: M= 3 H= 17 U= 0 Additional Notes:
N= 21
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
- 2. Questions with references/attachments provided are identified in the COMMENTS.
Page 12 of 12