ML17137A187
| ML17137A187 | |
| Person / Time | |
|---|---|
| Site: | Waterford  |
| Issue date: | 04/05/2017 |
| From: | Vincent Gaddy Operations Branch IV |
| To: | Entergy Operations |
| References | |
| Download: ML17137A187 (9) | |
Text
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9
- 1.
LOK
- 2. LOD 6
7 Ans Letter CFR 8
Q#
(F/H)
(1-5) stem focus cueing T/F cred dist partial job link minutia
- /
units back-ward K/A SRO-only B/M/N U/E/S A/B/C/
D 55.41x55
.43y Explanation 1
H 3
X X
N E / S C
41.7
- State, "Upon the trip, the auto transfer to the Startup" Change made - Question is now sat.
Change has been incorporated. Complete with review. Question is now SAT.
2 H
3 X
N E / S A
41.3
- Need to revise the question stem to separate the spurious opening of the relief from the current conditions. As it stands, the statement regarding the spurious opening of the relief could be interpreted as current conditions - this would confuse the applicant when asked the status of the relief valve. Need rx trip in the stem since went below the sp. Stem needs work. Added Rx tripped to the stem and reversed the order of question 1 and 2 to prevent confusion between past and current conditions. Seperated event conditions and current conditions as recommended.
Complete with review. Question is now SAT.
3 H
4 X
X N
U / S A
41.5
- Cueing: The statement that "the crew will verify all available charging pumps running" is in reference to the correct answer, i.e. the EOP direction during a LOCA. There is no reason why an operator would "verify all available charging pumps running" for emergency boration. This statement acts as a cue. Just state, "For this condition, the crew will start Charging Pump(s)_____." Partial: It does not appear to be 100% incorrect to have started all three charging pumps in this condition. The reference provided states that it "is not necessary to start the standby pump" in response to the SIAS signal. That does not mean that if you do start the standby pump, that it would be wrong. Therefore, there are 2 partially correct answers. Removed the portion of the stem that stated "all available charging pumps running" and replaced it with "To meet the minimum required number of charging pumps running". This change addresses the comments. Complete with review. Question is now SAT.
4 H
2 X
N E / S B
41.8
- Distractors C & D do not appear credible. Why is it plausible to consider that one train of ACCW might be unavailable? Replaced ACCW with Essential Chillers in part 1 of the question. More plausible because the applicant must recognize that ACCW is still available as a backup to the Essential Chiller on the isolated side. Complete with review. Question is now SAT.
5 H
2 X
B S
C 41.3
- 2009 NRC Exam. LOD-2. Changed to a LOD-2. Question is SAT.
6 H
3 X
N E / S D
41.7
- Partial: Asking a question in a way that states, "the crew will monitor___" leaves the question open to appeal. There is no way to be certain what a crew will monitor. Depending on the operators, a crew may monitor a number of letdown parameters, LDHX outlet temp included.
Need to state something to the effect: "If uncorrected, letdown will isolate on ______ heat exchanger outlet temperature." Whatever change is made needs to ensure that the correct answer is 100% correct and the distractors are definitely wrong. LOD: Question is LOD-3, not 4.
Changed part 2 of the stem to read, If uncorrected, letdown will isolate on __(2)__ heat exchanger outlet temperature. This addresses the comment above. Changed LOD from 4 to 3.
Complete with review. Question is now SAT.
7 H
3 X
N U / S C
41.10 Question does not test knowledge of the RHR system purpose - it test knowledge of procedure content. KA mismatch-doesn't address system purpose or function. If you asked what cooling loads were lost when you lost SDC (ie RHR) then that would meet knowledge of system purpose or fn. This might be a good SRO Q. Changed the K/A and wrote a new question. Could not write a question on the purposed of the offnormal procedure. Question is now SAT.
8 H
2 N
S C
41.7
- LOD-2. Changed to a LOD-2. Question is SAT.
9 F
2 N
S D
41.7 10 H
3 X
N U / S B
41.10 Question tests prevention of MSSV cycling, not prevention of ADV/SG PORV cycling. KA mismatch and stem focus are wrong. This KA is knowledge of reasons for preventing the use of secondary PORV (or MSSV for that matter, if possible). The question is asking if you use 1 or 2 ADV to do the coolodown which does ot meet the KA. Kept the K/A and changed the question to match the K/A. Question is now SAT.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 11 F
2 B
S A
41.10 2011 NRC Exam.
12 H
3 X
N E / S B
41.10
- Stopping RCPs reduces the heat input into the RCS, which in part does help prevent the lifting of PZR safeties by limiting the heatup. Therefore, need to state, "Per EOP XX basis, this action is intended to____." It is not acceptable to have one answer that is more correct than the other.
The correct answer has to be completely correct and the distractor must be wrong in a concrete way. Added Per the Tech Guide for OP-902-006. Complete with review. Question is now SAT.
13 F
2 B
S B
41.10 2011 NRC Exam 14 F
2 N
E / S D
41.8
- To prevent potential appeals - need to add "Per EOP XX basis, this action is required to ___."
Added Per the Tech Guide for OP-902-003, Loss of Offsite Power/Loss of Forced Circulation Recovery Complete with review. Question is now SAT.
15 F
2 X
B S
D 41.8 2007 NRC makeup exam.
16 H
3 B
S B
41.7 2010 NRC Exam. LOD-3, not LOD-4. Changed to a LOD-3. Question is SAT.
17 H
3 X
N E / S A
41.10
- It is best to not ask the question by stating the "the crew will ___." There is no way of verifying what the crew will do - they could always justify starting a shutdown if they feel it is necessary.
Ask, "When IA pressure lowers to 95 psig, _____," with answers being: "plant shutdown is required" or "the IA-123 bypass opens." Added the suggested changes to the stem and the distractors. Complete with review. Question is now SAT.
18 H
4 N
S A
41.5 19 H
2 B
S A
41.10
- 2010 NRC Exam. LOD-2, not 3. Changed to a LOD-2. Question is SAT.
20 H
3 X
B E / S B
41.6
- 2012 NRC exam. LOD-3, not 4. Stem editUsing the emergency boration procedure, which boration path would be used for these conditions? Added the AB bus is aligned to the A train to the stem. This would mean that CVC-507 has no power and cannot be used for Emergency Boration. Changed to a LOD-3 from a LOD-4. Complete with review. Question is now SAT.
21 H
3 X
N U / S B
41.7
- Distractor D is not credible. It is not plausible that a failed letdown controller output would result in normal letdown and charging flows. Is there a reason why we cannot use a RxReg Loop 2 Th instrument as Distractor D? C is not credible. The controlling channel, if failed, would cause a level change for PZR level. Replaced distractor D with the Loop 2 Th instrument as suggested.
Replaced "letdown flow is 40 gpm and steady" with "Pressurizer level is steady". The applicant must know that letdown flow is still at 40 gpm because only one charging pump is running to keep Pressurizer level steady. Distractor C is plausible because the applicant is required to know that the annunciators can come in at a normal pressurizer level because the level switches that bring in the annunciator tap in upstream of the level control channel selector switch. Complete with review. Question is now SAT.
22 H
2 X
N E / S C
41.10
- Add "Per Tech Guide XX, the BOP will close" to avoid any potential appeals. Cred Dist:
Distractors A & B appear to implausible - it would seem to be a fairly drastic measure to close the MSIVs in order to stop the feed pumps - wouldn't it just be easier to stop the pumps? The obvious choice to protect the condenser becomes even more obvious in the face of distractors A
& B. Why does the AOP have no guidance for shutting MSIVs on lowering vacuum? OP-901-220.Once the MSIVs are closed isn't it is a moot point that the dumps don't work now? A different angle on this question is when the dumps go closed and why during a lowering vacuum event?
Changed the order of Part 1 and Part 2 of the stem so the question will flow better. Changed distractors A and B in part 2 to a more plausible action prevent excessive cooldown". Did not add per the TG because the step for closing the MSIVs is located under the step for protecting the Main Condenser. Complete with review. Question is now SAT.
23 F
2 B
S A
41.10
- 2014 NRC exam.
24 F
2 X
N U / S D
41.13
- Cueing: The existence of CP-4 in the question stem cues the applicant to selecting CP-4 in the answer. LOD-1: It is elementary knowledge that the system auto-isolates on high rad. Could test which valve closes to isolate - GWM-309 or 311 or 104/105. Removed The ATC will verify from the stem because it could be considered cueing. Changed part 2 of the distractors to test if the applicant knows which Radiation Monitor will isolate the GDT discharge. Removed the annunciator from the stem. Complete with review. Question is now SAT.
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 25 F
2 X
N E / S B
41.10
- Stem Focus: Need to add to the stem that there is an ADV that is opening spuriously - otherwise the answer choices are confusing. Cueing with first part of distracter A and answer B and fire in the stem. To fix, place MS-116A ADV spuriously openes in the stem. Placed the initial condition that MS-116A has spuriously opened in the stem and adjusted the distractors accordingly.
Complete with review. Question is now SAT.
26 F
2 N
S C
41.10 27 H
3 B
S A
41.10
- 2008 NRC exam. LOD-3, not 4. Changed to a LOD-3. Question is SAT.
28 F
2 N
S C
41.10 29 H
2 N
E / S A
41.6 2011 NRC exam. Question source is stated as modified, however no parent question is attached.
Need to attach parent question to verify sufficient changes have been made. LOD-2, not 3. As modified, this is a borderline LOD-1 question and should be looked at for replacement in order to reduce the overreliance on the past 5 NRC exams. Changed the question entirely but kept the same K/A. The question developed is a "new" question. Question is now SAT.
30 H
3 B
S D
41.7
- 2015 NRC Exam. LOD-3, not 4. This is not an H4 question. More like H2. Changed to a LOD-3.
Question is SAT.
31 H
4 B
S B
41.10
- 2014 NRC Exam.
32 F
2 X
N E / S D
41.7
- 2008 NRC Exam. K/A mismatch. This question tests actuation signal knowledge, not knowledge of the many interlocks associated with SI valves, i.e. suction valve interlocks, SIT valve interlocks etc. This is marked as E (not a U) because it was missed on the 2008 review however it does not meet the KA for an interlock. An interlock occurs between two items, for example before valve x can be opened, the other valve in the interlock must be closed, or before valve y can be opened, the pressure must be below 400 psig. This question is written about a design feature but the specific part of the KA is an SI valve interlock. Changed the question to ask the interlocks associated with SI-106A on an RAS. SI-106A is interlocked with another valve, SI-602A. Changed the stem to get rid of the word "interlock" because that was considered cueing. Complete with review. New question has subset issue - add immediately to the end of A&C and question will be SAT. Changes made, question is now SAT.
33 H
2 X
N U / S A
41.3
- Is stopping a HPSI pump the first step you train operators to take in order to throttle HPSI? If so, the distractors are acceptable. If they are trained to throttle injection first, then the distractors need to be changed to reflect this. LOD-2, not 4. KA mismatch-you are giving them the item they are supposed to predict (overpressure on the PZR) in the stem and cueing to it in the second part of the stem by asking the safety sp.Also your distracters C and D are securing the pump and throttling-affects credibility of these two distractors. Removed wording in the stem take actions to control Pressurizer level and prevent challenging Pressurizer safeties by. Also changed the distractors C and D to secure one HPSI pump at a time from secure HPSI pumps. Review complete. Question is now SAT.
34 F
2 B
S D
41.7
- 2012 NRC Exam.
35 H
3 N
S A
41.7
- LOD-3, not 4 Changed to a LOD-3. Question is SAT.
36 H
4 N
S B
41.7 37 F
3 X
N U / S C
41.6 Distractors B & D are not credible. It is basic knowledge that a shutdown requires the addition of boron to the RCS. The plausibility statement that the distractors would be correct during a startup is not a sufficient justification when the question stem clearly states that a shutdown is to be performed. Please provide a more challenging question. Changed the question based on feedback. The question was made more challenging by requiring the applcant to determine when the heaters will cutout on high pressure. Need to add units after the blanks then will be sat.
Change made, question is now SAT.
38 H
2 N
S D
41.6
- LOD-2, not 3 Changed to a LOD-2. Question is SAT.
39 F
2 X
N E / S B
41.7
- 120v instrument AC is not a plausible distractor for power to a steam supply valve. These power supply questions are better left for larger loads in the ESFAS system. If you want to keep this question, can you add B-DC as a distractor (sometimes the valves in EFW/AFW systems are cross-powered) and come up with something else for the fourth distractor. All the secondary items are off of AB-DC so where is the disc validity on this one? As recommeded, changed distractors C and D to a 480V MCC power supply instead of a vital SUPS power supply for better plausibility. Complete with review. Question is now SAT.
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 40 H
2 X
X N
U / S D
41.10
- The second half of this question does not meet the KA and tests a very basic concept that will have been practiced repeatedly in the simulator. There is no use of procedures to mitigate consequences. There are more challenging questions that can be written regarding asyemetric SG pressure during an ESD, i.e. when should EFW be manually initiated. C is also correct based on your explanation of it. Changed SG2 pressure from 475 psia to 505 psia to make it closer to the 123 psid setpoint. Removed distractors B and D and replaced it with a distractor that was not also correct. Part 2 is mitigating Part 1 because both SG pressures are still lowering after the MSIS. The BOP needs to ensure MSIS actuation to verify all valves went closed. Change part 2 of distractor. The last step for isolating a SG is to check MSSVs closed for that generator. The location is important for the RO to know because he must direct the AO to look at the correct side or he/she may field a call stating there is a steam release on the west side and the applicant must know which S/G that is. Reviewed and good to go. Question is now SAT.
41 F
2 X
N E / S C
41.10 brackets are std for harsh containment everywhere, ref leg boiling (ie above 212F) causes the issue, so anything below that is not really credible either. Changed 220F to 180F to make it more plausible (distractors in part 2). Question is now SAT.
42 F
2 N
S B
41.4 43 H
3 B
S C
41.14
- 2008 NRC Exam.
44 H
4 B
S A
41.4
- 2011 NRC Exam.
45 H
3 N
S B
41.10
- LOD-3, not 4. Changed to a LOD-3. Question is SAT.
46 F
2 B
S C
41.4
- 2012 NRC Exam.
47 H
4 B
S C
41.8
- 2012 NRC Exam.
48 H
3 B
S A
41.8
- 2010 NRC Exam.
49 F
2 X
N E / S C
41.10 This should be a system knowledge question, not a procedure knowledge question. Question does not match the KA - 1/2 of the question tests procedure knowledge. What annuciators, indications, operational impacts would be associated with a low battery voltage? The stem states that the BOP has determined that neither DC bus is energized - how he/she determined this is the intent of the KA. This could be more to the KA with an alarm in during a float condition, verify voltage is valid on what panel, low bat voltage is in alarm on CP-35 (cont cabinet C) etc. The procedure selection aspect is not really part of the KA here and not needed to create the question. Kept the K/A and changed the question to address issues discussed above. Question is now SAT and tests at fundamental level.
50 H
3 B
S D
41.7
- 2010 NRC exam.
51 H
4 B
S D
41.5 2012 NRC exam. Reference in explanation does not match reference used to support answer.
Agree, the wrong reference was listed in the explanation. Send the correct reference whch is OP-001-001 page 25. Changed the explanation to indicate the correct reference. Question is SAT.
52 F
2 X
N E / S D
41.11
- 2007 RO makeup exam. Why are A and C credible distracters? There are no indications of using boric acid tanks or anything related to them in the stem of the question. This needs to be fixed. Created a new question with the same K/A. Reviewed and good to go. Question is now SAT.
53 F
3 X
N E / S C
41.7
- Remove the "To prevent void formation during the transition to ACCW" from the stem - it cues the answer to the first part of the question. Why would void formation be a concern if the ACCW valves opened prior to the CCW valves closing? Just test what the system does - it is up to the applicant to know why. Prefacing the question stem with the reason directs applicants to the answer. This question is an F-3, not an H-4. Removed "To prevent void formation" from the stem. Complete. Question is now SAT.
54 H
4 N
S C
41.10 55 H
4 B
E / S D
41.9
- 2012 NRC exam. Question source is stated as modified, however no parent question is attached. Need to attach parent question to verify sufficient changes have been made. After further review, determined that this question is a bank question, not modified. Question redesignated as bank and is SAT.
56 H
2 N
S A
41.6
- LOD-2, not 3. This is basic ASI knowledge which will have been practiced many times in the plant and on the simulator. Changed to a LOD-2. Not basic knowledge because the applicant must know that a downpower will cause ASI to move to the top of the core. The top of the core at W3 is a negative ASI which is confusing to some reactor operators. Added this explanation to the explanation in the worksheet. Review complete. Question is SAT.
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 57 F
2 X
N U / S C
41.8
- This question tests a procedural requirement when the KA is asking for system knowledge. How do these seal cooler isolation valves fail on loss of air/power? Do they close on CIS? Can they be overriden? How can the system be restored prior to a trip being required... Changed part 2 of the question by asking if the seal cooler isolations can or cannot by overridden on high CCW return temperature. The references already sent encompasses the information for this change. Review complete. Question is now SAT.
58 F
3 N
S D
41.6 59 H
2 X
N U / S C
41.2 There are too many questions regarding which panel something is on. This type of knowledge is best tested on scenarios and JPMs. An occassional question on instrument location is reasonable, but this approach appears to be the go-to method for "monitor" KAs on this exam.
The first half of the question is simple, but acceptable - it is basic knowledge that CETs are used when no RCPs are running. Second, this question does not quite meet the KA, as it is looking for how to determine RCP operation during ICC conditions, i.e. RCP trip strategies and requirements.
Here the RCPs are already tripped on a LOOP. Can we put the applicants in a somewhat challenging spot on Attachment 2A-D, i.e. in a situation where only 2 RCPs should be running (1 in each loop) between the 20 deg subcool and the 0 deg subcool lines and have them calculate subcooling based off of pressurizer pressure and temp? LOD-2, not 3 as written. KA mismatch.
There is nothing about indequate core cooling in the question. Also, the five highest CETs you have in the stem would be a good focus for at least part of the answer for this KA. Kept the K/A and changed the question. Question is now SAT.
60 F
3 N
E / S B
41.7
- The crew could makeup to the SFP from either source. Therefore, they could choose to use only the condensate storage pool. The question needs to be asked - which of the following can be used to makeup to the SFP, not what the crew will decide. Made the suggested correction.
Review complete.Question is now SAT.
61 H
3 B
S B
41.7
- 2012 NRC exam.
62 H
2 N
S A
41.7
- LOD-2, not 3. Changed to a LOD-2. Question is SAT.
63 F
2 N
S C
41.11 64 F
2 N
S A
41.12 65 F
3 N
S B
41.4 66 F
2 N
S B
41.10 67 F
2 X
B E / S D
41.4
- 2010 NRC exam. KA mismatch for this exam. There is no use of the plant computer to evaluate system or component status. KA mismatch-marked as E because previous bank question.Failure of the plant computer is not using it to verify the status of another system, which is the KA.
Created a new question to meet the K/A. Review complete. Question is now SAT.
68 F
2 B
S A
41.6
- 2012 NRC exam.
69 F
2 X
N E / S A
41.10 2012 NRC exam. Question source is stated as modified, however no parent question is attached.
Need to attach parent question to verify sufficient changes have been made. The Work Management Center is not a plausible distractor for a procedure. The answer is cued as the OPAG is the only answer that has "procedure" in it. Need to replace the second half of the question with something more significant about the procedure change process. When can a field change be performed? Who approves a pen & ink change? These are more applicable to the essential aspects of the RO's job. Peer evaluator is not credible with words in the stem. These cue to a manager position. Replace this with Shift manager and put RO in the stem so that CRS is still the immediate supervisor. Could not develop a question with credible distractors for the process of changing procedures. Replaced the K/A and developed a new question. Question is 70 F
2 B
E / S C
41.5 2007 RO makeup exam. LOD-1. Mode 1 and 2 are not plausible with all rods in. Please write a more challenging mode definition question. Removed Mode 1 and replaced it with Mode 5. Mode 5 is more plausible. Left Mode 2 because it is more plausible than inserting Mode 6 into the question. Question is now SAT.
71 F
3 N
S B
41.10 72 F
2 B
S D
41.12
- 2011 NRC exam.
73 F
2 B
S A
41.11
- 2014 NRC exam.
74 F
2 N
S C
41.10 75 F
2 N
S D
41.10
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 76 H
3 X
N U / S C
43.5
- The question stems are confusing. The first half of the question has the applicant determine that pressurizer pressure must be reduced. However, the second half of the question, which seems to be written to correspond with the same stem conditions, suggests that the pressurizer heater breakers will be reenergized in order to stay within the PT limits. Why would heaters be required in order to reduce pressurizer pressure? Step 30e of the procedure is applicable (not step 30b),
and no where in the contingency actions for step 30e is there a need to start heaters. Can a more applicable HPSI throttling during a LOCA question be used? stem is flawed. In 30a it requries recovering PZR heaters but 30e where it states check that pressure is less than the PTS curve you lower or stop the cooldown and lower pressure, which is opposite of your explanation.
Changed part 2 of the question to state "After the RCS is restored to within the limits of the PT curve, the crew will maintain..." to ensure Part 1 and Part 2 is two distinct questions. This is the change agreed upon after discussion with the examiner. The question was changed slightly after validation. Added "after LOCA is isolated" and "to minimize Pressurized thermal shock concerns" to the stem. Adjustment made to change "to minimize PTS concerns" to instead reference the 200 degree subcooling curve. Validation operators were unsure of which one of the curves was being discussed in the question stem. Question is now SAT.
77 H
2 N
E / S D
43.5 Need to add "only" to the end of distractor B and make the first half look like distractor A. LOD-2, not 3. Added only to distractor B. Changed to LOD-2. Question is now SAT.
78 H
3 X
X N
U / S C
43.5 There are no indications in the stem that provide a plausible reason to believe there might be a second event ongoing. There is no plausible reason to believe that one should stabilize RCS temp and pressure when Tcold is still dropping rapidly and the generators are clearly not yet at dryout. These are basic concepts that do not test at the SRO level. Question is LOD-1 with implausible distractors. Half of the question is written at the RO level - entry conditions for the EOPs is considered RO level knowledge. Writing half of the question at the RO level should be minimized. The concept of testing this unique leak location is a good idea, but the distractors need improvement - they can be easily eliminated and the applicant can arrive at the correct answer without diagnosing the leak location (purpose of the KA). Not SRO-only-what you are asking in this stem is can the applicant recognize the EFW common line break, then everything else falls out on the question, which is RO knowledge. Added NAO reports that steam issuing from the RCA so that it can be determined that the steam leak is not occurring upstream of the MSIVs and can be isolated. Added words tot the explanation to clarify the question and added OP-100-017as a reference. This procedure provides the guidance that the crew can stay in the ORP if the SFSC is not being met if immediate actions will restore the Safety Function. Question is now SAT 79 F
2 X
X N
E / S D
43.5 The second half of the question cues the answer to the first half. There is no reason why one would select that ELAP is not required when the second half of the question says that an ELAP declaration is required within a certain timeframe. Question is written at the RO level - ROs must know EOP entry requirements and the major strategies within the EOPs. EOP basis document information does not necessarily make for an SRO question. Question essentially asks, is an ELAP required or not (which is cued), and how soon. This is RO level knowledge. Additionally, procedure selection at the entry to major EOPs does not constitute an SRO question. Place the SRO in a decision point during the implementation of the SBO EOP and then have him/her choose between two directions with associated attachments/appendices. Cueing from stem to answer for first part of the choices. I don't think this is SRO-only knowledge either. Re-wrote the question such that any reference to the Extended Loss of AC Power (ELAP) has been removed from the stem. To answer the question, the applicant must know more than just entry condiitions into an EOP. The applicant must also know when deep load shedding is required. Add title of procedure. Added the title of the procedure "FIG-001" to part 1. Question is now SAT.
80 H
3 X
N E / S A
43.5 Need to build the statement "direct the crew to vent the Main Generator" into choices A & B and then pick another action to add to distractors C & D. As written, the above statement cues the applicant to the Generator Aux Ops procedure. For example: Enter Att 33-B and vent the generator, or Enter Section E4 and secure the DC seal oil pump. On a side-note: there appears to be a typo in the first note at the beginning of E4 - "POWER SUPPLIES FOR AC LUBE OIL PUMPS ARE:" with no follow-on information. Seems like all RO kowledge and cueing from stem to answer. Removed Vent the Main Generator IAW in the stem and added it to the distractors.
Added which step would be performed first in the stem and added secure the DC pump to distractors and D. Question is now SAT.
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 81 H
3 X
M E / S C
43.2 2012 NRC exam. The question analysis shows that the parent question was used on the 2012 RO exam. In order to determine if the question has been sufficiently modified, please submit the parent question. The question examines AOP entry conditions (RO level) and TS entry above-the-line knowledge (RO level). Does not appear to examine at the SRO level - need to add some below the line TS implemenation. Is the entire piece of knowledge needed to answer this Q is knowing the entry condition for the AOP and then if you know with gas bound pumps you secure all three and therefore 3.03 is all that is left, it still seems like RO knowledge to me. This was a free review question (why marked as an E but is really unsat) but I am not sure it will meet the SRO only guidance if this is true. This is probably why it was on the RO exam before? Changed the K/A and developed a new question. Delete second bullet in given. Re-arranage part 1 and 2.
Changed the K/A to the randomly selected K/A. Deleted the second bullet from the "Given section". Seperated part 1 and part 2 as recommended. Question is now SAT.
82 H
3 X
X N
U / S B
43.5 There is no SRO aspect to this question. There is no procedure direction given. Question asks for general knowledge of the strategies contained within an AOP. Cueing: if you want to know when to stop emergency borating, most applicants would likely consider that the emergency boration procedure would be the first place to look. Where are the procedure transitions or decision points of the SRO? If he was in the ESD procedure at step 40 then a kickout to OP-903-090 might be appropriate but the entry conditions for the AOP is RO knowledge. Also have cueing from stem to answer therefore unsat. Kept the same K/A and developed a new question. Clean part 2 to move direct, put in comment section why it is SRO only.. Re-worded part 2 and added to the explanation section why this question is "SRO only". Question is now SAT.
83 F
2 X
X N
U / S D
43.5 KA mismatch. This question does not examine the ability of an applicant to determine operability of a safety related component. This should be a tech spec operability question with ties to the TS basis to make it SRO-only. This is an admin process/procedure wording question - this is STA and/or RO level knowledge. This does not meet the KA for determining op/avail of equipment while in functional recovery therefore it is unsat. This is a tier 1 question so it needs to have equipment in it that need operability eval or availability checked.This might be used in the Tier 3 for SRO only but would need some. Wrote a new question and kept the K/A. Question is now SAT.
84 H
2 X
N E / S D
43.7 Question tests AOP entry conditions and systems knowledge - designated by 1021 to be RO level knowledge. See ES-401 Attachment 2 Section E. This would be a good opportunity for a fuel handling TS basis question - core alterations, containment pens, constant comms, etc... Part 2 is not credible with loss of refuel verbiage in the stem it cues to an abnormal event so outage operations would not be credible for this question. Put an attachment #. Kept the K/A, changed the question. Question is now SAT.
85 H
3 X
N S
C 43.5 Question tests AOP entry conditions and systems knowledge - designated by 1021 to be RO level knowledge. Have the applicant choose which section of the firefighting strategies procedure that the SRO should direct. Also an excellent opportunity for an E-plan call question. Either of these would be at the SRO level. You would not evacuate the CR for a fire in the relay room so part 2 is not credible. Knowing what equipment is in the relay room is RO knowledge isn't it. Therefore this is not SRO-only and unsat. Added more information to the explanation on why the distractors are plausible and why this question is SRO only. Added the sub-sections to the AOPs. This question is SRO only because the applicant must enter the correct subsection of the AOP and have knowledge of a decision step located in that off-normal procedure. Question is now SAT.
86 H
4 B
S A
43.5 2010 NRC exam.
87 H
3 X
N S
D 43.2 Why would it be plausible that the fans would be inoperable in FAST mode? Fast speed is plausible because if the control switch is placed in the FAST position the sequencer that starts/stops the fans automatically on changing temperatures is inoperable. Added to the explanation "The fan being in fast and inoperable is plausible because the sequencer at which the fan starts/stops on rising and lowering temperature is inoperable when the C/S is in FAST. This sequencer is not part of the fan operability." Question is now SAT.
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 88 H
3 X
X N
U / S A
43.5 Question tests AOP entry conditions and system level knowledge - designated by 1021 to be RO level knowledge. What TS implications are applicable with two CS pumps in stop? Reportability requirements? These are SRO-level questions. Also, it would not be hard to argue that the crew might enter the reactor trip procedure as well - it would not be unreasonable for the SRO to trip the reactor if containment is being sprayed down by CS. Therefore distractor C could be ruled as partially correct on appeal. You don't need to know SRO kowledge to answer this question. If I know I don't trip for this (RO knowledge) that eliminates C and D. The cueing in the stem to RCP gives away answer as A, therefore it is unsat. Kept the K/A and changed the question. Quesion is now SAT.
89 H
4 N
S B
43.2 90 H
4 X
B E / S D
43.5
- Question tests major EOP entry conditions and system level knowledge - designated by 1021 to be RO level knowledge. This tests entry conditions for the diagnostic and RO knowledge for second part, therefore unsat. Developed a new question to a new K/A. 2012 NRC Exam Question
- Question is now SAT.
91 F
3 X
N E / S D
43.5
- Question tests AOP entry conditions and system level knowledge - designated by 1021 to be RO level knowledge. Part A is RO knowledge and Part B is AOP entry condition (RO knowledge) therefore it is unsat. Wrote a new question for the same K/A. Testing TS knowledge "below the line" for TS 3.1.3.2. Question is now SAT.
92 F
3 X
N S
B 43.5 93 F
2 X
N E / S D
43.5
- Question tests EOP/AOP entry conditions and system level knowledge - designated by 1021 to be RO level knowledge. Also, you have already tested the concept explained in distractor analysis for choice A - see Question #1. With RO knowledge you get straight to either C or D. Might be able to modify them to complete SRO-only knowledge aspects. Could make them choose which procedure has the IA actions in it via the setup but question is still unsat after free review mods.
Changed the K/A and the question. Part 2 of the question is TS basis knowledge for required Spent Fuel Pool level. Question is now SAT.
94 F
2 X
N E / S A
43.1 2011 NRC exam. This question was used on the 2011 RO exam. Therefore it is an RO level question. If a question has been approved at the RO level on previous exams, it should not be submitted as a question on an SRO exam. From previous RO exam. Still looks like RO knowledge to me. Even though you have SRO in the stem, it is required for an RO to know this therefore it does not discriminate at the SRO-only level therefore it is unsat for the SRO portion of this exam. Developed a new question with the same K/A. The new question is written on shift staffing. Question is now SAT.
95 H
4 X
B S
A 43.5 2014 NRC exam.
96 F
2 N
S B
43.5 97 F
3 X
X N
E / S B
43.5 Is it not an RO-level function in the control room to perform these logs? This question does not test knowledge of surveillance procedures. It tests routine tracking mechanisms that are used by control room operators. This looks like a log question to me not surveillance procedure knowledge. What about things like preconditioning, stroke time challenges, etc? I think this is RO knowledge for log keeping and shift turnover and therefore unsat. Changed the question and kept the same K/A. Changed "the CRS must" to Technical Specifications require". This was done to fix a possible subset issue. Question is now SAT.
98 F
3 N
S A
43.2 LOD-3, not 4. Changed the LOD-4 to LOD-3. Question is SAT.
99 H
3 X
N S
C 43.5 Distractor B is not credible. Why is it plausible for the CRS implement RC-2 if it is not met? Can you change the answers to A) RC-1 only / RC-1; B) RC-1 and RC-2 / RC-1; C) RC-1 and RC-2 /
RC-2; D) RC-2 only / RC-2. I don't think this LOD is 4, but 3. All I need to know is that I have two CEAs stuck out and even with RC-1 met I am using RC-2. If only RC-1 is met, the guidance states that RC-2 must still be implemented to ensure emergency boration is in progress.
Emergency boration is in progress in this instance because there is a SIAS. If the applicant is not aware that emergency boration is in progress, he/she will select this answer. Question is now SAT 100 F
3 N
S C
43.5 Reference provided.
ES-401 WT-2017-03 Written Examination Review Worksheet Form ES-401-9 Results Table RO LOK -H 39 Avg RO LOD 2.56 Flaws 10 CFR Distribution RO LOK-F 36 AVG SRO LOD 2.88 Stem focus 5
41.1 24 43.1 1
SRO LOK - H 15 Overall LOD 2.64 Cues 6
41.2 1
43.2 4
SRO LOK - F 10 T/F 0
41.3 3
43.3 0
Cred Dist 13 41.4 5
43.4 0
12 Partial 6
41.5 4
43.5 19 RO Mod 0
0 SRO Mod 1
4 job link 0
41.6 7
43.6 0
RO New 49 65.33 SRO New 21 84 units 0
41.7 17 43.7 1
minutia 0
41.8 6
Total Bank 29 29 backward 0
41.9 1
Total Mod 1
1 KA 12 41.10 24 Total New 70 70 SRO-only 15 41.11 3
LOD = 1 0
41.12 2
RO Sat 43 57.33 SRO Sat 10 40 41.13 1
20 Answer Dist ( in %)
41.14 1
RO Edit 22 29.33 SRO Edit 10 40 RO-A 19 SRO-A 6
RO-B 18 SRO-B 5
Total Sat 53 53 Total Unsat 15 15 RO-C 20 SRO-C 6
Total Edit 32 32 RO-D 18 SRO-D 8
- 1. There are 27 questions out of the total 100 that were on the 2010 or later NRC exams at Waterford. While the provisions for bank useage and previous 2 NRC exams are met, the current submittal raises some intial concerns at the Chief Examiner level as to the predictabilty of the exam due to having over 25% of the questions being repeated from the previous 5 NRC exams at Waterford. The bank provision assumes that some of the bank questions will come from the initial training or requal question banks and/or from other facility exams. It is expected that applicants will review the most recent NRC exams at their facility, and as it stands, they will have already seen over 25% of their NRC exam having done so. 2. The exam level of difficulty was adjusted for a number of questions - too many of the new questions on this exam do not go into sufficient depth to adequately test between the qualified and unqualfied applicant. As questions are rewritten, please look for opportunities to increase some LOD-1/2 questions to the LOD-3 level. 3. The effects of commenst 1 & 2 combine to create a concern with the exam - the majority of the more difficult questions on the exam are those questions from the past 5 NRC exams at Waterford. Therefore, the effect is - the exam LOD is at an acceptable level by metrics, however the more difficult questions become much less difficult after the likely applicant review of the last 5 written exams at Waterford. Resolution: The licensee replaced multiple previous W3 exam questions and increased the LOD of several questions - the examination is now SAT. 4. There are too many questions for which the KA is system-based, but the question tests 1/2 system knowledge and 1/2 procedure knowledge. This is made evident by the 10 CFR 55.41.10 high numbers in the distribution below. Those types of questions should be reserved for the KAs where knowledge of how to mitigate a failure using procedures is a required part of the KA. 5. For any of the questions that are marked as modified from the bank - please attach the parent question after the modified question so we can verify that the question has been sufficiently modified. Please do this on the final exam submittal. In the interim, please provide the parent questions for any modified questions on the draft exam submittal. 6. The SRO exam has not been written at the SRO level. Take a look at Q86 (NRC exam bank question) - this question asks for the implementation of various sections within an AOP - this is a good example of a question that meets the minimum SRO-only level requirements. Too many new questions on this exam ask for major EOP or AOP entry condition knowledge and/or system knowledge only - this is specifically called out by ES-401, Attachment 2, Section E as not acceptable for an SRO level question. Resolution: Adjustments made to several questions to ensure exam tests at SRO level.
General Comments provided to the right (for individual question comments, Chief Examiner comments are in BLACK, and NRC peer review comments are in BLUE.
Please provide licensee response comments in RED.)