ML19322A952
| ML19322A952 | |
| Person / Time | |
|---|---|
| Site: | Millstone  |
| Issue date: | 08/22/2019 |
| From: | NRC Region 1 |
| To: | Dominion Nuclear Connecticut |
| Shared Package | |
| ML19052A319 | List: |
| References | |
| 000500, 000956 | |
| Download: ML19322A952 (21) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist link KIA Only 1
H 3
M E
ChangeD value in stem to 370, which is closer to 4 times the 7-day value, ensuring key answer correct.
Changed part 2 of choices B and D to 317°F. which is 50°F different from correct answer. requiring applicant to use steam 2
H 3
B E
tables to determine correct answer - to improve plausibility.
Changed stem of question to state PRT pressure has increased to 40 psia to ensure correct answer. Changed choices A and C to 267°F, which is correct for new PRT pressure.
3 F
3 B
E low LOD. Choices in 2x2 structure provide sufficient information 4
H 2
B E
to allow applicant to easily eliminate distractors. Removed the part 2 reasons. ending each part 2 at the word "direction",
Consider changing word in Choice C from "robbing" to "diverting".
Note: Q used on 2015 NRC Exam.
5 F
3 B
E Changed word in Choice C from "robbing" to "diverting" Refer to Section D of ES-401 and Appendix 8 for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
"Cues": The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
"T/F": The answer choices are a collection of unrelated true/false statements.
"Cred. Dist.": The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
"Partial": One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
"Job link": The question is not linked to the job requirements (i.e., the question has a valid KIA but. as written, is not operational in content).
"Minutia": The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e.* it is not required to be known from memory).
"#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
"Backward": The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those KIAs that are designated "SRO-only." (KIA and license-level mismatches are unacceptable.)
- 6.
Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" status ratings (e.g., how the Appendix 8 psychometric attributes are not being met).
Page 1 of 21
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only
- 1) Recommend remove bolding from stem. Not needed.
- 2) OP-3301G Section 4.2 states controller output decreased to close spray valves. So set point step rise would have decreased output to raise pressure.
Operator would then need to INCREASE output to lower pressure. making Choice D, not Choice C, the correct answer.
- 3) Key answer justification says pressing decrease with master controller in manual will lower "program setpoint". In manual, the decrease pushbutton should lower output, not change the auto setpoint.
6 H
3 M
E
- 4) Perhaps simpler wording to state fill in the blank as "The RO will depress the (1) pushbutton to reduce the amount of (2), with choices a) decrease, spray flow. b} increase, spray flow, c) decrease, energized heaters, d) increase, energized heaters.
Removed bolding from stem.
Verified controller operates as described in the correct answer.
Enhanced justification to describe design of controller to lower pressure by selecting "decrease".
Changed question statement to include two blanks, to simplify wording.
7 H
3 B
s Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
Recommend change word order in Choices C and D, consistent with A and B, and no need to mention the PEO:
c) HP Steam Isolation Valve to the "A" TDMFP (3TFM-V8) was 8
H 3
N E
inadvertently closed.
d} Long Recycle Manual Isolation Valve (3FWS-V986) was inadvertently opened.
Changed wording in Choices C and D to be consistent with A and B as described above.
Edit to reduce word count:
At which of the times shown above was the crew first...
a) 0100 b}
0110 9
H 3
B E
c) 0115 d}
0120 Edited the choices to times only, as described above.
Page 2 of 21
ES-401 3
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Is it valid to say no operator actions have been taken if the crew is in ES-0.1? Perhaps better to say no operator actions have been performed on the plant ventilation 10 F
3 N
E systems.
Changed to say no operator actions have been taken with plant ventilation systems.
Stem is leading in stating channel selection of pressurizer level control system without stating alignment of any other plant components. This makes it obvious that the Q is testing the effect of VIAC #1 on Pressurizer Level Channel
- 1. Is the information necessary? If Channel 1 / 2 is the normal selection then better to just state 100% power with 11 H
2.5 N
s systems in nonmal alignment and not mention pressurizer level control system.
No change was made to the question, since there is no procedural requirement to have the controlling channel to be selected in Chanel 1 versus Channel, and this infonmation is necessary to determine the correct answer.
Choice A is a subset of Choice B. Energizing the bus from the swing charger implies performing whatever actions are necessary to energize the bus from the swing charger, which, no doubt includes closing the charger's AC input 12 H
3 B
E breaker. Need to work on Choices A and B Changed choices A and B to describe the first action the PEO is required to take.
No need to state "and continue to monitor" in Choices C and D. Sufficient to iust sav "and monitor".
13 F
3 B
E Removed the words "continue to" from choices C and D.
14 F
3 B
s 15 F
3 N
s Submitted as a sample test item for preliminary NRG review per NUREG-1021, ES-201, Section C.2.c.
16 H
3 B
s Note: Q used on 2015 NRC Exam.
Page 3 of 21
ES-401 4
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only 0#16 is leading to the answer for this Q#17 by virtue of it immediately preceding Q#17 on the exam. Recommend move Q#17 to much later in the exam. Perhaps make it Q#65.
17 Now F
2.5 N
s Verified question #16 does not provide direct information to 27 help answer question# 17.
Moved question from #17 to #27 on the exam, which is the end of the Tier 1 questions, breaking the applicant's train of thought before answering a second question related to a loss of emergency coolant recirculation.
The justification states reset SIS and then either reset P-4 or pull logic cards. Recommend add to justification to explain that having entered FR-H.1, the low SG levels preclude 18 resetting P-4, therefore only option is to reset SIS and then Now H
3 B
E pull logic cards.
17 Verified Choice C is the only correct answer, since resetting P-4 would not clear the seal in feature caused by the SIS.
No change was made to the question.
Consider adding to Item 5 in the stem, to restore Tave to 19 program".
Now H
3.5 M
E Added "to restore Tave to program" to Item 5 in the stem.
18 20 Now H
2.8 B
s 19 21 Now H
2.2 N
s 20 Page 4 of 21
ES-401 5
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward 0-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Explain KIA match (use of reference material). Choice A plausibility adversely impacted by "excessive flow" out of the discharge. Recommend providing a flowrate, perhaps 65 scfm, for air ejector flow vice the conclusion statement that flow is excessive. Similarly, for Choice D, recommend providing a normal suction manifold temperature, perhaps 22 112 F, vice the conclusion statement that manifold temps Now H
2.4 B
E are normal.
21 Explained KA match in justification section.
Provided a flowrate of 65 scfm in the question stem.
Did not change stem with specific manifold temperature, since the PEO will check for elevated temperature with their hand, rather than read a specific temperature locally.
- 1) Add procedure to Q, "... the loss of Bus 34C in AOP 3577".
- 2) In justification, explain for key answer that "B" RPCCW is not affected by the loss of Bus 34C.
- 3) Is there a need to say "locally" in Choice A? Seems equally valid to just state "Shift the RCP seal return path to 23 the top of the VCT." It doesn't matter that this would be Now F
3 B
E performed locally. Recommend remove the word "locally".
22 Changed question to state actions... "directed by AOP 3577"...
Explained in justification that "B" RPCCW is not affected by the loss of Bus 34C.
Removed "locally" from choice A.
Recommend removing the 2nd sentence from Choices C and D. These sentences are not needed and have a cueing effect.
24 Removed second sentence from Choices C and D to avoid Now H
3 B
E cueing.
23 Changed basis information in Choices A and B to make the bases information less similar to descriptions in question
- 32, to prevent cueing.
Changed justification to explain why the new choice descriptions are plausible, but wrong.
Page 5 of 21
ES-401 6
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Recommend adjust the wording of the Q or of Choices B and D. As written, B and D don't appear to answer the Q.
Q asks which paths CAN BE monitored. Choices explain 25 what CANNOT BE monitored.
Now H
2.5 M
E 24 Changed question to ask which of the paths "are required to be verified locally by a PEO, versus being able to check them on the main boards".
26 Now H
3.5 B
s 25 Key answer length is much shorter than other answers.
Perhaps:
a) Turning Backup Heaters on b) Initiating Auxiliary Spray flow 27 c) Raising SG levels to 50% with AFW Now H
3 B
E d) Starting a cooldown at < 50F per hour 26 Changed answer lengths to wording very similar to that described above to make the four choices more similar in length.
28 H
3 B
s 29 H
3 N
s 30 F
3 N
s Page 6 of 21
ES-401 7
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only
- 1) The last stem bullet relating to 618 helps to answer Q#4.
If applicant is confused as to which valve has the auto controller, this Q clears up that confusion. Recommend delete last bullet and add to previous bullet that the system is "operating in automatic."
31 H
3 B
E
- 2) Is the qualifier "rapidly" necessary in the stem?
Recommend deleting "rapidly".
Changed bullet to read "RHR Main Board Controls: In
'AUTO"'
Changed "rapidly fails" to "breaks".
- 1) Explain in justification why this is SRO only. RO expected to know major action category and most effective strategy in FR-C. RO expected to know plant response as charging flow is initiated under the given conditions.
- 2) Poor linkage between paragraph before the fill-in-the-blank and the fill-in-the-blank: "effectiveness of level recovery" vs "effectiveness of core cooling method", "how level will respond" vs level AND reason AND effect of level recovery". One way to address is to replace the paragraph with which of the choices below correctly completes the following statement?" This change would also require some 32 H
3 N
E modification of the fill-in-the-blank to include the "in accordance with" and possibly other information.
No change made to justification about SRO only, since question is a RO level question.
Changed question statement from Charging Pump effectiveness at recovering reactor vessel level to its effectiveness "at restoring core cooling... "
Changed choice D to more closely align with WOG Background Document.
Page 7 of 21
ES-401 8
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only
- 1) Wording problems in Part 2 choices. RO will not close valve based on HIGH temperature. Rather, the RO will close valve based on high temperature condition being corrected. Better to change Part 2 to choices of
'temperature only" and "temperature OR level", without the "HIGH" qualifier.
- 2) Trivial Part 2 choice. Why would RO only close the fill valve based on temperature and not based on level? To believe that, the applicant would have to think okay to fill 33 F
3 X
N u
PRT power if needed to reduce temperature, regardless of effect on level. Not understanding plausibility of Part 2 for Choices A/C.
Redrew KA based on plausibility issues with the distractors, and inability writing a discriminating question about monitoring PRT temperature.
New question has been added to match the new KA statement. New question was reviewed by the NRC during prep week.
- 1) The Part 1 determination is too low level. The basic function of the PRT is to contain and quench the steam released from PORVs or Safeties via a sparger under a water volume. All applicants will know continuous discharge into the PRT will result in blowing its rupture disk and then discharging steam relief direct to containment. No discriminatory value to Part 1 of the choices
- 2) Is setup operationally valid? Stem states PORV leaking by AND WILL NOT RESEAT, and block valve will not close.
34 F
3 B
E Would stroking to reseat be attempted if block valve could not be closed?
- 3) No need to put specific valve identifiers in parenthesis in stem. Stem is clearer and shorter without these identifiers.
They are unnecessary. Recommend deleting them.
Removed Part 1 determination, expanding part 2 flowpaths.
Removed statement that the PORVwill not reseat.
Kept valve identifiers, as applicants are provided both of these in their procedure steps, and are used to seeing them.
35 F
3 B
s Note: Q used on 2015 NRC Exam.
36 H
3 B
s Page 8 of 21
ES-401 9
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only No need for Choice B to include failed open". Enough to state, "The PZR PORVwill auto-close... " Recommend 37 H
3 B
E delete "failed open".
Deleted failed open" in Choice B.
Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
38 H
3 N
E Changed Choice C from Train "A" to Train "B" (typo).
- 1) Choice D not plausible. Transmitters can fail high or low.
Applicants will have seen examples of this multiple times throughout their training program. No plausible basis or common misconception that would have an applicant think a pressure transmitter can only fail HIGH.
- 2) Better to expand the fill-in-the-blank, rather than have an explanatory preceding paragraph. Recommend: "Complete 39 F
3 X
B u
the following statement. Only 3 channels of Containment Pressure are required to input.. because the Containment pressure transmitters (blank]"
Changed choices C and D to discuss bistables, versus transmitters.
Shortened the preceding paragraph, and expanded the fill in the blank sentence as described above.
Part 1 seems trivial and not discriminating. Not reasonable for an applicant to think they must verify pressure within TS pressure limits but not also required to verify temperature within TS temperature limits.
Changed to a new question from 2017 NRG Exam to 40 F
3 X
N u
improve discriminatory validity. Replacement question focuses on manual operation to control Ctmt pressure, rather than on monitoring these parameters.
Update since NRG prep week, changed question stem to state RO is directed to lower pressure at the maximum rate, to make distractor "B", which could have been interpreted as a second correct answer, clearly wrong.
Page 9 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
0 LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Not necessary to have pump and valve identifiers under the pump and valve titles. Recommend delete the identifiers.
41 H
3 B
s No change made, since applicants are used to seeing descriptors and identifiers for valves and pumps.
Although not direct overlap, 0#42 provides infonmation that may prompt applicant to better reason their way through 0#41. Recommend move 0#42 elsewhere in the written exam, not immediately following 0#41 (similar to comment 42 on 0#17).
Now F
3 N
s 55 Moved question #42 to #55, which is at the end of Tier 2, Group 1, to break applicant train of thought before answering a second question on the Ctmt Recirc Spray System.
43 Now F
3 N
s 42 44 Now F
3 B
s 43 Not a KA match. KA intends testing a MFW design feature which provides for automatic turbine/reactor trip runback.
45 This O tests understanding of Stator Coolant interlocks that Now F
3 X
B u
would initiate a MT runback. It does not test MFW design 44 features.
Redrew new KA, since old KA is NA for Millstone 3.
New question reviewed by NRG during prep week.
46 Submitted as a sample test item for preliminary NRG review per Now F
3 N
s 45 NUREG-1021, ES-201, Section G.2.c.
47 Now H
3 M
s 46 48 Post NRG prep week, corrected typo in question, changing Now F
3 B
E "are" to "is". "Which of the following plant loads is powered 47 from... ", since only one listed load is powered from 35C.
49 Now F
3 N
s 48 Page 10 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Choice A is not plausible. Battery Chargers, by their name, charge batteries. If the charger is lost, the battery is no longer being maintained charged. If operator knows BC1 output goes to DC Bus 1 then he/she also would know a battery is connected to DC Bus 1. Recommend change Q to fill-in-the-blank: "Assuming no operator actions have been 50 taken, the BOP will observe DC Bus 1 voltage in the range Now H
3 X
B u
of[blank]". And change choices to bands: a) 111-118, 121-49 126, 132-137, and 138-143. Applicants should know the battery is designed to power the inverter for a minimum of 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> and would therefore be close to fully charged at 10 minutes into event.
Changed choices to voltage bands.
Changed question to fill in the blank.
- 1) Choice D appears to be a 2nd correct answer.
Justification states selecting "Parallel" while in "Unit" risks the EOG picking up too much load. This is in conflict with justification for Key Answer Choice A, which states placing speed droop in service will cause frequency to decrease while supplying a bus by itself. If the EDG speed lowered 51 such that bus frequency is less than grid frequency, then Now H
3 B
E upon paralleling the grid will pick up load from the EOG, 50 potentially causing the EOG to reverse power.
- 2) Inconsistent nomenclature in stem ("A" EOG vs RSST A).
Verified Choice Dis wrong, Changed "RSST A" to '"A' RSST" Capitalized Diesel in Choice D 52 Now F
3 B
s 51 Page 11 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Choice D is implausible because the stem implies the closed valve is an abnormal failure-type of condition and because any auto open signal for cooling should be designed to occur on engine start, not after the bus energizes since cooling needed even if running unloaded.
Recommend change stem statement from "discovers" valve closed to "determines" valve closed, to get away from 53 indicating the valve should have been open. Change Now F
3 B
E Choice D to "... will open when the diesel engine starts." It is 52 plausible, but wrong, to think the valve may have auto open signal off of engine start since that is when cooling is needed.
Changed "discovers to "determines".
Changed choice D to "when the diesel starts."
Changed justification to support changed Choice D 54 Choice D Part 2 has an extra space.
Now F
3 B
E Removed extra space in choice D 53 55 Now H
3 B
s 54 Interpolation is not a very good distractor. Recommend minimizing its use and improving distractors by modifying Choices C and D as shown below. The recommended change has both A and D using the same basis, and B and C using the same basis.
C. use the "Alf Rods In" curve, in order to provide sufficient "shutdown amount" to ensure an inadvertent MODE change does not occur during the withdrawal of the shutdown bank rods 56 F
3 B
E D. interpolate between the "Alf Rods In" curve and the "Shutdown Banks Out" curve, to provide increased Shutdown Margin due to the potential for a stuck rod after the shutdown banks are withdrawn.
Modified Choices C and D to use the "All Rods In" Curve.
Repeated the two bases twice each amongst the four choices.
Update since NRC prep week, corrected typo in questions stem, adding "is", to read "The plant is in MODE 3... "
Page 12 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only No need to underline "Group "E". There is nothing confusing in the Q requiring any emphasis. Recommend 57 F
3 N
E removing the underline.
Removed the underline from "Group E".
58 H
3 B
s
- 1) Is it necessary to state, "all selectable controllers on the Main Boards selected to Channel 1? If this is normal configuration, enough to say nothing or to just say the controllers are aligned normally for full power operation.
- 2) Is it a technically correct statement? There are LOTS of controllers beyond the ones for RCS pressure and level control.
- 3) Q structure (Part 2 seems to lead toward a 'yes' answer.
Consider rephrasing question and matching Choice A 59 H
3 B
E structure more closely to other choices. Perhaps:
Which of the following correctly describes whether or not the reactor automatically trips?
a) No, due to coincidence logic not being met.
Rephrased question and choice A to make choice A better match the other choices, as described above.
- 1) Typo in answer justification (""CC correct").
- 2) Clarify what is meant in justification for Choice A regarding "60% power based on each of the FOUR SGs 60 F
3 B
s providing 20% of the total steam flow".
Fixed typo in justification.
Changed justification to clarify plausibility of Choice A.
In the stem, replace "again start" with "be" for clarity.
61 H
3 N
s Reviewed wording. No change made.
62 F
3 N
s Page 13 of21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Is Q setup operationally valid? Colder feedwater due to loss of feedwater heating should result in higher reactor power due to a reduction in Tavg. Reactor power is changed by adjusting the core reactivity balance. If Tavg remains constant during a secondary transient, then reactor power should remain constant. The Q stem indicates the core delta 63 H
3 M
E T has increased but also states Tavg indication is still at its original value.
Ran on the simulator, than changed stem to match simulator response, showing Tave dropping slightly.
Modified Choice Oto make it incorrect with Tave decreasinQ.
Part 2 of the Q is too low LOO and easy to guess because a basic tenet of SBO design is that the machine not be dependent on plant equipment to enable its operation.
Recommend making minor change to Part 2 as shown, removing direct reference to the Service Water system:
64 F
3 B
E Heat is then transferred to [blank] heat exchanger.
A.
an air-cooled B.
a water-cooled C.
an air-cooled D.
a water-cooled Modified part 2 to "air cooled" and water cooled".
- 1) Disconnect between Q (what actions will US direct) and choices (the PEO will... ). Perhaps change Q to what US-directed actions will the PEO take?"
- 2) Should fuel" be capitalized in stem in phrase "EOG fuel 65 H
3 B
E Oil Vaults"?
Changed Question to what US-directed actions will the PEO take?"
Capitalized "Fuel".
Page 14 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
0 LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only
- 1) If the applicant knows it is never acceptable to withdraw rods in a transient to restore temperature, then there is no need to review other choices and no other choice could be a correct answer. Recommend modify Oto "Which of the following is the FIRST improper action taken by the crew during this event." Might also need to modify some stem items, such as adding to #4 that "the crew continues the downpower, in order to support distractor A.
- 2) There is misalignment between the O asked {what improper action taken) and Choices A, Band D {the crew was required to... ). For example, the proposed improper 66 H
3 M
E action in Choice A should be stated, "the crew did not trip the reactor when... " The only answer stated as an improper action is the key answer.
Changed question to ask about "the FIRST improper action" the crew took.
Removed "continuously" from the rod withdrawal.
Changed Choices to state that the "crew did not" or "should/should not" to state the choices as improper actions.
Added statement that the crew continues the downpower to item 4.
This 067, which immediately follows 066 asks about requirements of OP-AP-300, which is basis of key answer in 066. It is somewhat cueing, similar comments on 0#17 and 0#42. Recommend moving this O to somewhere else on 67 written exam, separated from 0#66 by many questions.
Now F
3 B
s 75 Moved question 67 to 75, which is the end of the RO exam, to break applicant train of thought about reactivity management prior to the second question related to reactivity management.
68 Now F
3 B
s 67 Please provide explanation/justification as to why this is an appropriate RO level question as it appears to be testing 69 knowledge of activities more reasonably in the job scope of Now F
3 N
s an SRO.
68 Added description to justification as to why this is a RO level question, testing high level concept of what equipment is related to Shutdown Risk, without having to take any action related to the status of the equipment.
Page 15 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only
- 1) Q source states "Robinson". It should state "Bank, Robinson... ", as this field is intended to state whether Bank, New or Modified.
- 2) Part 2 is not discriminating. Fairly obvious that announcement should include stand clear of the high voltage, dangerous switchgear that could fault during breaker operation. Applicants will have heard this announcement many, many times in the course of their normal in-plant duties. Change Q to ensure all choices 70 have discriminatory value.
Now F
3 B
s
- 3) Q is clearly structured as two distinct Q's, even to 69 directing the applicant to answer the following TWO questions. Ok to have Q with multiple parts, to support a 2x2 choice structure. However, should adjust structure to show it as a single Q with multiple aspects.
Changed question source to "Bank - Robinson".
Made fill in the blank format.
Did not change content of question, due to random draw of fairly straightforward topic.
- 1) Too easy to eliminate all distractors. The Q asks ADDITIONAL requirement for entering a specific LHRA within containment, BEYOND those already required for a LHRA. Each of the 3 distractors is a type of requirement that might be applicable for general entry into a LHRA. All distractors can therefore be eliminated without knowing whether or not they actually are LHRA requirements because the Q asks for a requirement in ADDITION TO the LHRA entry requirements.
- 2) Q does not state reason for MIDS area entry. OP 3361A 4.1.5 provides guidance if controlled MIDS movement is 71 required during the entry and 4.1.6 only requires drive motor Now F
3 B
E tagging if movement is NOT required. Since need for movement is an unknown in the Q, the applicant could apply 70 either 4.1.5 or 4.1.6, however, the key answer is only correct under the assumption that MIDS movement is not required (Step 4.1.6).
- 3) Key answer stands out as it is only choice with info specific to the particular area to be entered. All distractors relate generally to radiation areas, not specifically to the given area.
Modified question to add more information about the status of the MIDS System, and changed two distractors to discuss the status of the MIDS System.
Page 16 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/MIN)
(U/E/S)
Focus Dist Link KJA Only 72 Now F
2 B
s 71 Q should not be set up and asked as two questions (see 3rd comment on Q#70). Perhaps reword as "which of the 73 following is correct regarding the reactor and procedure Now H
2.5 N
E implementation?'
72 Converted the question to fill in the blank format.
Per ES-401 of the Examiner's Standard, exam developers are to "Ensure that the questions selected for Tier 3 maintain their focus on plantwide generic K/As and do not become an extension of Tier 2." Contrary to that guidance, this 074 tests knowledge specific to the Service Water System. A Tier 3 question for Generic 2.4.31 should test knowledge of alarms, indications, or procedures that is applicable plant-wide rather than specific to a system. For example, one might test understanding of how to operate or 74 interpret certain kinds of instruments or controllers that are Now H
3 B
s used in multiple systems. Or to demonstrate generic plant-73 wide knowledge of procedures might mean testing requirements for different types of procedures (in-hand, reference use only).
Modified justification to explain why this question is acceptable as Tier 3, due to testing knowledge of AOP versus ARP usage, in spite of operation focus of the question.
Update since NRC prep week, added specific bank question number.
- 1) Question is testing understanding of specific system response, not plant-wide generic knowledge.
- 2) Stem should say BOP observes rather than BOP reports as the report of these particular alarms seems contrived and unlikely in the given situation.
75
- 3) Answer choices give more information than is being Now H
3 X
B u
asked. Answers don't align well with the Q as the Q doesn't 74 ask for reasons or actions that are subsequently required.
Redrew KA, since unable to test ability to verify alarms are consistent with plant conditions without testing specific system responses.
New question was reviewed by NRC during prep week.
Page 17 of21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only 76 H
3 N
s ES-1.3 has a unique note in the front of the procedure different from most other EOPs which states that FR procedures are NOT to be implemented until after Step 5 (after CL recirc established). This appears to be an RO 77 F
3 B
s knowledge level Q as ROs are required to know procedure notes and cautions and general strategy of procedures.
NRG determined question was OK as written.
78 H
2 B
s 79 H
3 B
s 80 H
3 N
s Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
81 H
3 B
s Too many Q's being asked. Recommend reword as "which of the following is correct regarding limitations in TS 3/4. O?"
82 H
3 N
E Reworded as "which of the following is correct regarding the limitations/allowances in Tech Spec Section 3/4.0?"
- 1) Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
- 2) Revised Q written as two Q's that are not closely linked.
83 H
3 N
s Replaced original question, due to NRC feedback from 10-question early submittal that distractors were not plausible.
NRC reviewed new question during prep week.
Page 18 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only
- 1) Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
- 2) Why is Distractor Ca wrong answer? 1-131 has an 8.02 day half-life. Using formula for radioactive decay (N=Noe'(-
lambda*T), the 1-131 concentration will only have decayed to 84% of its initial peak value 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> after the spike. Does the degasifier also play a role in reducing the RCS noble gas 84 F
3 N
E concentration? Per TS bases, 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> is acceptable since the NORMAL iodine concentration would be restored WITHIN THIS TIME. Radioactive decay, by itself, wouldn't seem to be able to restore NORMAL concentration in 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> so the analysis appears to credit a cleanup effect in addition to decay.
Added "Tech Spec Bases states... " to make the wrong choices clearly wrong.
Q used on 2015 NRC Exam. SRO section has 3 Q from last two NRC exam (Q85, Q86, Q97). Per Form ES-401.6, there should be nor more than two 'repeat' SRO Qs.
85 H
3 B
s Removed repeat question 86, to reduce number of repeated questions to 2 questions
- 1) Q used on 2015 NRC Exam. SRO section has 3 Q from last two NRC exam (Q85, Q86, Q97). Per Form ES-401.6, there should be nor more than two 'repeat' SRO Qs.
- 2) Appears this Q can be answered with RO "above the line" knowledge of TS 3.4.9.3. The applicant does not need to assess the progression that led to the current plant state.
The answer can be determined by applying given plant conditions (220°F, one PORV isolated, one RHR train 86 H
3 X
B u
isolated) to the TS LCO 3.4.9.3 relief capability requirement of at least one PORV and one RHR suction relief in MODE 4 < 226°F.
Redrew KA due to being unable to make SRO level question related to the original KA.
NRC reviewed the new question concept during prep week.
Update since NRC prep week, finalized new Question #86 87 F
3 N
s Does Item #1 in the stem help to answer Q#34?
88 H
3 B
s Verified question does not help answer question 34 Page 19 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOO Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only 89 H
3 M
s 90 H
3 B
s 91 F
3 N
s Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
92 H
3 M
s 93 H
3 M
s 94 F
3 B
s Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
- 1) Choice 02 does not align with the Q. The fill-in-the-blank states '1he crew IS REQUIRED TO,... ". Choice D2 is to "continue the startup, and initiate a CR". No reactivity controlling procedure would demand or REQUIRE a startup to continue. The crew always has the option of stopping the startup. Therefore, implausible as written. Perhaps the fill-in-the-blank can be modified to encompass all choices.
- 2) Choice D2 is implausible based on wording of D1. If applicant has determined criticality will be outside of admin 95 H
3 M
E LIMITS, then applicant would not think ok to continue startup. Perhaps can fix by deleting admin limits from C1 and C2. Recommend Part 1 choices state either "below RIL" or "above RIL" and leave out any mention of admin limits. Applicants are expected to know, without cueing, that there are admin limits in addition to actions for below RIL.
Changed "required" to "required/allowed".
Removed "administrative limit" from choices C and D.
Made Part 1 choices choose between "below RIL" or "above RIL".
96 F
3 B
s Submitted as a sample test item for preliminary NRC review per NUREG-1021, ES-201, Section C.2.c.
Q used on 2015 NRG Exam. SRO section has 3 Q from last two NRG exam (085, Q86, 097). Per Form ES-401.6, there 97 F
3 B
s should be nor more than two 'repeat' SRO Qs.
Removed repeat question 86, to reduce number of repeated questions to 2 questions.
Page 20 of 21
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
Q LOK LOD Source Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward 0-SRO (B/M/N)
(U/E/S)
Focus Dist Link KIA Only Typo in stem. The "1104" entry should say "2 out of 4". The word "of' is missing.
- 2) Choice A doesn't line up well with the a. The Q asks if the AOP WILL require a plant shutdown. The answer choice makes a definitive conclusion ("NO") but then states 98 H
3 N
E the AOP will have the crew wait to confirm a leak exists by chemistry sample before starting the shutdown. Looks like from Choice A that the Q should be, "based on given conditions has AOP threshold for requiring a plant shutdown been met?"
Corrected typo in 1104 entry.
Changed 'Will" the AOP... to "At this point"...
99 H
3 N
E Update since NRG prep week, corrected typo in choice B, making "1)" into "(1)".
100 F
3 N
s Page 21 of 21