ML12165A137
| ML12165A137 | |
| Person / Time | |
|---|---|
| Site: | Prairie Island  |
| Issue date: | 05/22/2012 |
| From: | NRC/RGN-III |
| To: | |
| References | |
| ES-401-9 | |
| Download: ML12165A137 (9) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- questions reviewed in depth.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 2.0 N
U S/G levels lowered because given values could only be achieved with additional failures and is not indicative of initial conditions. At the point SG Levels are checked in 1ES-0.1 the level could not have achieved 80+ % without additional failures. Agreed. Replaced Q.
2 H
2.0 N
U USAR ratings for the PRZR Safeties include assumptions of the PRZR variables. The 'B', 'C', and 'D' distracters can not be proven incorrect as the condition of the PRZR is not given. Specifically, the
'B' distracter is inclusive and would be more appropriate for the normal NOP/NOT.
Initial question had no operational validity as it asked a USAR design setting which is not readable at any location and the operator will not use that information for any operational decision point.
Agreed. Replaced Q.
3 H
2.0 N
U Initial question was SRO level because candidate had to assess conditions and select recovery procedure. Agreed. Replaced Q.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 4
H 2.0 x
N U
Two improbable distractors. Removed break flow portion of distracter 'C' because this distracter becomes correct if the action taken as the second part is performed (would make reflux cooling occur if SG pressure were lowered sufficiently.)
Distracter 'D' changed because it's implausible to have loop seals involved in reflux cooling.
5 H
2.0 N
E Reformatted Q.
6 H
2.0 N
E Procedurally, the question is inaccurate. 1ES-1.2 would not be reperformed for the opposite train (12 RHR pump). The procedure directs aligning the selected train's (11) SI pump and then transition to 1ES-1.3, Transfer to recirc with one safeguards train out of service, when, presumably, the SI pump fails. Agreed. Re-worked the Q.
7 F
2.0 N
U NRC version was validated and is useable, but is exactly the same question as R51. R51 is a K/A match for R7, but not vice versa.
Recommend swapping R51 for R7, write a new question for R51 Agreed. swapped questions, wrote new R7 question.
8 F
2.0 N
S Re-formatted question.
9 H
2.0 N
S Re-format question. Remove SI actuation as this does not affect the answer or affect plausibility. Remove window dressing - Unit 1 parts.
Clean up/clarify 3" MOV operation. Change failure to MFP to match K/A since RCP failure does not. Agree. Incorporated suggested changes.
10 H
2.0 N
U E Question stem conditions do not result in a loss of all AC therefore there is no correct answer. Question is SRO level. Recommend replacing question. Disagree. Corrected EDG# making one correct answer. Question is not SRO, ROs are required to recognize EOP entry conditions.
11 F
2.0 N
U Re-formatted and removed window dressing. Facility added three plausible distractors.
12 H
2.0 N
S Re-formatted question.
13 H
2.0 N
S Re-formatted question.
14 H
2.0 B
S Re-formatted question. Added diagram.
15 H
2.0 x
N U
Two implausible distracters: a. and c. SRO only question. Agreed.
Replaced Q.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 16 F
2.5 x
N U
Question has no correct answer. ~6% of the RWST volume is required to enable recirculation (Sump B level adequate). RWST is a 300k tank, a quarter would be 75k gallons of water which is sufficient to perform recirc when combined with the entire contents of the RCS.
Agreed. Replaced question with a modified version of last ILT exam question.
17 F
2.0 N
U Question is LOD=1. Agreed. Replaced question with a Bank question: LOK = H.
18 F
2.0 x
B U
Question was partially developed when placed in the bank. Question is LOD=1. Agreed. Replaced question.
19 H
2.0 N
E Question was keyed incorrectly as it did not reflect that this controller is reverse acting. Remove VCT pressure change reference as it is not an observable effect. Also this effect is not relevant to the given conditions as the only mitigation action would be to increase flow, which is not possible with the valve fully open. Add picture of controller to preclude testing trivial knowledge of which controllers are reverse acting because this information would be available when operating them. Agreed. Added controller picture, re-worked question, but retained original idea.
20 H
2.0 N
E Clarify the direction of the controller failures. Agreed. Added information.
21 H
2.0 B
S Re-formatted question.
22 F
2.0 B
E Re-formatted question. Moved two items from stem to initial condition bullets. When finished, changed from Bank to Modified.
23 H
2.0 B
E During validation, it was determined original question has low operational validity. Agreed. Modified question to determine status of fan regardless of light indication on panels.
24 F
2.0 N
S Re-formatted question.
25 F
2.0 N
S Re-formatted question.
26 F
2.0 B
S Re-formatted question.
27 F
2.0 N
S Re-formatted question.
28 H
2.0 N
E Initial question was written assuming AMSAC/DSS does not exist.
Modify question to reflect system design with AMSAC in procedurally directed state. Agreed. Modified question to include AMSAC.
29 H
2.5 N
E Remove window dressing from stem, re-format. Agreed. Removed window dressing, reformatted question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 30 H
2.0 N
U Question had too many conditions for an applicant to apply to arrive at the correct answer. Recommend replacing the question. Agreed.
Replaced question.
31 H
2.0 N
S Re-formatted question.
32 H
2.0 N
E Suggest adding picture of controllers as this information would be available when operating them. Agreed. Added controller picture, re-formatted question.
33 H
2.0 N
S Re-formatted question.
34 F
2.0 N
S Re-formatted question.
35 H
2.0 N
S Re-formatted question.
36 F
2.0 N
E Recommend changing distracter a.to reflect alternate air compressor lineups which would make starting of 1st Standby a correct statement. Agreed. Replaced distractor.
37 F
2.0 N
E Re-formatted question, clarified who was in containment.
38 H
2.0 B
S Re-formatted question.
39 F
2.0 N
U Stem conditions are not adequate to select keyed answer.
Recommend replacing question. Agreed. Reworked the question retaining the original idea.
40 F
2.0 N
E Re-formatted question. Removed window dressing and procedure step description. Added picture of procedure step.
41 F
2.0 N
S 42 F
2.5 B
E Re-formatted question, removed window dressing.
43 H
2.5 N
U Question requires the operator to determine end state of a non design basis accident. This is not practical to describe in the stem and makes the answer indeterminate. Recommend replacing question. Agree. Reworked question.
44 H
2.0 M
E Question is SRO level. Disagree. Operators are required to recognize entry conditions into EOP. Re-formatted question.
45 F
2.0 B
S 46 F
2.0 N
E Question uses design values from the USAR which do not reflect actual equipment installed. Change question to reflect real plant conditions. Agreed. Changed question to reflect operating procedures.
47 F
2.0 B
S Re-formatted question.
48 H
2.5 N
S Re-formatted question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 49 F
2.5 B
S Re-formatted question.
50 H
2.5 x
B U
Initial question does not match the K/A as no Process Rad Monitor equipment exists that displays DAC. Agreed. Selected new K/A, replaced question.
51 F
2.0 M
S Moved original question to question 7 for K/A match. Replaced with a bank, higher skill question.
52 H
2.0 B
S Re-formatted question.
53 H
2.5 N
E Remove 121 Cooling Water pump statements as they are all correct system responses for conditions. Add drawing to clarify which valves are being addressed. Change 'D' valve positions as they are identical to 'B'. Agreed. Incorporated changes.
54 F
2.0 B
U Question tests the same knowledge as question 14. Recommend replacing question. Agreed. Replaced question.
55 F
2.0 N
E Question stem has insufficient information to select keyed answer and it is not practical to provide adequate information. Disagree.
Modified question to ask the knowledge item.
56 H
2.5 N
S Reformatted question.
57 H
2.0 N
U Recommend replacing question. 1E-3 caution provides guidance to ignore the red path due to known false nature. Question is unclear as to where in the procedure the candidate is. E-3 Bases states the flow is essentially stagnant which could be argued to be reversed/stopped or restarted. Question does not address K/A statement as requires knowledge of Natural Circulation causes.
Agreed. Re-worked question.
58 H
3.0 x
N E
Re-formatted question.
59 H
x N
U Initial question hinges on recognition of trivia (P-10 is energize to activate AND not addressed in the annunciator response like all the other bistables associated with the failure). Then this knowledge must be used to address the blocking of P-6 and subsequent failure to energize source range high volts. Recommend replacing the question. Agreed. Replaced the question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 60 H
2.0 N
U Question is unanswerable with the conditions given. The Tave failure would be addressed by placing the Steam Dump system into Steam Pressure Mode vice Tave Mode (normal). Upon Reactor Trip, with the Steam Dump System in Steam Pressure Mode and the steam header pressure indication stuck at pre-trip value, the Steam Dump System does not control RCS temperature at any specific value, but rather attempts to control header pressure, which it sees as a constant acceptable value, thus RCS temperatures fluctuate.
Recommend modifying question to show a clear failure path and remove window dressing. Agreed. Made suggested changes.
61 H
2.0 N
U E Remove the procedural selection portion of the question as this is SRO only knowledge. Clarified stem and re-organize the distracters.
Disagree. The procedures being entered are EOPs and AOPs.
ROs are required to know entry procedures to those questions.
Re-formatted the question.
62 H
2.0 N
E Remove window dressing. Agreed. Removed window dressing.
63 H
2.0 N
S Reformatted the question.
64 H
2.0 N
E Question is SRO level. ROs do not determine what procedures should be implemented. Disagree. ROs are required to recognize entry conditions into EOPs/AOPs. Re-formatted the question.
65 F
2.0 B
S 66 F
2.0 N
S 67 F
2.5 B
E Original question requires memorization of a specific procedural step.
The keyed answer omits turbine manipulation that would be necessary to make the answer a viable response. Whereas the 'B' distractor offers a correct answer based on reactivity management.
Recommend replacing the question. Added needed turbine manipulation statement, retained question.
68 H
2.0 N
U Original question does not test conservative decision making as the keyed answer is simply a 1E-0 entry condition. Agreed. Replaced question.
69 H
2.0 N
S Re-formatted the question.
70 F
2.0 N
U Question is based on a procedure that has since been removed from use. Recommend replacing the question. Agreed. Replaced the question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 71 H
2.5 N
U Question has multiple correct answers because the initial conditions do not suggest that 1R-37 would not be reading the same as 1R-30; therefore, 121 Aux Bldg Special Vent would be expected to be operating. The knowledge tested in the question is true for all NMC Monitors; therefore, independent of the conditions given. Agreed.
Modified the question incorporating suggested changes.
72 H
2.0 N
E Remove ambiguity from the stem and align distracters with procedure requirements of F2, Radiation Safety. Station practice is to check flux mapping secured placards are hanging prior to sending people into containment. Make distractor b. incorrect. Agreed. Reworked question.
73 H
2.0 N
E Question is procedurally accurate, but conflicts with sign postings with the Security Access Facility. Agree. Remixed the answers/distractors.
74 F
2.0 N
E Question is SRO level. Determination of entry conditions based on CSF trees is an SRO only task. Keyed answer is incorrect as 1E-0 Step 1 RNO must also be performed without successfully tripping the reactor before FR-S.1 entry is directed. Recommend replacing the question. Disagree. ROs have to know entry conditions for EOPs. Removed the steps to be executed after entry to the EOP.
75 F
2.0 N
S 76 H
2.5 N
S Re-formatted question. Re-worked stems initial conditions.
77 H
2.0 N
E Not an SRO question. Align distractors to match bank question style.
Agreed. Re-formatted the question and made SRO Only.
78 H
2.5 N
U Question is not SRO Only. Recommend replacing the question.
Agree. Reworked the question.
79 H
2.0 N
U Not an SRO question. Agree. Replaced the question.
80 H
2.5 N
M In order to replace 12 DDCLP with 121 MDCLP, Bus 27 must be supplied by Bus 25. Entry into LCO 3.7.8 Cond A is required at 0500 July 9th. On 0600 July 14th, upon loss of 21 Screenhouse Exhaust Fan, 22 DDCLP is declared inoperable and T.S. LCO 3.0.3 would be entered. 121 MDCLP would be aligned to the 'B' train (Bus 27 supplied from Bus 26) and LCO 3.0.3 would be exited because 121 MDCLP would be credited as the safeguards pump for train 'B'. Tthe question has no correct answer. The only LCO not met would be 3.7.8 Cond A; which would require no further action at this time, as two days are remaining for the action statement. Agreed. Question was modified to incorporate comments.
81 H
2.5 N
U Question is dependent on having memorized Step 4 of ES-0.1 which is not required. With the reference provided this question is a direct lookup. Agreed. Replaced question with a bank question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 82 H
2.5 N
U Question stem lacks sufficient information for an SRO to make a judgment call on where a 'Safe' location would be for the fuel cell.
Question also implies actions will be performed by the SRO in containment and the fuel handling crew, when in reality they would be in the process of evacuating. Agreed. Reworked the question to eliminate judgment call.
83 H
3.0 N
U Recommend replacing the KA for this question. Unable to identify SRO level required knowledge relevant to the KA. Agreed. Replace K/A, wrote new question.
84 H
2.5 N
E Remove window dressing and provide reference necessary to answer question. Agreed. Provided needed reference, removed window dressing.
85 H`
2.0 N
E Remove window dressing, clean up stem and answer/distractors.
Agreed. Re-formatted question.
86 H
2.5 N
U Question requires memorization of a step within a procedure that is not required to be memorized. Agree. Replaced question.
87 H
2.5 N
E Change amps from rising to lowering as would be expected in this condition. Change containment level to expected level for this condition and the nomenclature for the level detectors. Agree.
Incorporated suggested changes.
88 H
2.5 N
E Distractor (c.) is potentially correct as gas binding is a possible outcome and entry into 1C12.1 AOP2 is appropriate with flashing taking place in letdown which would occur when the charging pumps are pumping high temperature water. Recommend changing distractor (c.) and answer (d.). Agreed. Revised distractors.
89 H
2.0 N
E Change answer (d.) to remove implication load sequencing does not occur as it will. Agreed. Re-worked question. Added picture of Caution from the procedure.
90 H
2.0 N
U Original question is duplicate of R16. Agree. Replaced question.
91 H
2.5 N
E Original question requires memorization of a specific procedural step that are not required to be memorized. Add reference and remove window dressing. Disagree. The question tested a concept, not a specific step. Agree - add reference & removed significant window dressing.
92 H
2.5 N
E Change to mode 4 to avoid possible questioning of timing. i.e. LLRT has been performed and is satisfied. Distractor 'B' is questionably correct as without meeting the containment integrity LCO there would be good reason to declare that barrier as non-functional. Replace may/may not squirrel words. Disagree. Reformatted the question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 93 H
2.5 N
E Clean up stem. Add bullet indicating rod was on the bottom of the core as the given indications are accurate for a bowed assembly becoming wedged. In which case 'A' is correct. Reformatted the question to a 1 of 2 taken twice. Removed the need for a bottom of the core location.
94 H
2.5 M
E Question requires memorization of a specific procedural step that are not required to be memorized. Add reference and remove window dressing. Disagree. The question tests concept, not a specific step. Agree - added reference and removed significant window dressing. Reclassified the question to a NEW question.
95 H
2.0 N
E Remove window dressing and clean up stem and answer/distractors.
Agree. Reformatted the question.
96 F
2.0 N
E Reformatted question. Added picture of the T.S. statement.
97 H
2.0 N
S Reformatted question. Removed window dressing.
98 F
2.5 N
E Clarify distracter for R-5 so it is no longer a correct statement.
Agreed. Reformatted the question, fixed distractor with R-5.
99 H
2.0 N
U Distractor (c.) is correct. Stem conditions do not support exceeding 25R. Agreed. Replaced question to eliminate needed assumptions.
100 F
2.0 N
Correct answer is highly dependent on who discovers the discrepancy and at what point. If the RO discovers it any time following performance of immediate actions from memory it is appropriate for them to perform this action. IF the SRO discovers it they should direct the action be taken at any point following being told immediate actions are complete. However management expectation is the read-through be performed prior to the SRO directing any other actions. Agree. Reworked the question to remove ambiguity from the stem.