ML13354B989
| ML13354B989 | |
| Person / Time | |
|---|---|
| Site: | Dresden  |
| Issue date: | 11/25/2013 |
| From: | Walton R NRC/RGN-III/DRS/OLB |
| To: | Exelon Generation Co |
| Walton R | |
| Shared Package | |
| ML11355A017 | List: |
| References | |
| 50-237/13-301, 50-249/13-301, ES-401, ES-401-9 | |
| Download: ML13354B989 (8) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 2.0 X
N(B)
U Placing the mode switch in STARTUP/HOT STANDBY would place the plant in a condition not allowed by procedure. On a loss of recirc flow, the operator is required to place the mode switch in SHUTDOWN. Replace/rework the question. RE: Replaced with question from bank.OK 2
F 2.0 x
N U
Question does not match K/A. K/A deals with cause of a partial or complete loss of AC. Question deals with the power supply to Condensate/Condensate Booster pump. Rework Question. RE:
Question reworked.OK 3
H 2.0 N
S 4
H 2.0 x
N U
The K/A refers to a main turbine generator trip which is a relatively fast reactivity insertion. The question involves a slow reactivity insertion. Re-work/replace question. RE: No change. Question asks what happens BEFORE trip, not after trip. What caused trip?
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 5
F 1.0 M
U LOD = 1. RE: Changed question away from decay heat and towards neutron population levels post-scram.OK 6
H(F) 2.0 M
S Not a (H)igher skill question. Two memory items. RE: Changed to memory question.OK 7
F 2.0 X
N U
K/A asks for monitoring loads on a loss of CCW. Question asks what an operator should do with RBCCW on a scram. No monitoring of loads, no loss of RBCCW. RE: Question replaced.OK 8
H(F) 2.0 N
E Not a (H)igher skill question. Two memory items. RE: Question changed to memory. OK 9
F 2.0 N
S 10 F
2.0 N
S 11 H
2.0 N
S 12 F
2.0 x
N U
K/A asks for the reasons for the ARI/RPT/ATWS. Question asks for a setpoint. RE: K/A was previously repicked, but K/A on sheet was not changed. See Q for explanation!! Q not changed. OK 13 H
2.0 x
N U
Is this material required to be memorized? Immediate action?
RE: Q meets K/A (where to read torus temperature.) Operators should know this even though if this is not an immediate action!
14 H
2.5 B
S 15 H
2.0 N
E As written there are arguably 2 correct answers. Ask for the FIRST concern associated with primary containment to eliminate distractor (c.). RE: Added FIRST to question stem. OK 16 H
1.0 N
U LOD =1. Replaced question with bank question. OK 17 F
2.0 N
U Not a higher skill question. Answer must be memorized. Arguably two correct answers. Distractor (a.) can be shown to be correct. RE:
Distractor A replaced. Changed to memory. OK
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 18 H
2,0 N
E The K/A for this question asks about biological effects. The question asks which organ would be affected. This question should be modified to conform more closely comply with the K/A. This is not a higher skill question. RE: Question replaced. Now requires placing in sequence - higher cog. OK 19 H(F) 2.0 N
S Not a higher skill question. Two memorized items put together.
RE: Changed to memory.
20 H(F) 2.0 B
S Not a higher skill question. Two memorized items put together.
RE: Higher cog since 2 swyd voltages require2 different actions.
Was higher cog in last NRC Exam! OK 21 H
2.0 M
S 22 H
2.0 M
S 23 F
2.0 N
S 24 F
2.0 N
S 25 F
2.0 N
S 26 F
2.0 N
S 27 H
2.0 N
S 28 H
2.5 N
S 29 H
2.5 N
U Arguably 2 correct answers. Distractors (a.) and (b.) are subsets of distractor (d.). If (d.) is correct, then (a.) or (b.) is also correct. Add ONLY to distractors (a.) and (b.). OK 30 H
2.0 N
S 31 F
2.0 N
S Doesnt necessarily meet the K/A, but dont know any other way to meet this K/A with LOD >1.0.
32 F
2.0 N
S 33 F
2.0 N
S 34 F
2.0 N
S 35 H
2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 36 H
2.0 M
S 37 F
2.0 N
S 38 F
2.0 N
S 39 F
2.0 N
S 40 H
2.0 N
S Can (d.) be construed as a correct answer? Subset of (c.)? Added ONLY to distractors (b.) and (d.). OK 41 H
2.0 B
S 42 H
2.5 M
S 43 H
2.0 B
S Check to see if this is required knowledge. This is an action from a subsequent step in a procedure.
44 F
2.5 N
S The question is a bit abstract and may cause confusion among candidates as they try to determine the answer. There are many questions for this K/A that would be better questions. Suggest replacing this question.
RESP: Attempted to rewrite the stem to clarify. Basically the question is looking for approvals needed to make changes in non-50.54X situations - a good thing for operators to know!
45 F
2.0 B
S 46 H
2.5 B
S 47 H
2.5 N
S 48 F
2.0 M
S 49 F
2.0 M
S 50 F
2.0 N
S 51 H
2.0 N
U The second half of the question is not needed. If you know the answer to the first part of the question, one does not need to know how to answer the second part of the question. Rework. RE:
Question modified slightly. But operation of the valve going open is different than operation of valve going closed! (Answer is auto open but manual reset to close.)
52 H
2.5 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 53 H
2.5 N
S 54 H
2.5 N
S 55 F
2.0 N
S 56 H
2.0 B
S 57 H
2.0 N
S 58 H
2.0 N
E (d.) will cause (b.) One of the two distractors should be replaced.
RE: Replaced all distractors. OK 59 F
2.0 N
S 60 F
2.0 N
S Do we need the plant conditions to answer this question? RE: I changed the question such that plant conditions are needed to answer the question. Different alarms make distractors more plausible.
61 H
2.5 N
?
Not sure how this meets the K/A. RE: Rewrote question to new K/A since could not write a question to previous K/A.
62 H
2.5 x
N U
Good question. However, the K/A asks for relationship of feedwater and Main generator. The question asks relationship and reactor power. Minor changes to the question to fix. Re: Modified question (changes are highlighted) and repicked K/A to match the question.
63 F
2.0 B
S 64 H
2.0 N
S 65 F
2.5 x
N U
From memory by an RO? SRO task to direct this action. RE: See references section for explanation. OP-DR procedure should be known to ROs.
66 F
2.0 N
S 67 H
2.0 N
S 68 F
2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 69 F
2.0 N
U I dont think we want to ask this from memory. The key sheet is on top of all the print stacks, and I would expect them to look this up when they start reviewing prints. Defer to chief and facility reviewer RE: This is specific knowledge required to be known from memory per the Elec. Print Reading Lesson Plan.
70 H
2.5 N
S 71 H
2.0 B
S 72 F
2.0 x
N U
K/A asks how to use, the question asks what to use. RE: Question was replaced.
73 H
2.0 N
S 74 H
2.0 N
S 75 H.
2.0 N
S 76 H
2.5 X
N S
77 H
2.5 X
B S
78 F
2.0 X
N E
The question is directed to an SRO who needs to get permission from an SRO - needs to be re-worked so he is directing the activities.
For example distractor (a.) would read do not allow operators to perform actions that would violate technical specifications. RE:
Comments incorporated.
79 H
2.0 X
N E
The distractors do not follow the stem with correct English. The applicants can eliminate distractors because of incorrect grammar.
Comment incorporated.
80 F(H) 2.0 X
N U
Distractors (a.) and (b.) are implausible Stem states normal response, so drastic action is not needed. Dont know if anyone would select (c.) as the card may impact the bypass valve controllers.
RE: Q modified, distractor (a) replaced, distractor (B) is correct answer. Distractor (c) would be considered correct if applicant does not recognize ADSVs cycling. The new distractor (a.) is implausible.
81 F
2.0 X
N U
Probably not SRO Only. Looks like system knowledge.
82 H
2.5 X
N S
83 F
2.0 X
N S
84 H
2.0 X
N S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 85 H
2.0 X
N U
Not SRO Only. Needs to be an SRO question. RE: Agreed. This is common knowledge in Mode 1. Changed Q since not common knowledge with Rx in Mode 3!
86 H
3.0 X
N S
87 F
2.0 X
N U
Not SRO Only. System knowledge would provide the correct answer for this question.
88 H
2.0 X
N U
Not SRO Only. RO should know when AGAFs are out of spec.
89 H
2.0 X
N S
90 H(F) 2.0 X
N S
Appears to be done from memory. Will the LOCA impact the operator locally checking RBCCW head tank level?
91 H
2.0 X
N U
Distractor (a.) is a correct answer with the minimum information provided in the stem. Distractors (b.) and (c.) are implausible since the control rods are already at 00. The correct answer is not SRO Only information.
92 H
2.0 X
N U
The first thing that would be done was to enter distractor (c.) until the report comes from the field of high vibrations. (Possible correct answer - appeal) Operators are taught not to rely on automatic functions and will scram the reactor first followed immediately by a turbine trip. (Possible correct answer (b.)) Not an SRO question, ROs are required to know all the entry conditions into off-normal procedures and DEOPs. RE: Agreed. The SRO is NOW required to transition to the required procedure in the correct order, once the gate condition of no MAIN TURB SUPV TROUBLE is verified.
93 H
2.0 X
N U
Depending on where the operability statement is in the TRM, this material is probably RO knowledge. Not SRO Only. RE: Operability AND entry into Condition A is required. To more clearly require SRO knowledge, distractor a was modified to allow Condition B to be selected, and clarified the stem to match DAN XL3 51-13 terminology for the cabinet.
94 H
2.0 X
N U
When provided DEOP-300, this is probably a direct look-up based on recognizing that the area is not above max safe. RE: Replaced question with operator exposure limit question. I have RP-AA-103 if you need it.
95 H
2.0 X
X N
U K/A asks for Knowledge of radiation hazards, question asks for classification of event. RE: Question replaced.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 96 H(F) 2.0 X
N S
Answers are based on memory items. This may not be an SRO Only question. Part (2) is RO knowledge, is Part (1) SRO Only? RE:
Question replaced.
97 H
2.0 X
N E
Distractor (d.) is implausible under the given conditions. Change distractor (d.) to read, Evacuate all sectors to 2 miles, Evacuate 5 miles downwind. RE: Modified Q. Added wind direction and had applicant pick subsectors.
98 H
2.0 X
N S
99 H
2.0 X
N S
The explanation seems to indicate distractor (d.) is the correct answer. It says orange risk is not acceptable on-line and all others are to be done off-line. Is distractor (c.) the correct answer? RE:
Changed distractors, deleted Orange risk to ensure distractors are correct.
100 F
2.0 x
N S
This Form 401-9 was used for internal review of NRC-developed written exam questions.
NRC-written exam questions were subject to a peer review prior to being released to the licensee for comment.
The comments on this 401-9 form reflect internal comments.