ML11104A069
| ML11104A069 | |
| Person / Time | |
|---|---|
| Site: | Dresden  |
| Issue date: | 03/18/2011 |
| From: | Bielby M Operations Branch III |
| To: | Exelon Generation Co |
| Shared Package | |
| ML11104A071 | List: |
| References | |
| Download: ML11104A069 (8) | |
Text
Page 1 of 8 ES-401 Written Examination Review Worksheet Rev 0 Form ES-401-9
- questions reviewed in depth.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1) C and D not plausible, asking for TAF, but only two TAFs (one above and one below 500psig). Maybe better question to give U conditions, then ask for actions.
1 F 2 B E Licensee: Used on 2008 RIII exam (verified by NRC). Agreed with comment, replaced with acceptable question.
NRC: 1) A is not plausible, ie, would have only Group 1(-54). 2)
DEOP entry condition is +8 and Group 2 and 3 are +6 so applicant should know he gets almost immediate Group 2 and 3 isolations E when hit entry condition.
2 H 2 B Rewrite question to eliminate entry condition and give RWL problem, give level, pressure, etc. and ask what isolations should have occurred. Maybe replace A with Group 2 or Group 3.
Licensee: Comments incorporated.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 2 of 8 ES-401 2 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1) Note: picture shows a switch still in FWD, should it be in REV? 2) B and D implausible. B placing switch in MAN and D placing switch in purge are not believable actions to isolate the TIP 3 H 2 B U tube. May be better to rewrite and give indications and conditions, then ask what actions are taken..
Licensee: Comments incorporated, revised selections to make B and D plausible.
4 F 2 N S NRC: 1) QKA, change in CST level associated with operating HPCI 2) A and B not plausible if never use CST. Is CST ever 5 F 3 B U used? When?
Licensee: Replaced question.
6 F 2 N S 1/2 twice 1/2 twice NRC: 1) B and C implausible, B is implausible because loss of both divisional ATS is always a full scram; C is implausible because loss of one divisional ATS would not cause full scram. 2) 1 LOD=1 because concept of loss of two divisional power supplies is 7 H N U required to cause scram is always true. 3) LOK=F because only 3
asking fundamental knowledge concept of whether loss of one power supply will cause half or full scram.
Licensee: Rewrote question to address comments. LOK=H, LOD=3 NRC: 1) Are operators required to memorize MAX NORMAL RAD U LEVEL setpoints? 2) Why D is plausible (not purple placard)?
8 H 2 B Licensee: 1) MAX NORMAL RAD LEVELS are DEOP 200-1 entry S
conditions, required to be memorized. 2) Recognition of the purple placard is part of the question knowledge being tested.
NRC: Clarify RBCCW L1, LII supply to SDC HX and LIII.
9 F 2 B E Licensee: Identified the three loops to examiners on drawing.
Page 3 of 8
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1) Distractor A, add OFF at end to clarify both systems have tripped off, vice one tripped ON and one tripped OFF. 2) Is 2 minutes sufficient time to allow B flow to increase to 4100 scfm and A flow decrease less than 3500? 3) What is significance of -70 inches RWL? 4) B not plausible. Why is 3500 scfm plausible? 5) A not U plausible. The heater is designated A. It is not plausible for the A 10 H 2 N heater to trip and cause a trip of both the A and B SGTS trains.
S Licensee: 1) Dresden terminology: tripped=OFF, dont use tripped ON. 2) Ops confirmed 2 min is sufficient time. 3) -70 inches would have caused group isolation resulting in trip of PRI SGTS. 4) 3500 scfm is alarm setpoint for SGTS. 5) Not convention at Dresden for A items to be associated with A components.
1/2 twice NRC: 1) Provide reference for RRP stop and RRMG start setpoints based on RR discharge valve position. 2) What is bases for amber light? Blue light?
11 H 2 B E Licensee: 1) Examiners verified RRP and RRMG start setpoints based on RR Discharge Valve position. 2) Blue=normal operation; amber=tripped condition.
NRC: Delete is on stem, 1st line.
12 F 3 B E Licensee: Incorporated.
13 H 3 B S 14 F 3 N S NRC: 1) Why are distracters A, C and D plausible? Base times on 5 Rem (whole body for all), 25 Rem (life saving), and 45 (3 X LDE).
Distractors would be 7.5, 37.5 and 67.5 minutes, respectively. 2)
Are operators expected to know max TEDE for protecting valuable U
15 H 3 M property?
S Licensee: 1) Times were intentionally meant to be in between the allowed times. Revised choices to make them closer to allowed times. 2) Per the learning objective, lesson plan and Operations confirmation, operators are expected to know the max TEDE times.
NRC: How does applicant discount distractor D?
16 H 3 N E Licensee: Per the learning objective and Operations Rep, operators are expected to know when the Torus Temp violates the HCL for the operating conditions given.
1/2 twice NRC: A and B implausible, dont meet laws of physics, ie, 17 H 3 N U pump trip and dP increases.
Licensee: Agreed with comment, replaced the question.
Page 4 of 8
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 18 F 2 M S 19 H 3 N S Reference.
U NRC: SRO only. Execution of the EOPs is SRO function 20 F 2 B Licensee: Per the learning objective, lesson plans and Operations S rep, operators are required to identify reentry condition.
21 F 3 N S 1/2 twice 22 H 3 N S 1/2 twice NRC: 1) For clarification, is there a local synchscope? If not, why is part 2) of distracters B and D plausible? If there is a local U synchscope, D could be considered a correct answer, ie, would still 23 F 2 N accomplish closing the output breaker. 2) What equipment on on S 51i7 level?
Licensee: 1) There is local synchscope, but cant synch from there.
- 2) EDGs.
NRC: Is SOS or SE ever required to be notified of anything?
U Distractors A, B and C are implausible.
24 F 2 N S Licensee: Specific examples of notification of SOS and SE (only) were identified.
25 H 3 B S U NRC: SRO only. Interpretation of TS is SRO function 26 H 2 N Licensee: Not TS interpretation. Knowledge of time limit extension S for LCO is generic knowledge (also above double line for TS).
27 F 3 B S 28 F 3 B S 29 F 3 B S 30 H 3 N S Reference. 1/2 twice 31 H 3 N S 32 F 2 B S 33 F 2 N S 34 H 2 B S 35 F 3 B S 1/2 twice
Page 5 of 8
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: D not plausible, ADS switch in INHIBIT.
36 F 2 N E Licensee: Agreed, revised D to have ADS in Normal.
37 H 3 B S 38 F 3 B S 39 H 2 N S 1/2 twice 1/2 twice 40 F 2 N S 1/2 twice NRC: C and D not plausible, ie, withdrawal of two control U rods.
41 F 2 B S Licensee: Could have instance where one rod withdrawn for testing, when move second rod get rod block.
42 F 3 B S 43 H 3 B S 1/2 twice NRC: 1) LOK=F, Take manual control only requires recall of fundamental knowledge. 2) Stem gives cue to first part of answer 1 (manual) and also makes A and B implausible distractors. LOD=1, 44 H N U not discriminating, Replace or rewrite question.
2 Licensee: Rewrote stem of question to basically remove the term manual and make question higher cognitive level.
45 F 3 N S NRC: C and D are basically the same (both are cross-tie valves that perform same purpose). Replace one.
46 H 3 M E Licensee: Verified that both valves are in separate lines (not same cross-tie line).
NRC: 1) Stem, second sentence, should be known. 2) LOD=1. Not 1 U discriminating, recognition of RWM colors (especially red).
47 F N 2 S Licensee: This has been high miss concept, also 33% miss on validation (4/12). Keep question as is.
48 F 2 B S 49 F 3 B S NRC: C and D implausible. The HPCI Motor Gear Unit is never U used to control RWL.
50 H 2 B S Licensee: The HPCI MGU is used to control RWL. Procedures provided to verify (DGA 2300-03).
Page 6 of 8
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only U NRC: A and B implausible. Why is 7 psid believable?
51 F 2 N S Licensee: Refer to DOP 1500-02.
52 F 3 N S 1/2 twice 53 F 3 N S 54 H 3 B S NRC: 1) SRO only. As written the stem is asking for the US 55 H 3 N E directions. Rewrite so US not giving direction Licensee: Incorporated comment.
NRC: 1) A and B implausible. There is no RRP trip on high RR loop temp.
56 H 3 N U Licensee: Replaced question.
57 H 3 N S NRC: 1) LOD=1, not discriminating. 2) KA? How is second part of KA addressed, ie, and understand how?
1 58 F N U Licensee: Rewrote question, still asking about green board concept, 2 but requiring specific knowledge of concept portion only used at Dresden.
59 H 3 N S NRC: Distractor C, Quality misspelled.
60 H 2 N E Licensee: Incorporated comment.
NRC: 1) Is question N, M or B? 2) D is implausible.
61 H 2 N E Licensee: Revised distractor 62 H 2 N S 63 F 2 N S 1/2 twice Reference. NRC: Rather than provide DEOPs with blacked out entry conditions, provide a set of DEOP curves, let applicants decide 64 H 3 N E which curves to use.
Licensee: Limit DEOP flowchart references to 3, DEOP curves for this question are included on those flowcharts.
65 H 2 N S 66 H 3 N S NRC: D is implausible (ie cross tie Unit 2 SBLC with Unit 3 SBLC),
rewrite.
67 F 2 N E Licensee: Rewrote distractor.
Page 7 of 8
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 68 H 3 N S 69 F 3 N S 70 F 2 N S 71 H 2 N S 1/2 twice NRC: 1) Distractors C and D not plausible. When is paralleling done at Dresden where synchscope rotates in the slow (counter-clockwise) direction? 2) B and D implausible - When does 72 F 2 N U anyone synch the D/G with the pointer horizontal?
Licensee: 1) Dresden does parallel in slow counter-clockwise direction. 2) Rewrote stem and second part of all distracters.
NRC: LOD=1, not discriminating, recognition of alarm setpoint color 1 U on SPDS.
73 H N Licensee: Rewrote stem and changed correct answer. Note:
2 E Question used on RIII NRC Exam 09-1.
74 H 3 N S 75 H 3 B S 76 H 3 B S 77 F 2 B S 78 H 3 B S 79 H 2 B S Reference.
NRC: A and B not plausible, asking for operational requirements.
80 F 3 B U Wiring and breakers are not operational requirements.
Licensee: Rewrote stem to make distracters plausible.
81 H 3 B S 82 F 2 B S Reference NRC: 1) Stem, 4th bullet, replace pressure, which fails to reset.with pressure and fail to reset. 2) Question indicates 83 H 2 B E no reference; however, appears one is needed.
Licensee: Reference required.
Reference NRC: Question indicates no reference; however, 84 H 2 B E appears one is needed.
Licensee: Reference required.
Page 8 of 8
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 85 H 2 B S 1/2 twice NRC: 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> is standard. Maybe better to give in between times 86 F 2 B E and ask for latest time without exceeding required time.
Licensee: Modified stem to incorporate comment.
87 F 2 N S 88 F 2 M S 1/2 twice NRC: 1) Delete is in stem, 1st line.
89 H 3 B E Licensee: Incorporated comment.
90 H 2 N S 1/2 twice 91 H 3 N S 1/2 twice 92 H 2 N S 93 F 2 B S 1/2 twice 94 H 2 N S 1/2 twice NRC: 1) LOD=1, Minimal difficulty and discrimination. 2) LOK=F, 1 U fundamental knowledge of definition of Mode 4.
95 H M 2 S Licensee: This question has been used on more than one occasion.
RIII NRC exam 09-1.
96 H 2 B S Reference. NRC: Clarify answer.
97 H 2 N S Licensee: Licensee used reference to clarify answer.
98 F 2 N S 99 H 3 N S 100 H 3 N S 1/2 twice 1.Single implausible distracter: 5(RO); 0(SRO, 4%) with two or more implausible distracters; 10(RO); 0(SRO) 2.1/2 twice: 15(RO, 20%); 7(SRO, 28%)
3.