ML13308A636
| ML13308A636 | |
| Person / Time | |
|---|---|
| Site: | Duane Arnold  |
| Issue date: | 04/08/2013 |
| From: | NRC/RGN-III |
| To: | Duane Arnold |
| Randal Baker | |
| Shared Package | |
| ML11354A335 | List: |
| References | |
| Download: ML13308A636 (10) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1+ F 2 X Y N U Why are distractor A & C plausible? Why would an applicant choose
. . . when both valves begin to open . . .?
S 1 out of 2 taken twice.
RESOLUTION: Agreed question was low LOD but credible.
2 F 2 X Y N E Stem potentially cues applicants to focus on interlocks; question should only ask . . . will not open?
S NOTE: Explanation for C distractor discusses Torus spray vice cooling.
RESOLUTION: Change made 3+ F 3 X Y B E NRC: Used on NRC 2011 Exam-Duane Arnold (#4)
S Stem should simply state . . . with power being supplied from the preferred source. vs 1B33 to eliminate potential cueing.
What is the normal position of HSS-3002? If Primary is also normal, then distractor D may not be plausible.
RESOLUTION: Question completely reworded.
4 H 3 Y B E Does this question provide cueing for question #11, or vice versa, concerning the 24 V DC power to the IRMs?
S RESOLUTION: Question #4 was replaced 1 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
2 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Why is distractor B plausible? Why is the second part of distractor C plausible? Should the term spray cooling be replaced with core cooling? (Also replace in other distractors.)
E RESOLUTION: The question was replaced.
5+ H 3 X Y N For the new question consider adding without first depressing the CORE SPRAY INITIATION SEALED-IN reset pushbuttons. To A.
S RESOLUTION: Change made.
Will the rod block cause the withdraw permitted light to go out? If so, then C is not a correct answer. May want to add this condition to the E
6 H 3 X Y N correct answer, and consider adding the word Only to distractor D for S symmetry.
RESOLUTION: Changes made.
Why are distractors A & B plausible? Is there a Slow Transfer associated with any of the transformers?
U 7+ H 3 X Y N Why would an applicant believe a transformer lockout would lockout S an ESF bus?
RESOLUTION: Distractor changed.
If a deluge actuates wouldnt you also dispatch the fire brigade?
E 8 F 3 Y N Should that be added to distractors A & B?
S RESOLUTION: Change made.
The question as written is primarily a fundamentals question, and does not address the KA for how is TURBINE speed measured.
E 9 H 2 N N RESOLUTION: The question was corrected with a simple change to S
refer to turbine rather than pump speed and referring to the speed indicator.
NRC: Used on NRC 2007 Exam-Duane Arnold E Consistent use of capitalization required for the word AND in 10 F 2 Y B*
S distractors C & D RESOLUTION: Done 3 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 11+ H 2 X Y N E Does this question provide cueing for question #4, or vice versa, concerning the 24 V DC power to the IRMs?
S RESOLUTION: Question #4 was replaced 12 H 3 X Y B E Is distractor C plausible? Would this be more plausible (only 2 valves have alternate power) if the 2 valves are both LLS, 4401 & 4407?
S RESOLUTION: Changed distractor C.
13 H 2 Y B S 14+ H 3 Y B* S NRC: Used on NRC 2005 Exam-Monticello 1 out of 2 taken twice 15 H 3 Y B S 1 out of 2 taken twice 16 F 2 Y N S 1 out of 2 taken twice 17 F 2 Y B S 18 F 2 Y B S Not very discriminating in that if the candidate knows that SPDS must always be displayed, the answer is very simple.
19 H 3 Y N S 1 out of 2 taken twice 20+ F 2 Y B* S NRC: Used on NRC 2007 Exam-Susquehanna 1 out of 2 taken twice 21 H 3 Y B* S NRC: Used on NRC 2007 Exam-Duane Arnold 1 out of 2 taken twice 22 F 3 Y N S 23 H 2 Y B* S NRC: Previously Used on an exam? No.
24+ F 3 Y B S 4 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 25 H 2 X Y B E Is Distractor B plausible? Perhaps swap inboard and outboard MSIV between the second parts of distractors A & B.
S 1 out of 2 taken twice?
RESOLUTION: Agreed to accept distracters as plausible without change.
26 F 2 Y B S 1 out of 2 taken twice 27 F 2 Y B S 28+ F 1 Y N U Simple power supply question is considered very low level of difficulty. Is there an alternate power supply to these valves?
S RESOLUTION: Question replaced.
29 H 2 X Y N U/E Is throttling the HX RHR outlet valve plausible; would use of the RHRSW outlet valve be more plausible?
S 1 out of 2 taken twice RESOLUTION: Accepted as is because procedure does discuss throttling the HX RHR inlet valve.
30 F 3 Y B S Typo in explanation of Bank question 31 H 2 Y B S 1 out of 2 taken twice 32 F 2 Y N S 33+ H 2 Y B* S NRC: Used on NRC 2008 Exam-Brunswick 1 out of 2 taken twice 34 H 2 X Y N E Are distractors C & D partly correct? If Offgas isolates, will the conditions given in the stem exist? (Delete < 270 psig to Jet S
Compressor)
RESOLUTION: Jet Compressor pressure in distractors C & D increased to 290 psig.
35+ H 2 X Y B* E NRC: Used on NRC 2005 Exam-Oyster Creek S Why are distractor C & D plausible? (Why ROD DRIFT alarm; or RWM ROD BLOCK if RWM is always bypassed for rod testing?)
RESOLUTION: The question was replaced.
5 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 36 H 3 Y B S 37 F 2 Y N S 38 H 3 Y B* S NRC: Used on NRC 2011 Exam-Pilgrim 39+ F 2 Y B* S NRC: Used on NRC 2010 Exam-Nine Mile Point 2 40 H 3 Y N S The scale of the graph provided in the exam is too small to be accurately read. Enlarge the insert.
41 H 2 N N U Selecting the correct level band appears to be based more upon a concern with subsequent recirc pump operations than it does with S
temperature stratification. Need to show correlation to the KA.
RESOLUTION: Explanation of correlation to the KA provided.
42 H 2 Y N S 1 out of 2 taken twice 43 F 2 Y N S 44+ F 2 X Y N U/E As written, distractors C & D do not appear to be plausible; need to change the description of the portable diesels to match actual S
possibilities. (i.e., using staged portable cables in the East Warehouse.)
1 out of 2 taken twice RESOLUTION: The question was replaced.
45 H 2 Y N S 1 out of 2 taken twice 46 F 2 Y B S This question directly relates to the cautions observed when performing SIM-JPM S-1??
RESOLUTION: The question was replaced.
47 H 2 X Y N S Should this say following a large break inside primary containment.?
RESOLUTION: Stem change to make it clear a RCS break occurred.
48 H 2 Y N S Recommend re-sequencing distractors to B, C, D, A (shortest to longest) and then re-number; this will make the correct proposed answer C.
RESOLUTION: Done 6 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 49+ HF 3 Y N S This appears to be a memory question. Remembers either division of EOC-RPT trips both pumps.
50 F 2 X Y N U C&D not credible. It is not creditable that we would continue to operate a main transformer after it had been on fire.
S RESOLUTION: Question replaced.
51 H 3 Y N S 52 H 4 Y N S 53 H 2 Y N S 54+ FH 3 Y N S This appears to be a higher order question.
55 H 3 Y N S 56 H 3 Y B S 57 H 3 Y N S 58+ F 3 Y N S 59 F 3 Y N S 60 H 3 Y B S 61 H 3 Y N S 62 H 4 Y N S 63+ H 3 Y N S 64 H 4 Y N S 65 H 3 Y N S 66 F 2 Y B S 67+ F 2 Y B S 68 F 3 Y N S 7 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 69 F 3 Y N S 70 F 3 Y N S 71+ F 3 X N N U KA problem, knowing that you can enter the containment with the reactor critical is not a rad safety principal. Knowing Rx power cannot S
be raised during a containment entry would be.
Psychometric flaw with distracter C, entry can be made above 400 psig with plant manager approval, therefore also correct.
RESOLUTION: Question changed.
72 F 3 Y N S 73 F 4 Y N S 74+ H 2 Y N S 75 F 2 Y N S For RO portion of Exam: New = 47Q, Modified = 0Q, Bank = 28Q (37%): LOK: 34/75 = 45%, H= 55% U=6Q = 8%, E=11Q = 15%
76 H 2 Y Y N S 77+ H 1 Y Y N U With the provided references this appears a direct look-up that is to low in difficulty.
S RESOLUTION: Question Changed 78 FH 3 Y Y N S This appears to be a higher order question.
79 H 3 Y Y N S 80+ H 3 Y Y N S 81 H 3 Y Y N S 82 H 3 Y Y N S 83+ H 3 Y Y N S 84 H 3 Y Y N S 85 H 3 Y Y N S 8 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 86+ H 3 Y Y B* S NRC: Used on NRC 2011 Exam-Pilgrim 1 out of 2 taken twice 87 H 2 Y Y N S 88+ H 2 X Y Y N E Why is distractor C plausible? (Keep pressure below MSCP)?
S RESOLUTION: Pressure in stem and distractor reworded.
89 H 2 Y Y B* S NRC: Used on NRC 2008 Exam-Nine Mile Point 2 90+ H 3 X Y Y B* U NRC: Used on NRC 2004 Exam-Nine Mile Point 1 S Re-word distractor B to say continued operability of equipment important to safety . . .?
Are distractors B & D plausible? (Performing an ED to protect the integrity of SECONDARY containment)
RESOLUTION: After further review with author question was accepted as written 91 H 3 Y Y N S 92 H 2 Y Y N E Is distractor D plausible, especially if the candidate is given the TSs?
S RESOLUTION: Replaced distractor D 93 H 3 Y Y N S 94+ F 2 Y N N U Can this question be answered by using only RO knowledge of procedure use and adherence?
S RESOLUTION: Question replaced.
95 F 2 X Y Y B* E NRC: Used on NRC 2011 Exam-Duane Arnold (#95)
S Distractor B is implausible. (See corrected question used in 2011)
RESOLUTION: Question changed.
96+ FH X X Y Y N E This appears to be a higher function question.
S What does entering the spent fuel pool from the reactor cavity really mean? Why is distractor A not a correct answer?
RESOLUTION: Stem changed to make it clear the platform is in the fuel pool.
9 of 10
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 97 F 2 Y N N U Can this question be answered by using only RO knowledge of procedure use and adherence?
S RESOLUTION: Question changed.
98+ H 2 Y Y N E Where is calendar year defined? Is it really 12 consecutive months?
S 1 out of 2 taken twice RESOLUTION: New question written.
99 H 2 Y N N U Can this question be answered by using only RO knowledge of procedures? (AOP immediate actions and IPOI3)
S Very weak KA match but accepted.
RESOLUTION: Question changed to be a direct operations question.
100+ H 3 Y Y B E Restructure distractor B to be consistent with A & C. Distractor C would be more accurate if it stated . . . enter the SAGs and exit the EOPS.
S RESOLUTION: Changes made.
For SRO portion of Exam: New = 20Q, Modified = 0Q, Bank = 5Q (20%): LOK: 3/25 = 12%, H= 88% U=4Q = 16%, E=6Q = 24%
10 of 10