ML11206A844
| ML11206A844 | |
| Person / Time | |
|---|---|
| Site: | Duane Arnold  |
| Issue date: | 07/08/2011 |
| From: | NRC/RGN-III |
| To: | NextEra Energy Duane Arnold |
| References | |
| 50-331/11-301, ES-401-9 | |
| Download: ML11206A844 (8) | |
Text
ES-401 Written Examination Review Worksheet Rev 0 Form ES-401-9
- questions reviewed in depth.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - Suction Transfer cued by 1 H 1 Y B U annunciator and Overcurrent implausible. New question provided.
2 F 2 Y N S NRC: 1 Out of 2 Taken Twice format NRC: Isolation reset is irrelevant to the stem and No Isolation implausible. TYPO - MO-4814A should be MO-4841A. The typo 3 H 2 Y N UE was corrected and a few minor distractor changes were made, making the question satisfactory.
4 F 3 Y B E NRC: Delete extra is in Distractor A Fixed 5 H 2 Y B S NRC: 2005 NRC Exam question 6 H 3 Y N S
- Instructions
- [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 7 H 2 Y N E NRC: D distractor implausible. Distractor replaced NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice, Remaining in SDC is not plausible because of the given Lo-Lo-Lo level Group 4 8 H 2 Y B E isolation. The plausibility of remaining in SDC was defended, however, the distractors were replaced to enhance the question.
NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice, No time delay is not plausible / ADS Timers referenced in Q#11. Distractors replaced 9 F 2 Y B E with a 5 second time delay - No time delay was plausible. The ADS timer overlap with question #11 found not to be an issue.
NRC: C distractor is implausible. (Actually, the preferred method to provide an alternate Decay Heat Removal is to raise RPV level to between 230-240 inches and start a Recirc pump, if available, to 10 H 2 Y B E establish forced circulation flow. Does having both Recirc pumps shutdown as stated in the stem preclude this? What is the correct answer?) Stem and D adjusted to clarify the correct answer.
NRC: Should use the correct valve nomenclature for PSV-11 H 2 Y N E 4400PSV-4406. Valve names corrected.
12 H 2 Y N S NRC: Since (2) is always ODD and (4) is always EVEN, these should 13 F 2 Y N E be moved up into the stem for clarity. Moved info into the stem.
NRC: 1) How is K/A met? (NSSS=NSSSS?) 2) OI reference says
?Y may but the stem says must - therefore no correct answer?
14 F 2 B UE (Actually does not address the K/A which references the NSSSS-Shutoff System i.e. Group 1 Isolation Logic, which has nothing to do with RPS.) K/A found to be SAT, MUST was changed to MAY 15 H 2 Y N S NRC: 1 Out of 2 Taken Twice NRC: 1) C(2) distractor seems implausible. 2) LOK = H since you have to know failure mode and resulting level control methodology.
16 H 2 Y N E (NOTE: reference quotes AOP 518, yet Stem states IAW AOP 644.)
Reference to AOP 518 removed and LOK revised.
NRC: 1 Out of 2 Taken Twice - Pumps NOT starting is implausible.
(The stated rational for more ESW flow after the LOOP is wrong.
There are not any ESW valves opened for Torus Cooling, but the 17 H 2 Y N UE Control Building Chillers cooling water supply will shift automatically from Well Water to ESW for Seismic component isolation, as well as flow through the EDGs.) Stated rationale clarified.
NRC: 2007 NRC Exam (Are all the Uninterruptable Power System 18 H 2 Y B S loads - as asked by the Stem - reenergized once power is transferred to 1Y2?) YES
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: A and C distractors implausible - See distractor B explanation. (Concur. Also C distractor is to D is essentially the 19 H 3 Y B UE same as A is to B and therefore C is implausible and the distractors combined cue the correct answer D.) Distractors clarified and plausibility was explained adequately.
NRC: Distractor C implausible ( C is not plausible since, as stated in the Stem, the SRMs are being withdrawn per IPOI-2, using OI 20 H 2 Y B E 878.1 which states that SRM counts will be maintained between 100 and 100,000 cps until IRMs on range 3 or above, then fully retract.
Couldnt end up with all 4 SRMs reading 90 cps.) Revised C NRC: Its difficult to have refueling in progress with the plant still in 21 H 2 Y N E Mode 4. Do they mean refueling outage in progress, with fuel moves in the SFP occurring, or should it be Mode 5?. Mode 5 used NRC: What is the final APRM configuration? (A & C paired initially)
(Per OI 878.4, when C is bypassed, then B should also be bypassed. However, since the plant is in Mode 4, none are required 22 F 2 Y N E to be operable, so D may remain bypassed, IAW approved procedures. The verb shall should not be used in the distractors.)
Changed stem to Mode 1, 90% power.
23 H 2 Y B S Stem clarification made due to licensee validation.
24 F 2 Y N ES NRC: D distractor is implausible. Satisfactory as written (OE) 25 F 2 Y N E NRC: Use CST (abbreviation) in distractor D for consistency. Fixed NRC: LOK = F (memory question), Loss of 1Y21 has NO effect on initiation logic (Is this important for an RO to know?). (Perhaps not 26 FH 2 Y N ES important to know since there are no RHR related actions given in AOP 317.) Satisfactory as written - No changes.
NRC: A distractor is implausible. (Dont normally have significant 27 H 3 Y B E time delays directly associated with annunciators.) Suggest a NOT normal distractor for symmetry in the distractors. A enhanced NRC: 2007 NRC Exam (Does the 1C06A C-12 really come in?) NO, 28 H 2 Y B E reference to 1C06A C-12 deleted.
29 F 2 Y N S NRC: 1 Out of 2 Taken Twice (the Actual TS requirement is > 36, 30 F 2 Y N E not > 36; Again this is also an example of what ROs should know about TSs.) > used in all distractors NRC: Verify Design Basis knowledge is required for ROs. Yes, but I 31 F 2 Y N S think 55.41(b)(6) is a better fit. Need to check lesson plans.) Lesson plans OK, added 55.41(b)(6).
NRC: 1 Out of 2 Taken Twice - Leaving MSIVs open is implausible.
32 H 1 Y B U Revised distractors to fix the question.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - Do all OI procedure P & Ls need to 33 H 2 Y N S be known from memory? YES they do.
NRC: 1) To more closely meet the K/A, the distractors should reference a procedure. (Ref OI 170) (Also, is Primary Containment isolation plausible for Fuel Pool exhaust Rad monitor?) 2) Adjust 34 F 2 Y N E distracters C(2) and D(2) to clearly differentiate between the 2 choices (maybe use shall not must.) Revised distractors to add procedure references and clarify choices.
NRC: (The Stem should be changed to reference the second part of 35 F 2 Y N E the correct answer since it is not part of the automatic function)
Added Fan to stem to clarify the actions taken NRC: The stem is unnecessarily complex - all distractors contain indication 5. Removing 5 shortens the list and clarifies the 36 H 3 Y B E question. Are any additional simplifications possible? (Also should reword Stem to state Which set of the following) Done NRC: A distractor is implausible. (Concur, due to part 2 actions.
37 H 3 Y N E Also EOP 3 Title is Secondary Containment Control not Secondary Control.) Revised distractors procedural references.
NRC: 1 Out of 2 Taken Twice - Lower indicated level is implausible, 38 F 1 Y B U eliminating three distractors. (Concur; perhaps use the same as vs.
lower.) Revised the question and distractors.
NRC: Verify this TS knowledge level is required for ROs. (Concur-This appears to be SRO only knowledge requirements since asking for the bases-need to check licensee lesson plans. Also, the TS 39 F 2 Y B U requirement is based upon height of water above seated fuel assemblies in the RPV, not the Reactor Vessel Flange-therefore the Stem is wrong.) Revised the question and distractors.
NRC: 1 Out of 2 Taken Twice - SHRINK is implausible.
40 H 1 Y N U Revised the question and distractors.
41 H 2 Y N S NRC: 1 Out of 2 Taken Twice - It is obvious that load valves would 42 F 1 Y N U close on a Turbine Trip, changing it to the Turbine Extraction Relay Dump Valve would differentiate between Extraction valves. Done NRC: LOK = F (memory question), Loss of 1Y23 is not credible because it is an Uninterruptable power supply. (Concur. Also Why, 43 F 2 Y B UE assuming no operator action, would MO-4208 go closed? Not plausible.) Distractor D/ LOK revised and stem clarified 44 H 2 Y B S 45 F 2 Y N S NRC: Shutdown under all conditions is the obvious answer.
46 F 2 Y N UE Enhanced the question and distractors for better focus.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - There is no Compressor Cooling Water System. (GSW is backup. I also think that the reason that 1K1 47 H 2 Y N S starts at 90# is to prevent it starting before the two normal standby air compressors start.) Actually, the system does exist.
NRC: 1 Out of 2 Taken Twice (Are the ROs expected to memorize 48 H 2 Y N S Reference Use procedure steps for how to maintain Circ Water Pit inventory?) Yes, satisfactory as written - no changes needed.
49 H 3 Y N S NRC:1) A is the obvious answer. No doubt this has been done many times in the simulator and is the MINIMUM required action.
50 H 12 Y B UE
- 2) LOK = F (memory question) (2 is True based upon 1.) Enhanced the question and distractors to clarify the question.
NRC: 1 Out of 2 Taken Twice (The second part of the correct answer should state The consequence of continuing to operate HPCI may be 51 F 2 Y N E a failure) Enhanced the question and distractors to clarify the question.
NRC: The D distractor should read The A RHR pump trips . . .
52 F 2 Y B E suction path is sensed . . . valves begin to CLOSE. Done 53 H 2 Y N S 54 F 2 Y B E NRC: Uncovering fuel at 87 inches is not credible. Done - fixed.
55 H 3 Y B S NRC: D should be shortened so its not the longest answer (
56 H 2 Y N E scoop tube locks up.) (Or possibly make B longer i.e. pumps will trip but the RWCU system will NOT isolate.) RWCU spelled out in B NRC: Explain why the candidate doesnt know a position other than 57 F 2 Y B S SHUTDOWN is used (D Answer Key).Explained adequately.
NRC: 1) The stem is unnecessarily confusing and should be 58 F 3 Y N E reworded. DONE 2) Verify that EOP bases are required RO knowledge YES (per Lesson Plan objectives) 59 H 2 Y B E NRC: B reads Possible possible release DONE NRC: Collection of True/False statements - UNCLEAR. (Whats the 60 F 12 Y N U point/bottom line/knowledge test? LOD=1.) Revised question NRC: 1) LOK = F (memory question) 2) Verify that TS bases are 61 F 2 Y B E required RO knowledge YES (per Lesson Plan objectives)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: The question is unnecessarily complex. This level of detail 62 H 3 Y N UE would not be required from memory. Revised the question and distractors to clarify the question.
63 H 2 Y N S NRC: A distractor is implausible. Minor edits were made to 64 H 2 Y B E enhance the question.
65 H 2 Y N E NRC: C distractor does not need to say ONLY. DONE NRC: 1 Out of 2 Taken Twice. Both parts are LOD = 1. Revised the 66 F 1 Y N U question and distractors to have an acceptable question.
67 F 2 Y N S NRC: D distractor provides information to help answer the question 68 F 2 Y N E and is implausible. D revised to address concerns.
NRC: This question tests for what is not allowed. Refocused the 69 F 2 Y B E question to elicit the correct information.
70 F 2 Y N S 71 F 2 Y N S NRC: 1 Out of 2 Taken Twice.
72 F 2 Y B E NRC: 1) LOK = F 2)Typical day unneeded. Changes made.
NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice. Confusing as written 73 H 2 Y B E - needs to be rewritten to be one question with a two part answer.
Distractors now restate their assumptions.
74 F 2 Y B S NRC: Verify this is required RO knowledge. YES - verified.
75 F 2 Y B S NRC: Verify this is required RO knowledge. YES - verified.
76 H 2 Y N S NRC: 1 Out of 2 Taken Twice. 1) 50% chance that A RR pump trips, and since Lube Oil Pressure is lowering, its a good guess. 2) The 77 H 2 Y B U 2nd part of the answer doesnt work with distractors A or D if A RR pump either trips or does not trip. All problems addressed with a revised question.
NRC: 1 Out of 2 Taken Twice. Why would venting the Drywell be 78 F 2 Y N E plausible? Plausibility explained and NSPEO changed to Operator.
79 H 2 Y N S NRC: Effectively 1 Out of 2 Taken Twice.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: Why would entering EOP 1 be plausible? EOP-1 reference 80 H 2 Y N E changed to IPOI-5 to enhance the question.
NRC: With the EALs provided, would this be a direct lookup? Why 81 H 2 Y N US would an ALERT be plausible? No - Satisfactory as written NRC: Distractors C and D would both be correct since the head 82 H 2 Y N U would be on and tensioned in Mode 4 New Question provided.
83 H 3 Y N S NRC. 1) Why would No EAL be plausible? 2) Would the EALs 84 H 3 Y B U provided for Q#89 provide useful information for this question? .
Revised the question and distractors to an acceptable question.
NRC. 1) Is information deep in the AOPs required to be known from 85 H 2 Y N E memory? 2) The correct answer is also the longest answer.
Revised the question and distractors to improve focus.
86 H 2 Y N E NRC: LOK=H DONE NRC: Distractors A and C are both implausible - it is clear that a 87 H 2 Y N UE flow instrument failure requires an AOP entry. Failure term eliminated from the stem for question enhancement.
88 H 2 Y N S Stem reworded for additional clarification NRC: 55.43(5) requires selection of appropriate procedures. EALs 89 H 2 Y N UE provided allow the EAL classification to be looked up. Changed to 55.43(1).
90 H 2 Y B E NRC: Distractor A requires an IPOI-5 reference. DONE 91 H 2 Y N S 92 F 2 Y N S 93 H 2 Y B S NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice.
NRC: Modified 1 Out of 2 Taken Twice. 1) Distractor C(1) is implausible. 2) The requirement to be an active SRO is obvious, 94 H 2 Y N E especially to an SRO applicant. 3) Directly supervising core alterations also seems obvious. FH SRO became FH Supervisor and is less obvious.
95 F 2 Y B E NRC: 2007 NRC Exam; : Distractor B is implausible DONE NRC: Ensure that the supplied reference does not impact any other 96 H 3 Y B S questions. DONE 97 F 2 Y B E NRC: LOK = F (memory question), DONE
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - 55.43(2) is more applicable than 98 H 2 Y N E 55.43(5)since no procedure was selected. DONE 99 F 2 Y B E NRC: Add only to distractors A, B, and D. DONE 100 F 2 Y B S NOTES
- 1. Minimal references provided for SRO exam. Verified no overlap to other questions.
- 2. No RO questions identified as requiring references.
- 3. Implausible distracters: 6 with two or more implausible distracters; 17 with one implausible distractor (or other enhancement needed)
- 5. LOK = F vice H, 4
- 6. LOK = H vice F, 2