ML11206A844
| ML11206A844 | |
| Person / Time | |
|---|---|
| Site: | Duane Arnold  |
| Issue date: | 07/08/2011 |
| From: | NRC/RGN-III |
| To: | NextEra Energy Duane Arnold |
| References | |
| 50-331/11-301, ES-401-9 | |
| Download: ML11206A844 (8) | |
Text
ES-401 Written Examination Review Worksheet Rev 0 Form ES-401-9 # questions reviewed in depth.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 1 Y B U NRC: 1 Out of 2 Taken Twice
- Suction Transfer cued by annunciator and Overcurrent implausible.
New question provided
. 2 F 2 Y N S NRC: 1 Out of 2 Taken Twice format 3 H 2 Y N U E NRC: Isolation reset is irrelevant to the stem and 'No Isolation' implausible.
TYPO - MO-48 14A should be MO
-48 41 A. The typo was corrected and a few minor distractor changes were made , making the question satisfactory
. 4 F 3 Y B E NRC: Delete extra "is" in Distractor "A" Fixed 5 H 2 Y B S NRC: 2005 NRC Exam question 6 H 3 Y N S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO
-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES
-401 Section D.2.f.
- 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2 Form ES-401-9 Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 7 H 2 Y N E NRC: "D" distractor implausible.
Distractor replaced 8 H 2 Y B E NRC: 2005 NRC Exam
- 1 Out of 2 Taken Twice, Remaining in SDC is not plausible because of the 'given' Lo
-Lo-Lo level Group 4 isolation.
The plausibility of remaining in SDC was defended, however, the distractors were replaced to enhance the question.
9 F 2 Y B E NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice, No time delay is not plausible / ADS Timers referenced in Q#11
. Distractors replaced with a 5 second time delay
- No time delay was plausible
. The ADS timer overlap with question #11 found not to be an issue. 10 H 2 Y B E NRC: "C" distractor is implausible
. (Actually, the preferred method to provide an alternate Decay Heat Removal is to raise RPV level to between 230
-240 inches and start a Recirc pump, if available, to establish forced circulation flow. Does having both Recirc pumps 'shutdown' as stated in the stem preclude this? What is the correct answer?) Stem and "D" adjusted to clarify the correct answer
. 11 H 2 Y N E NRC: Should use the correct valve nomenclature for PSV-4400-PSV-4406. Valve names corrected. 12 H 2 Y N S 13 F 2 Y N E NRC: Since (2) is always ODD and (4) is always EVEN, these should be moved up into the stem for clarity.
Moved info into the stem
. 14 F 2 ?Y B U E NRC: 1) H ow is K/A m et? (NSSS=NSSSS?) 2) OI reference says "may" but the stem says must
- therefore no correct answer?
(Actually does not address the K/A which references the NSSSS
-Shutoff System i.e. Group 1 Isolation Logic, which has nothing to do with RPS.)
K/A found to be SAT, "MUST" was changed to "MAY" 15 H 2 Y N S NRC: 1 Out of 2 Taken Twice 16 H 2 Y N E NRC: 1) "C(2)" distractor seems implausible
. 2) LOK = H since you have to know failure mode and resulting level control methodology.
(NOTE: reference quotes AOP 518, yet Stem states 'IAW AOP 644.)
Reference to AOP 518 removed and LOK revised
. 17 H 2 Y N U E NRC: 1 Out of 2 Taken Twice - Pumps NOT starting is implausible.
(The stated rational for more ESW flow after the LOOP is wrong. There are not any ESW valves opened for Torus Cooling, but the Control Building Chillers cooling water supply will shift automatically from Well Water to ESW for Seismic component isolation, as well as flow through the EDGs.)
Stated rationale clarifi ed. 18 H 2 Y B S NRC: 200 7 NRC Exam (Are all the Uninterruptable Power System loads - as asked by the Stem
- reenergized once power is transferred to 1Y2?) YES Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 19 H 3 Y B U E NRC: "A" and "C" distractor s implausible
- See distractor "B" explanation
. (Concur. Also 'C' distractor is to 'D' is essentially the same as 'A' is to 'B' and therefore 'C' is implausible and the distractors combined cue the correct answer 'D'.)
Distractors clarified and plausibility was explained adequately.
20 H 2 Y B E NRC: Distractor "C" implausible
( 'C' is not plausible since, as stated in the Stem, the SRMs are being withdrawn per IPOI
-2, using OI 878.1 which states that SRM counts will be maintained between 100 and 100,000 cps until IRMs on range 3 or above, then fully retract.
Couldn't end up with all 4 SRMs reading 90 cps.)
Revised "C
" 21 H 2 Y N E NRC: It's difficult to have refueling in progress with the plant still in Mode 4. Do they mean refueling outage in progress, with fuel moves in the SFP occurring, or should it be Mode 5?. Mode 5 used 22 F 2 Y N E NRC: What is the final APRM configuration?
(A & C paired initially)
(Per OI 878.4, when 'C' is bypassed, then 'B' should also be bypassed. However, since the plant is in Mode 4, none are required to be operable, so 'D' may remain bypassed, IAW approved procedures. The verb 'shall' should not be used in the distractors.)
Changed stem to Mode 1, 90% power.
23 H 2 Y B S Stem clarification made due to licensee validation
. 24 F 2 Y N E S NRC: "D" distractor is implausible
. Satisfactory as written (OE) 25 F 2 Y N E NRC: Use CST (abbreviation
) in distractor "D" for consistency.
Fixed 26 F H 2 Y N E S NRC: LOK = F (memory question), Loss of 1Y21 has NO effect on initiation logic (Is this important for an RO to know?)
. (Perhaps not important to know since there are no RHR related actions given in AOP 317.) Satisfactory as written
- No changes.
27 H 3 Y B E NRC: "A" distractor is implausible
. (Don't normally have significant time delays directly associated with annunciators.)
Suggest a NOT normal distractor for symmetry in the distractors.
"A" enhanc ed 28 H 2 Y B E NRC: 200 7 NRC Exam (Does the 1C06A C
-12 really come in?
) NO , reference to 1C06A C
-12 deleted. 29 F 2 Y N S 30 F 2 Y N E NRC: 1 Out of 2 Taken Twice (the Actual TS requirement is
> 36', not > 36'; Again this is also an example of what ROs should know about TSs.)
> used in all distractors 31 F 2 Y N S NRC: Verify Design Basis knowledge is required for ROs. Yes, but I think 55.41(b)(6) is a better fit. Need to check lesson plans.)
Lesson plans OK, added 55.41(b)(6)
. 32 H 1 Y B U NRC: 1 Out of 2 Taken Twice
- Leaving MSIVs open is implausible.
Revised distractors to fix the question.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 33 H 2 Y N S NRC: 1 Out of 2 Taken Twice
- Do all OI procedure P & Ls need to be known from memory?
YES they do. 34 F 2 Y N E NRC: 1) To more closely meet the K/A, the distractors should reference a procedure. (Ref OI 170)
(Also, is Primary Containment isolation plausible for Fuel Pool exhaust Rad monitor?)
- 2) Adjust distracters C(2) and D(2) to clearly differentiate between the 2 choices (maybe use 'shall' not 'must'.)
Revised distractors to add procedure references and clarify choices.
35 F 2 Y N E NRC: (The Stem should be changed to reference the second part of the correct answer since it is not part of the automatic function
) Added "Fan" to stem to clarify the actions taken 36 H 3 Y B E NRC: The stem is unnecessarily complex
- all distractors contain indication "
5." Removing "5" shortens the list and clarifies the question. A re any additional simplifications possible? (Also should reword Stem to state 'Which set of the following
) Done 37 H 3 Y N E NRC: "A" distractor is implausible.
(Concur, due to part 2 actions.
Also EOP 3 Title is Secondary Containment Control not 'Secondary Control'.)
Revised distractors' procedural reference
- s. 38 F 1 Y B U NRC: 1 Out of 2 Taken Twice
- Lower indicated level is implausible, eliminating three distractors. (Concur; perhaps use 'the same as' vs. 'lower'.) Revised the question and distractors
. 39 F 2 Y B U NRC: Verify this TS knowledge level is required for ROs. (Concur-This appears to be SRO only knowledge requirements since asking for the bases
-need to check licensee lesson plans. Also, the TS requirement is based upon height of water above seated fuel assemblies in the RPV, not the Reactor Vessel Flange
-therefore the Stem is wrong
.) Revised the question and distractors
. 40 H 1 Y N U NRC: 1 Out of 2 Taken Twice
- SHRINK is implausible
. Revised the question and distractors
. 41 H 2 Y N S 42 F 1 Y N U NRC: 1 Out of 2 Taken Twice
- It is obvious that "load" valves would close on a Turbine Trip, changing it to the Turbine Extraction Relay Dump Valve would differentiate between Extraction valves
. Done 43 F 2 Y B U E NRC: LOK = F (memory question), Loss of 1Y23 is not credible because it is an "Uninterruptable" power supply. (Concur. Also Why, assuming no operator action, would MO
-4208 go closed? Not plausible.)
Distractor "D"/ LOK revised and stem clarified 44 H 2 Y B S 45 F 2 Y N S 46 F 2 Y N U E NRC: "Shutdown under all conditions" is the obvious answer
. Enhanced the question and distractors for better focus.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 47 H 2 Y N S NRC: 1 Out of 2 Taken Twice
- There is no Compressor Cooling Water System. (GSW is backup. I also think that the reason that 1K1 starts at 90# is to prevent it starting before the two normal standby air compressors start.)
Actually , th e system does exist. 48 H 2 Y N S NRC: 1 Out of 2 Taken Twice (Are the ROs expected to memorize Reference Use procedure steps for how to maintain Circ Water Pit inventory?)
Yes , satisfactory as written
- no changes needed. 49 H 3 Y N S 50 H 1 2 Y B U E NRC:1) "A" is the obvious answer. No doubt this has been done many times in the simulator and is the MINIMUM required action. 2) LOK = F (memory question)
(2 is True based upon 1.
) Enhance d the question and distractors to clarify the question.
51 F 2 Y N E NRC: 1 Out of 2 Taken Twice (The second part of the correct answer should state 'The consequence of continuing to operate HPCI may be a failure) Enhanced the question and distractors to clarify the question. 52 F 2 Y B E NRC: The 'D' distractor should read 'The 'A' RHR pump trips . . . suction path is sensed . . . valves begin to CLOSE.'
Done 53 H 2 Y N S 54 F 2 Y B E NRC: Uncovering fuel at 87 inches is not credible. Done - fixed. 55 H 3 Y B S 56 H 2 Y N E NRC: "D" should be shortened so it's not the longest answer (- scoop tube locks up.)
(Or possibly make 'B' longer i.e. 'pumps will trip but the RWCU system will NOT isolate.')
RWCU spelled out in "B" 57 F 2 Y B S NRC: Explain why the candidate doesn't know a position other than SHUTDOWN is used
("D" Answer Key)
.Explained adequately.
58 F 3 Y N E NRC: 1) The stem is unnecessarily confusing and should be reworded. DONE 2) Verify that EOP bases are required RO knowledge YES (per Lesson Plan objectives) 59 H 2 Y B E NRC: "B" reads "Possible possible release-"
DONE 60 F 1 2 Y N U NRC: Collection of True/False statements
- UNCLEAR. (What's the point/bottom line/knowledge test?
LOD=1.) Revised question 61 F 2 Y B E NRC: 1) LOK =
F (memory question)
- 2) Verify that TS bases are required RO knowledge YES (per Lesson Plan objectives)
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 62 H 3 Y N U E NRC: The question is unnecessarily complex. This level of detail would not be required from memory
. Revised the question and distractors to clarify the question.
63 H 2 Y N S 64 H 2 Y B E NRC: "A" distractor is implausible
. Minor edits were made to enhance the question. 65 H 2 Y N E NRC: "C" distractor does not need to say "ONLY
." DONE 66 F 1 Y N U NRC: 1 Out of 2 Taken Twice. Both parts are LOD = 1.
Revised the question and distractors to have an acceptable question.
67 F 2 Y N S 68 F 2 Y N E NRC: "D" distractor provides information to help answer the question and is implausible.
"D" revised to address concerns.
69 F 2 Y B E NRC: T h is question tests for what is not allowed. Refocused the question to elicit the correct information.
70 F 2 Y N S 71 F 2 Y N S NRC: 1 Out of 2 Taken Twice.
72 F 2 Y B E NRC: 1) LOK = F 2)"Typical" day unneeded. Changes made.
73 H 2 Y B E NRC: 2005 NRC Exam
- 1 Out of 2 Taken Twice.
Confusing as written
- needs to be rewritten to be one question with a two part answer.
Distractors now restate their assumptions
. 74 F 2 Y B S NRC: Verify this is required RO knowledge.
YES - verified. 75 F 2 Y B S NRC: Verify this is required RO knowledge.
YES - verified. 76 H 2 Y N S 77 H 2 Y B U NRC: 1 Out of 2 Taken Twice.
- 1) 50% chance that "A" RR pump trips, and since Lube Oil Pressure is lowering, it's a good guess. 2) The 2 nd part of the answer doesn't work with distractors "A" or "D" if "A" RR pump either trips or does not trip.
All problems addressed with a revised question
. 78 F 2 Y N E NRC: 1 Out of 2 Taken Twice.
Why would venting the Drywell be plausible?
Plausibility explained and "NSPEO" changed to Operator. 79 H 2 Y N S NRC: Effectively 1 Out of 2 Taken Twice.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 80 H 2 Y N E NRC: Why would entering EOP 1 be plausible?
EOP-1 reference changed to IPOI-5 to enhance the question.
81 H 2 Y N U S NRC: With the EALs provided, would this be a direct lookup
? Why would an ALERT be plausible?
No - Satisfactory as written 82 H 2 Y N U NRC: Distractors "C" and "D" would both be correct since the head would be on and tensioned in Mode 4 New Question provided. 83 H 3 Y N S 84 H 3 Y B U NRC. 1) Why would "No EAL" be plausible?
- 2) Would the EALs provided for Q#8 9 provide useful information for this question?
. Revised the question and distractors to an acceptable question.
85 H 2 Y N E NRC. 1) Is information "deep" in the AOPs required to be known from memory? 2)
The correct answer is also the longest answer
. Revised the question and distractors to improve focus.
86 H 2 Y N E NRC: LOK=H DONE 87 H 2 Y N U E NRC: Distractors "A" and "C" are both implausible
- it is clear that a flow instrument failure requires an AOP entry.
"Failure" term eliminated from the stem for question enhancement.
88 H 2 Y N S Stem reworded for additional clarification 89 H 2 Y N U E NRC: 55.43(5) requires selection of appropriate procedures. EALs provided allow the EAL classification to be looked up
. Changed to 55.43(1). 90 H 2 Y B E NRC: Distractor "A" requires an IPOI
-5 reference.
DONE 91 H 2 Y N S 92 F 2 Y N S 93 H 2 Y B S NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice.
94 H 2 Y N E NRC: Modified 1 Out of 2 Taken Twice. 1) Distractor "C(1)" is implausible. 2) The requirement to be an active SRO is obvious, especially to an SRO applicant. 3) Directly supervising core alterations also seems obvious.
FH SRO became FH Supervisor and is less obvious.
95 F 2 Y B E NRC: 200 7 NRC Exam; : Distractor "B" is implausible DONE 96 H 3 Y B S NRC: Ensure that the supplied reference does not impact any other questions.
DONE 97 F 2 Y B E NRC: LOK = F (memory question)
, DONE Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 98 H 2 Y N E NRC: 1 Out of 2 Taken Twice
- 55.43(2) is more applicable than 55.43(5)since no procedure was selected.
DONE 99 F 2 Y B E NRC: Add "only" to distractors "A," "B," and "D."
DONE 100 F 2 Y B S NOTES 1. Minimal references provided for SRO exam. Verified no overlap to other questions.
- 2. No RO questions identified as requiring references
. 3. Implausible distracters:
6 with two or more implausible distracters; 17 with one implausible distractor (or other enhancement needed) 4. LOD = 1 (all RO), 6