ML11206A844

2011 Duane Arnold Energy Center Initial License Examination Written Examination ES-401-9

ML11206A844
Person / Time
Site:	Duane Arnold
Issue date:	07/08/2011
From:	NRC/RGN-III
To:	NextEra Energy Duane Arnold
References
50-331/11-301, ES-401-9
Download: ML11206A844 (8)

Text

ES-401 Written Examination Review Worksheet Rev 0 Form ES-401-9

1. questions reviewed in depth.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - Suction Transfer cued by 1 H 1 Y B U annunciator and Overcurrent implausible. New question provided.

2 F 2 Y N S NRC: 1 Out of 2 Taken Twice format NRC: Isolation reset is irrelevant to the stem and No Isolation implausible. TYPO - MO-4814A should be MO-4841A. The typo 3 H 2 Y N UE was corrected and a few minor distractor changes were made, making the question satisfactory.

4 F 3 Y B E NRC: Delete extra is in Distractor A Fixed 5 H 2 Y B S NRC: 2005 NRC Exam question 6 H 3 Y N S

• Instructions

• [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

• Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

• Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

• Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

• 4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

• 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

• 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

• 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

• 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 7 H 2 Y N E NRC: D distractor implausible. Distractor replaced NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice, Remaining in SDC is not plausible because of the given Lo-Lo-Lo level Group 4 8 H 2 Y B E isolation. The plausibility of remaining in SDC was defended, however, the distractors were replaced to enhance the question.

NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice, No time delay is not plausible / ADS Timers referenced in Q#11. Distractors replaced 9 F 2 Y B E with a 5 second time delay - No time delay was plausible. The ADS timer overlap with question #11 found not to be an issue.

NRC: C distractor is implausible. (Actually, the preferred method to provide an alternate Decay Heat Removal is to raise RPV level to between 230-240 inches and start a Recirc pump, if available, to 10 H 2 Y B E establish forced circulation flow. Does having both Recirc pumps shutdown as stated in the stem preclude this? What is the correct answer?) Stem and D adjusted to clarify the correct answer.

NRC: Should use the correct valve nomenclature for PSV-11 H 2 Y N E 4400PSV-4406. Valve names corrected.

12 H 2 Y N S NRC: Since (2) is always ODD and (4) is always EVEN, these should 13 F 2 Y N E be moved up into the stem for clarity. Moved info into the stem.

NRC: 1) How is K/A met? (NSSS=NSSSS?) 2) OI reference says

?Y may but the stem says must - therefore no correct answer?

14 F 2 B UE (Actually does not address the K/A which references the NSSSS-Shutoff System i.e. Group 1 Isolation Logic, which has nothing to do with RPS.) K/A found to be SAT, MUST was changed to MAY 15 H 2 Y N S NRC: 1 Out of 2 Taken Twice NRC: 1) C(2) distractor seems implausible. 2) LOK = H since you have to know failure mode and resulting level control methodology.

16 H 2 Y N E (NOTE: reference quotes AOP 518, yet Stem states IAW AOP 644.)

Reference to AOP 518 removed and LOK revised.

NRC: 1 Out of 2 Taken Twice - Pumps NOT starting is implausible.

(The stated rational for more ESW flow after the LOOP is wrong.

There are not any ESW valves opened for Torus Cooling, but the 17 H 2 Y N UE Control Building Chillers cooling water supply will shift automatically from Well Water to ESW for Seismic component isolation, as well as flow through the EDGs.) Stated rationale clarified.

NRC: 2007 NRC Exam (Are all the Uninterruptable Power System 18 H 2 Y B S loads - as asked by the Stem - reenergized once power is transferred to 1Y2?) YES

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: A and C distractors implausible - See distractor B explanation. (Concur. Also C distractor is to D is essentially the 19 H 3 Y B UE same as A is to B and therefore C is implausible and the distractors combined cue the correct answer D.) Distractors clarified and plausibility was explained adequately.

NRC: Distractor C implausible ( C is not plausible since, as stated in the Stem, the SRMs are being withdrawn per IPOI-2, using OI 20 H 2 Y B E 878.1 which states that SRM counts will be maintained between 100 and 100,000 cps until IRMs on range 3 or above, then fully retract.

Couldnt end up with all 4 SRMs reading 90 cps.) Revised C NRC: Its difficult to have refueling in progress with the plant still in 21 H 2 Y N E Mode 4. Do they mean refueling outage in progress, with fuel moves in the SFP occurring, or should it be Mode 5?. Mode 5 used NRC: What is the final APRM configuration? (A & C paired initially)

(Per OI 878.4, when C is bypassed, then B should also be bypassed. However, since the plant is in Mode 4, none are required 22 F 2 Y N E to be operable, so D may remain bypassed, IAW approved procedures. The verb shall should not be used in the distractors.)

Changed stem to Mode 1, 90% power.

23 H 2 Y B S Stem clarification made due to licensee validation.

24 F 2 Y N ES NRC: D distractor is implausible. Satisfactory as written (OE) 25 F 2 Y N E NRC: Use CST (abbreviation) in distractor D for consistency. Fixed NRC: LOK = F (memory question), Loss of 1Y21 has NO effect on initiation logic (Is this important for an RO to know?). (Perhaps not 26 FH 2 Y N ES important to know since there are no RHR related actions given in AOP 317.) Satisfactory as written - No changes.

NRC: A distractor is implausible. (Dont normally have significant 27 H 3 Y B E time delays directly associated with annunciators.) Suggest a NOT normal distractor for symmetry in the distractors. A enhanced NRC: 2007 NRC Exam (Does the 1C06A C-12 really come in?) NO, 28 H 2 Y B E reference to 1C06A C-12 deleted.

29 F 2 Y N S NRC: 1 Out of 2 Taken Twice (the Actual TS requirement is > 36, 30 F 2 Y N E not > 36; Again this is also an example of what ROs should know about TSs.) > used in all distractors NRC: Verify Design Basis knowledge is required for ROs. Yes, but I 31 F 2 Y N S think 55.41(b)(6) is a better fit. Need to check lesson plans.) Lesson plans OK, added 55.41(b)(6).

NRC: 1 Out of 2 Taken Twice - Leaving MSIVs open is implausible.

32 H 1 Y B U Revised distractors to fix the question.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - Do all OI procedure P & Ls need to 33 H 2 Y N S be known from memory? YES they do.

NRC: 1) To more closely meet the K/A, the distractors should reference a procedure. (Ref OI 170) (Also, is Primary Containment isolation plausible for Fuel Pool exhaust Rad monitor?) 2) Adjust 34 F 2 Y N E distracters C(2) and D(2) to clearly differentiate between the 2 choices (maybe use shall not must.) Revised distractors to add procedure references and clarify choices.

NRC: (The Stem should be changed to reference the second part of 35 F 2 Y N E the correct answer since it is not part of the automatic function)

Added Fan to stem to clarify the actions taken NRC: The stem is unnecessarily complex - all distractors contain indication 5. Removing 5 shortens the list and clarifies the 36 H 3 Y B E question. Are any additional simplifications possible? (Also should reword Stem to state Which set of the following) Done NRC: A distractor is implausible. (Concur, due to part 2 actions.

37 H 3 Y N E Also EOP 3 Title is Secondary Containment Control not Secondary Control.) Revised distractors procedural references.

NRC: 1 Out of 2 Taken Twice - Lower indicated level is implausible, 38 F 1 Y B U eliminating three distractors. (Concur; perhaps use the same as vs.

lower.) Revised the question and distractors.

NRC: Verify this TS knowledge level is required for ROs. (Concur-This appears to be SRO only knowledge requirements since asking for the bases-need to check licensee lesson plans. Also, the TS 39 F 2 Y B U requirement is based upon height of water above seated fuel assemblies in the RPV, not the Reactor Vessel Flange-therefore the Stem is wrong.) Revised the question and distractors.

NRC: 1 Out of 2 Taken Twice - SHRINK is implausible.

40 H 1 Y N U Revised the question and distractors.

41 H 2 Y N S NRC: 1 Out of 2 Taken Twice - It is obvious that load valves would 42 F 1 Y N U close on a Turbine Trip, changing it to the Turbine Extraction Relay Dump Valve would differentiate between Extraction valves. Done NRC: LOK = F (memory question), Loss of 1Y23 is not credible because it is an Uninterruptable power supply. (Concur. Also Why, 43 F 2 Y B UE assuming no operator action, would MO-4208 go closed? Not plausible.) Distractor D/ LOK revised and stem clarified 44 H 2 Y B S 45 F 2 Y N S NRC: Shutdown under all conditions is the obvious answer.

46 F 2 Y N UE Enhanced the question and distractors for better focus.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - There is no Compressor Cooling Water System. (GSW is backup. I also think that the reason that 1K1 47 H 2 Y N S starts at 90# is to prevent it starting before the two normal standby air compressors start.) Actually, the system does exist.

NRC: 1 Out of 2 Taken Twice (Are the ROs expected to memorize 48 H 2 Y N S Reference Use procedure steps for how to maintain Circ Water Pit inventory?) Yes, satisfactory as written - no changes needed.

49 H 3 Y N S NRC:1) A is the obvious answer. No doubt this has been done many times in the simulator and is the MINIMUM required action.

50 H 12 Y B UE

2) LOK = F (memory question) (2 is True based upon 1.) Enhanced the question and distractors to clarify the question.

NRC: 1 Out of 2 Taken Twice (The second part of the correct answer should state The consequence of continuing to operate HPCI may be 51 F 2 Y N E a failure) Enhanced the question and distractors to clarify the question.

NRC: The D distractor should read The A RHR pump trips . . .

52 F 2 Y B E suction path is sensed . . . valves begin to CLOSE. Done 53 H 2 Y N S 54 F 2 Y B E NRC: Uncovering fuel at 87 inches is not credible. Done - fixed.

55 H 3 Y B S NRC: D should be shortened so its not the longest answer (

56 H 2 Y N E scoop tube locks up.) (Or possibly make B longer i.e. pumps will trip but the RWCU system will NOT isolate.) RWCU spelled out in B NRC: Explain why the candidate doesnt know a position other than 57 F 2 Y B S SHUTDOWN is used (D Answer Key).Explained adequately.

NRC: 1) The stem is unnecessarily confusing and should be 58 F 3 Y N E reworded. DONE 2) Verify that EOP bases are required RO knowledge YES (per Lesson Plan objectives) 59 H 2 Y B E NRC: B reads Possible possible release DONE NRC: Collection of True/False statements - UNCLEAR. (Whats the 60 F 12 Y N U point/bottom line/knowledge test? LOD=1.) Revised question NRC: 1) LOK = F (memory question) 2) Verify that TS bases are 61 F 2 Y B E required RO knowledge YES (per Lesson Plan objectives)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: The question is unnecessarily complex. This level of detail 62 H 3 Y N UE would not be required from memory. Revised the question and distractors to clarify the question.

63 H 2 Y N S NRC: A distractor is implausible. Minor edits were made to 64 H 2 Y B E enhance the question.

65 H 2 Y N E NRC: C distractor does not need to say ONLY. DONE NRC: 1 Out of 2 Taken Twice. Both parts are LOD = 1. Revised the 66 F 1 Y N U question and distractors to have an acceptable question.

67 F 2 Y N S NRC: D distractor provides information to help answer the question 68 F 2 Y N E and is implausible. D revised to address concerns.

NRC: This question tests for what is not allowed. Refocused the 69 F 2 Y B E question to elicit the correct information.

70 F 2 Y N S 71 F 2 Y N S NRC: 1 Out of 2 Taken Twice.

72 F 2 Y B E NRC: 1) LOK = F 2)Typical day unneeded. Changes made.

NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice. Confusing as written 73 H 2 Y B E - needs to be rewritten to be one question with a two part answer.

Distractors now restate their assumptions.

74 F 2 Y B S NRC: Verify this is required RO knowledge. YES - verified.

75 F 2 Y B S NRC: Verify this is required RO knowledge. YES - verified.

76 H 2 Y N S NRC: 1 Out of 2 Taken Twice. 1) 50% chance that A RR pump trips, and since Lube Oil Pressure is lowering, its a good guess. 2) The 77 H 2 Y B U 2nd part of the answer doesnt work with distractors A or D if A RR pump either trips or does not trip. All problems addressed with a revised question.

NRC: 1 Out of 2 Taken Twice. Why would venting the Drywell be 78 F 2 Y N E plausible? Plausibility explained and NSPEO changed to Operator.

79 H 2 Y N S NRC: Effectively 1 Out of 2 Taken Twice.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: Why would entering EOP 1 be plausible? EOP-1 reference 80 H 2 Y N E changed to IPOI-5 to enhance the question.

NRC: With the EALs provided, would this be a direct lookup? Why 81 H 2 Y N US would an ALERT be plausible? No - Satisfactory as written NRC: Distractors C and D would both be correct since the head 82 H 2 Y N U would be on and tensioned in Mode 4 New Question provided.

83 H 3 Y N S NRC. 1) Why would No EAL be plausible? 2) Would the EALs 84 H 3 Y B U provided for Q#89 provide useful information for this question? .

Revised the question and distractors to an acceptable question.

NRC. 1) Is information deep in the AOPs required to be known from 85 H 2 Y N E memory? 2) The correct answer is also the longest answer.

Revised the question and distractors to improve focus.

86 H 2 Y N E NRC: LOK=H DONE NRC: Distractors A and C are both implausible - it is clear that a 87 H 2 Y N UE flow instrument failure requires an AOP entry. Failure term eliminated from the stem for question enhancement.

88 H 2 Y N S Stem reworded for additional clarification NRC: 55.43(5) requires selection of appropriate procedures. EALs 89 H 2 Y N UE provided allow the EAL classification to be looked up. Changed to 55.43(1).

90 H 2 Y B E NRC: Distractor A requires an IPOI-5 reference. DONE 91 H 2 Y N S 92 F 2 Y N S 93 H 2 Y B S NRC: 2005 NRC Exam; 1 Out of 2 Taken Twice.

NRC: Modified 1 Out of 2 Taken Twice. 1) Distractor C(1) is implausible. 2) The requirement to be an active SRO is obvious, 94 H 2 Y N E especially to an SRO applicant. 3) Directly supervising core alterations also seems obvious. FH SRO became FH Supervisor and is less obvious.

95 F 2 Y B E NRC: 2007 NRC Exam; : Distractor B is implausible DONE NRC: Ensure that the supplied reference does not impact any other 96 H 3 Y B S questions. DONE 97 F 2 Y B E NRC: LOK = F (memory question), DONE

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only NRC: 1 Out of 2 Taken Twice - 55.43(2) is more applicable than 98 H 2 Y N E 55.43(5)since no procedure was selected. DONE 99 F 2 Y B E NRC: Add only to distractors A, B, and D. DONE 100 F 2 Y B S NOTES

1. Minimal references provided for SRO exam. Verified no overlap to other questions.

2. No RO questions identified as requiring references.

3. Implausible distracters: 6 with two or more implausible distracters; 17 with one implausible distractor (or other enhancement needed)

4. LOD = 1 (all RO), 6

5. LOK = F vice H, 4

6. LOK = H vice F, 2

7. U (RO) = 8 (10.7%); SRO = 3 (12%)