ML15287A295

2015 Fermi Power Plant Initial License Examination Form ES-401-9

ML15287A295
Person / Time
Site:	Fermi
Issue date:	10/13/2015
From:	Division of Reactor Safety III
To:	Detroit Edison, Co
Shared Package
ML15268A032	List: ML15282A474 ML15287A289 ML15287A293 ML15287A295 ML15287A303 ML15287A334 ML15287A338 ML15348A168
References
Download: ML15287A295 (11)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1# H 2 X N E NRC: 1) LOK F vice H. Answer obtained by eliminating Limiter functions (memory). 2) Choice D includes the resulting core region F

for power/flow which is not part of the other 3 choices. Either include the resulting core region in other 3 choices or eliminate the core region from D and rewrite.

LIC: LOK=F. Removed core region from D. Added Limiter 1.

2 F 3 X N S Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• T/F: The answer choices are a collection of unrelated true/false statements.

• Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• Partial: One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• Backward: The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

1. Seven RO questions and three SRO Only questions submitted for NRC pre-review 1 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 3# H 3 X N E NRC: 1) Clarify what loads for A(1) and A(2), ie, BOP, ESF, both BOP and ESF, or all bus loads.

LIC: Added BOP and ESF to A(1) and A(2).

4+ H 2 X X B U NRC: 1) LOK F vice H. Answer obtained by knowledge (memory).

2) Info from TS basis, SRO Only.

F S LIC: LOK=F. Lesson Plan and Learning Objective references added to justify asking RO question.

NRC: 1) Capitalize or bold Immediately in last stem sentence.

2) Move SRVs to last stem sentence because it is common to all 5 H 2 X B E four choices.

LIC: Capitalized Immediately and moved SRVs to last stem sentence.

6 H 3 X N S 1 of 2 twice 7 H 3 X N S NRC: 1) LOK F vice H. Answer obtained by knowing pressure trip H setpoints for the three valves. 2) Give number to which system 8+ 2 X N U pressure recovers in stem to make choices more believable.

F LIC: LOK=F. Added pressure recovery number in stem.

9 H 3 X B S 10 H 2 X N S 11 F 2 X X X N U 1 of 2 twice S NRC: 1) Part 1 of question is RO, but Part 2 is TS bases which is SRO knowledge and must be known to correctly answer the question. 2) Eliminate best from stem, implies more than one correct answer.

LIC: Removed best from stem. Lesson Plan and Learning Objective references added to justify asking RO question.

2 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 12+ F 2 X X N U NRC: Reference provided is TS 3.3.1.1 bases which makes this an SRO Only question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

R13 F 3 X N S 14 H 3 X  ? N E Handout NRC: 1) Add QR14 to the Handout list for 29.100.01 Sheet 6. EOP curves SRO Only? 2) Revise question: delete ..is the..; delete s on Temperatures and pressures in the stem.

LIC: Removed Sheet 6 reference from question. Deleted is and s as requested. ROs are expected to know Sheet 6 EOP curves.

15 H 3 X  ? N E 1of 2 twice NRC: EOP curve SRO Only?

Lic: ROs are expected to know and use Sheet 6 EOP curves, have access in procedures, computers, and sometimes SRO hands them curves.

16+ H 2 X N E 1 of 2 twice F NRC: 1) LOK F vice H. Answer obtained by knowing (memory) min RPV level and max clad temp.

LIC: LOK=F.

17 H 3 X N S 18 H 2 X X B U NRC: 1) LOK F vice H. Knowledge (memorization) of trip setpoints and auto actions. 2) No SGTS trips identified so SGTS initiation not F S believable, and B. and D. not plausible.

LIC: LOK=F. SGTS will auto start during a high radiation in the RBHVAC exhaust duct. The system response would be a RBHVAC trip at 16,000 cpm with an auto start of both SGTS divisions. The RBHVAC upscale alarm is 12,000 cpm. If applicant mistakenly determines that the RBHVAC trip setpoint has been reached, this would result in RBHVAC trip and SGTS auto start.

19 F 3 N S 3 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 20# H 3 X B U Handout NOTE: Proposed Q20 different from Sample Q20. Kept proposed question.

21+ H 2 X N E NRC: 1) LOK F vice H. Knowledge (memorization) of directed power reduction for decreasing vacuum.

F LIC: LOK=F 22 H 3 X N S 1 of 2 twice 23 H 3 X N S 24+ H 3 X B S 25 H 3 X N S 26 H 3 X X N U NRC: Answer is TS bases which is SRO Only knowledge and must be known to correctly answer the question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

27# H 3 X N S 28+ H 3 X N S 29 H 3 X B S 30 F 3 X N S 31 H 3 X B E NRC: 1) LOK F vice H. Knowledge (memorization) of trip setpoints and auto actions.

F LIC: LOK=F 32+ H 3 X B S 33 H 3 X M S 34 H 3 X N S 1 of 2 twice 35 H 2 X N E NRC: LOK F vice H, knowledge (memorization) of trips/interlocks.

F LIC: LOK=F 4 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 36+ H 2 X B E NRC: LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 37 H 3 X N S 38 H 2 X B E NRC: LOK F vice H, knowledge (memorization) of trips/interlocks.

F LIC: LOK=F 39 H 3 X N S 40+ F 2 X X N U NRC: Reference provided is TS 3.3.6.1 bases which makes this an SRO Only question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

41 H 4 X B E NRC: 1) LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 42 H 3 X N S 43 H 2 X B S NRC: 1) LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 44+ H 4 X B S 45 F 3 X X N U NRC: Reference provided is TS 3.6.1.6 bases which makes this an SRO Only question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

46 H 2 X B S 47 H 3 X M S 48+ F 2 X B S 49 H 3 X N S 50 H 3 X N S 5 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 51 H 2 X N S Handout 52+ H 2 X B S 53 F 2 X B S 54 H 2 X N S 55 H 3 X N S 56+ H 3 X N S 57 H 2 X N E NRC: 1) LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 58 H 3 X B S 59 H 3 X N S 1 of 2 twice 60+ H 3 X N S 61 H 3 X N S 62 H 3 X N S 63# H 3 X B S 64+ F 2 X X N U NRC: Reference provided is TS 3.7.3 bases which makes this an SRO Only question.

S LIC: (RO and SRO) Lesson Plan and plant procedure referenced added to justify asking RO question.

65 H 3 X N S 66 F 2 X N S 67 H 3 X B S 68+ H 3 X N S 69 F 3 X X B S 70 F 2 X N S 6 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 71 F 2 B S 2001 ILE NRC Exam 72+ H 3 X N S 73# H 2 X X B S 2001 ILE NRC Exam 74# F 2 X X N U 1 of 2 twice NRC: 1) The last sentence of stem uses the term normally which indicates that other phone systems may be used. Suggest rephrasing to eliminate term normal. Use something like which of following phone systems are used because page 8 of EP-290, Section 4.7.5 states that HPN phones to be used for radiological, meteorological conditions, and protective measure reports.

2) Last stem sentence, the title of EP-290 is Emergency Notifications which is also the title of the phone to be used in the possible answer and eliminates C. and D. (makes them implausible).

LIC: Replaced normally; rephrased last stem sentence to remove matching terms used in choices A and B.

75 H 2 X B E 2008 ILE NRC Exam F NRC: 1) LOK F vice H. Knowledge (memorization) of trip setpoints and auto actions.

LIC: LOK=F.

R For RO portion of Exam: N= 48/75(64%), B= 25/75(33%), M=2/75(2.7%): LOK: H=45/75(60%); F=30; U=2/75(2.7%); E=13/75(17%) LOD=2.64 76 H 3 X N U NRC: 1) Not SRO Only. System (electrical distribution) knowledge X is RO. RO responsible for procedure entry conditions.

S LIC: The answer is action the SRO would direct from AOP for Loss of 120KV. System knowledge alone will not answer the question because there is more than one way to align power.

Knowledge of the AOP strategy and directing steps is required to correctly answer the question. Requires assessment of plant conditions, then selecting part of procedure with which to th proceed (ref. ES-401, Att.2, 5 box, bullet 1).

7 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only X

77 H 2 X N E Handout: No TS 3.8.4 listed in handout as indicated by question.

NRC: Delete procedure reference from C. It is different format than other choices (ie, they have no reference to procedures)

LIC: Deleted procedure reference from C. Reordered choices.

78 F 2 X X B U Handout; 1 of 2 twice S NRC: 1) Not SRO Only. Part 1 are actions directed from AOP (SRO),

but Part 2 is RO knowledge and is needed to correctly answer the question.

LIC: Both parts are required to answer the question which makes it SRO. NOTE: Switched stem Parts 1) and 2), still same question and answer.

79 F 2 X X N U NRC: 1) Not SRO Only knowledge. ROs responsible for Initial Response (ie, immediate) action(s) of ARP.

S LIC: ARPs have no Immediate Actions for ROs to take at Fermi.

SROs are responsible for directing RO actions and when to take them. The correct steps to take are based on an assessment of plant conditions, system knowledge and mitigating CAUTION in th the AOP procedure (ref. ES-401, Att.2, 5 box, bullet 1).

80 H 2 X X N S 81 H 2 X X B S Handout:

82 H 3 X X N S 83 H 3 X X B S Handout: 2008 ILE NRC Exam 84 H 2 X B U 2008 ILE NRC Exam 1 of 2 twice NRC: 1) Not SRO Only knowledge. Part 1 is RO system knowledge and can be used to answer the question; Part 2 is selection of procedure and mitigating action to take which is SRO. Currently the question can be answered by choosing the correct Part 1. Suggest making choices a 1 of 2 twice format so two Part 1s are correct, and choice would be based on the SRO Part 2.

LIC: Rewrote choices and put them in 1 of 2 twice format.

8 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 85 H 3 X X N U NRC: Add only at end for B. and C. Part 1 otherwise they would be partial answers for C. and D.

LIC: Revised choices format to 1 of 2 twice, and added only to A and B to eliminate partial answers to C and D.

86# H 3 X M U NRC: 1) Part 2 of Q does not address KA for using procedure(s) to address the abnormal consequences.

LIC: Replaced question.

87 H 2 X X N E NRC: Add CRS direct and procedure number to last stem sentence.

LIC: Added verbage and procedure number as requested.

88 H 3 X X N S Handout:

89 H 2 X X N U 1 of 2 twice S NRC: 1) Not SRO Only. Part 1 is integrated plant and system knowledge which is RO knowledge. Part 2 is System Operating Procedure knowledge/actions with is also RO.

th LIC: Meets ES-401, Att.2, 5 box, bullet 1. In addition, the SRO would have to know the flow band required and bases for the flow band upper limit to meet the safety function of the system which is comprised of a high and low band.

90 H 2 X X N S 91 H 2 X N U 1 of 2 twice NRC: 1) Not SRO Only. System knowledge can be used to answer both parts of question.

LIC: Replaced question.

92# H 3 X X M S Handout 93 H 2 X X N U NRC: Choices not a complete selection and application of procedure(s).

E LIC: Added procedure nomenclature to the mitigating actions to th clarify meeting ES-401, Att.2, 5 box, bullet 1.

94 F 2 X X N S 9 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 95 F 2 X X B E 2008 ILE NRC Exam NRC: Add ..only. after choices B. and C. because they are partial answers of D.

LIC: Added only to B and C.

96# H 3 X X B S 2010 ILE NRC Exam 97 H 3 X X M E 98 H 2 X B U Handout S 2008 ILE NRC Exam NRC: 1) Q does not match KA. Q tests ability to estimate core nd rd damage, not how to operate the monitors. 2) Delete 2 and 3 columns of choices, unnecessary to determine correct answer. 3)

Explanations should explain why A, C, and D (numbers) are plausible.

LIC: KA says use not operate. All three columns are required to complete the task. Same question/KA used on previous NRC exam.

99 H 3 X X N U NRC: 1) Not SRO Only. Question can be answered with system knowledge (RO). No decision to select appropriate action/procedure.

2) Stem states RO in act of inserting CRAM array. If rods are being inserted, then power must still be above 65%. Just because Choice D contains a power level below the expected runback power level, doesnt make the answer correct LIC: Replaced question.

10 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 100 H 3 - X X X N E NRC: 1) B and C should be preceded by CONTINUE in 20.000.22 and...; choices A and D, replace PERFORM with TRANSITION to S

and include a mitigation strategy for controlling water level to be consistent with B and C. 2) It is not clear whether the Battery Charger Room is in a 3L Zone. If so, what difference would it make if the fire spreads from the Battery Charger Room? 3) Is the applicant expected to memorize which zones are 3L? 4) In the explanation of B and C, explain how HPCI and RCIC are compromised.

LIC: B and C are fire strategies listed in 20.000.22 and PERFORM cannot be replaced with TRANSITION because it is defined term at FERMI 2. 2) Battery Charger Room is in Zone 11, 3L Zones are 8, 9A, 11, 12, 12A and 16. The DC MCC is adjacent to the Battery Charger Room, so that would make the fire in a 3L Zone.

3) Applicants are expected to know 3L Zones.

For SRO portion of Exam: N=15/25(64%), B=7/25(28%), M=3/25(12%): LOK: H=21/25(84%); F=4/25(16%); U=4/25(16%); LOD=2.44 Were any Bank questions from previous NRC exams? Identified during validation.

Remove handout reference from questions. Done during validation week. Make sure question numbers arent included on the Handout list to applicants during exam.

Submit original questions for Modified questions to verify they were modified IAW ES 1021. Only two questions submitted.

QA sheet listed 59 H (higher level, c/a) type questions for RO which exceeds the 50-60 percent allowed for RO exam.

11 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1# H 2 X N E NRC: 1) LOK F vice H. Answer obtained by eliminating Limiter functions (memory). 2) Choice D includes the resulting core region F

for power/flow which is not part of the other 3 choices. Either include the resulting core region in other 3 choices or eliminate the core region from D and rewrite.

LIC: LOK=F. Removed core region from D. Added Limiter 1.

2 F 3 X N S Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• T/F: The answer choices are a collection of unrelated true/false statements.

• Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• Partial: One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• Backward: The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

1. Seven RO questions and three SRO Only questions submitted for NRC pre-review 1 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 3# H 3 X N E NRC: 1) Clarify what loads for A(1) and A(2), ie, BOP, ESF, both BOP and ESF, or all bus loads.

LIC: Added BOP and ESF to A(1) and A(2).

4+ H 2 X X B U NRC: 1) LOK F vice H. Answer obtained by knowledge (memory).

2) Info from TS basis, SRO Only.

F S LIC: LOK=F. Lesson Plan and Learning Objective references added to justify asking RO question.

NRC: 1) Capitalize or bold Immediately in last stem sentence.

2) Move SRVs to last stem sentence because it is common to all 5 H 2 X B E four choices.

LIC: Capitalized Immediately and moved SRVs to last stem sentence.

6 H 3 X N S 1 of 2 twice 7 H 3 X N S NRC: 1) LOK F vice H. Answer obtained by knowing pressure trip H setpoints for the three valves. 2) Give number to which system 8+ 2 X N U pressure recovers in stem to make choices more believable.

F LIC: LOK=F. Added pressure recovery number in stem.

9 H 3 X B S 10 H 2 X N S 11 F 2 X X X N U 1 of 2 twice S NRC: 1) Part 1 of question is RO, but Part 2 is TS bases which is SRO knowledge and must be known to correctly answer the question. 2) Eliminate best from stem, implies more than one correct answer.

LIC: Removed best from stem. Lesson Plan and Learning Objective references added to justify asking RO question.

2 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 12+ F 2 X X N U NRC: Reference provided is TS 3.3.1.1 bases which makes this an SRO Only question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

R13 F 3 X N S 14 H 3 X  ? N E Handout NRC: 1) Add QR14 to the Handout list for 29.100.01 Sheet 6. EOP curves SRO Only? 2) Revise question: delete ..is the..; delete s on Temperatures and pressures in the stem.

LIC: Removed Sheet 6 reference from question. Deleted is and s as requested. ROs are expected to know Sheet 6 EOP curves.

15 H 3 X  ? N E 1of 2 twice NRC: EOP curve SRO Only?

Lic: ROs are expected to know and use Sheet 6 EOP curves, have access in procedures, computers, and sometimes SRO hands them curves.

16+ H 2 X N E 1 of 2 twice F NRC: 1) LOK F vice H. Answer obtained by knowing (memory) min RPV level and max clad temp.

LIC: LOK=F.

17 H 3 X N S 18 H 2 X X B U NRC: 1) LOK F vice H. Knowledge (memorization) of trip setpoints and auto actions. 2) No SGTS trips identified so SGTS initiation not F S believable, and B. and D. not plausible.

LIC: LOK=F. SGTS will auto start during a high radiation in the RBHVAC exhaust duct. The system response would be a RBHVAC trip at 16,000 cpm with an auto start of both SGTS divisions. The RBHVAC upscale alarm is 12,000 cpm. If applicant mistakenly determines that the RBHVAC trip setpoint has been reached, this would result in RBHVAC trip and SGTS auto start.

19 F 3 N S 3 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 20# H 3 X B U Handout NOTE: Proposed Q20 different from Sample Q20. Kept proposed question.

21+ H 2 X N E NRC: 1) LOK F vice H. Knowledge (memorization) of directed power reduction for decreasing vacuum.

F LIC: LOK=F 22 H 3 X N S 1 of 2 twice 23 H 3 X N S 24+ H 3 X B S 25 H 3 X N S 26 H 3 X X N U NRC: Answer is TS bases which is SRO Only knowledge and must be known to correctly answer the question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

27# H 3 X N S 28+ H 3 X N S 29 H 3 X B S 30 F 3 X N S 31 H 3 X B E NRC: 1) LOK F vice H. Knowledge (memorization) of trip setpoints and auto actions.

F LIC: LOK=F 32+ H 3 X B S 33 H 3 X M S 34 H 3 X N S 1 of 2 twice 35 H 2 X N E NRC: LOK F vice H, knowledge (memorization) of trips/interlocks.

F LIC: LOK=F 4 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 36+ H 2 X B E NRC: LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 37 H 3 X N S 38 H 2 X B E NRC: LOK F vice H, knowledge (memorization) of trips/interlocks.

F LIC: LOK=F 39 H 3 X N S 40+ F 2 X X N U NRC: Reference provided is TS 3.3.6.1 bases which makes this an SRO Only question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

41 H 4 X B E NRC: 1) LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 42 H 3 X N S 43 H 2 X B S NRC: 1) LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 44+ H 4 X B S 45 F 3 X X N U NRC: Reference provided is TS 3.6.1.6 bases which makes this an SRO Only question.

S LIC: Lesson Plan and Learning Objective references added to justify asking RO question.

46 H 2 X B S 47 H 3 X M S 48+ F 2 X B S 49 H 3 X N S 50 H 3 X N S 5 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 51 H 2 X N S Handout 52+ H 2 X B S 53 F 2 X B S 54 H 2 X N S 55 H 3 X N S 56+ H 3 X N S 57 H 2 X N E NRC: 1) LOK F vice H, knowledge (memorization) of power supply.

F LIC: LOK=F 58 H 3 X B S 59 H 3 X N S 1 of 2 twice 60+ H 3 X N S 61 H 3 X N S 62 H 3 X N S 63# H 3 X B S 64+ F 2 X X N U NRC: Reference provided is TS 3.7.3 bases which makes this an SRO Only question.

S LIC: (RO and SRO) Lesson Plan and plant procedure referenced added to justify asking RO question.

65 H 3 X N S 66 F 2 X N S 67 H 3 X B S 68+ H 3 X N S 69 F 3 X X B S 70 F 2 X N S 6 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 71 F 2 B S 2001 ILE NRC Exam 72+ H 3 X N S 73# H 2 X X B S 2001 ILE NRC Exam 74# F 2 X X N U 1 of 2 twice NRC: 1) The last sentence of stem uses the term normally which indicates that other phone systems may be used. Suggest rephrasing to eliminate term normal. Use something like which of following phone systems are used because page 8 of EP-290, Section 4.7.5 states that HPN phones to be used for radiological, meteorological conditions, and protective measure reports.

2) Last stem sentence, the title of EP-290 is Emergency Notifications which is also the title of the phone to be used in the possible answer and eliminates C. and D. (makes them implausible).

LIC: Replaced normally; rephrased last stem sentence to remove matching terms used in choices A and B.

75 H 2 X B E 2008 ILE NRC Exam F NRC: 1) LOK F vice H. Knowledge (memorization) of trip setpoints and auto actions.

LIC: LOK=F.

R For RO portion of Exam: N= 48/75(64%), B= 25/75(33%), M=2/75(2.7%): LOK: H=45/75(60%); F=30; U=2/75(2.7%); E=13/75(17%) LOD=2.64 76 H 3 X N U NRC: 1) Not SRO Only. System (electrical distribution) knowledge X is RO. RO responsible for procedure entry conditions.

S LIC: The answer is action the SRO would direct from AOP for Loss of 120KV. System knowledge alone will not answer the question because there is more than one way to align power.

Knowledge of the AOP strategy and directing steps is required to correctly answer the question. Requires assessment of plant conditions, then selecting part of procedure with which to th proceed (ref. ES-401, Att.2, 5 box, bullet 1).

7 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only X

77 H 2 X N E Handout: No TS 3.8.4 listed in handout as indicated by question.

NRC: Delete procedure reference from C. It is different format than other choices (ie, they have no reference to procedures)

LIC: Deleted procedure reference from C. Reordered choices.

78 F 2 X X B U Handout; 1 of 2 twice S NRC: 1) Not SRO Only. Part 1 are actions directed from AOP (SRO),

but Part 2 is RO knowledge and is needed to correctly answer the question.

LIC: Both parts are required to answer the question which makes it SRO. NOTE: Switched stem Parts 1) and 2), still same question and answer.

79 F 2 X X N U NRC: 1) Not SRO Only knowledge. ROs responsible for Initial Response (ie, immediate) action(s) of ARP.

S LIC: ARPs have no Immediate Actions for ROs to take at Fermi.

SROs are responsible for directing RO actions and when to take them. The correct steps to take are based on an assessment of plant conditions, system knowledge and mitigating CAUTION in th the AOP procedure (ref. ES-401, Att.2, 5 box, bullet 1).

80 H 2 X X N S 81 H 2 X X B S Handout:

82 H 3 X X N S 83 H 3 X X B S Handout: 2008 ILE NRC Exam 84 H 2 X B U 2008 ILE NRC Exam 1 of 2 twice NRC: 1) Not SRO Only knowledge. Part 1 is RO system knowledge and can be used to answer the question; Part 2 is selection of procedure and mitigating action to take which is SRO. Currently the question can be answered by choosing the correct Part 1. Suggest making choices a 1 of 2 twice format so two Part 1s are correct, and choice would be based on the SRO Part 2.

LIC: Rewrote choices and put them in 1 of 2 twice format.

8 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 85 H 3 X X N U NRC: Add only at end for B. and C. Part 1 otherwise they would be partial answers for C. and D.

LIC: Revised choices format to 1 of 2 twice, and added only to A and B to eliminate partial answers to C and D.

86# H 3 X M U NRC: 1) Part 2 of Q does not address KA for using procedure(s) to address the abnormal consequences.

LIC: Replaced question.

87 H 2 X X N E NRC: Add CRS direct and procedure number to last stem sentence.

LIC: Added verbage and procedure number as requested.

88 H 3 X X N S Handout:

89 H 2 X X N U 1 of 2 twice S NRC: 1) Not SRO Only. Part 1 is integrated plant and system knowledge which is RO knowledge. Part 2 is System Operating Procedure knowledge/actions with is also RO.

th LIC: Meets ES-401, Att.2, 5 box, bullet 1. In addition, the SRO would have to know the flow band required and bases for the flow band upper limit to meet the safety function of the system which is comprised of a high and low band.

90 H 2 X X N S 91 H 2 X N U 1 of 2 twice NRC: 1) Not SRO Only. System knowledge can be used to answer both parts of question.

LIC: Replaced question.

92# H 3 X X M S Handout 93 H 2 X X N U NRC: Choices not a complete selection and application of procedure(s).

E LIC: Added procedure nomenclature to the mitigating actions to th clarify meeting ES-401, Att.2, 5 box, bullet 1.

94 F 2 X X N S 9 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 95 F 2 X X B E 2008 ILE NRC Exam NRC: Add ..only. after choices B. and C. because they are partial answers of D.

LIC: Added only to B and C.

96# H 3 X X B S 2010 ILE NRC Exam 97 H 3 X X M E 98 H 2 X B U Handout S 2008 ILE NRC Exam NRC: 1) Q does not match KA. Q tests ability to estimate core nd rd damage, not how to operate the monitors. 2) Delete 2 and 3 columns of choices, unnecessary to determine correct answer. 3)

Explanations should explain why A, C, and D (numbers) are plausible.

LIC: KA says use not operate. All three columns are required to complete the task. Same question/KA used on previous NRC exam.

99 H 3 X X N U NRC: 1) Not SRO Only. Question can be answered with system knowledge (RO). No decision to select appropriate action/procedure.

2) Stem states RO in act of inserting CRAM array. If rods are being inserted, then power must still be above 65%. Just because Choice D contains a power level below the expected runback power level, doesnt make the answer correct LIC: Replaced question.

10 of 11

ES-401 Written Examination Review Worksheet Form ES-401-9, Rev. 10 2015 Fermi Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 100 H 3 - X X X N E NRC: 1) B and C should be preceded by CONTINUE in 20.000.22 and...; choices A and D, replace PERFORM with TRANSITION to S

and include a mitigation strategy for controlling water level to be consistent with B and C. 2) It is not clear whether the Battery Charger Room is in a 3L Zone. If so, what difference would it make if the fire spreads from the Battery Charger Room? 3) Is the applicant expected to memorize which zones are 3L? 4) In the explanation of B and C, explain how HPCI and RCIC are compromised.

LIC: B and C are fire strategies listed in 20.000.22 and PERFORM cannot be replaced with TRANSITION because it is defined term at FERMI 2. 2) Battery Charger Room is in Zone 11, 3L Zones are 8, 9A, 11, 12, 12A and 16. The DC MCC is adjacent to the Battery Charger Room, so that would make the fire in a 3L Zone.

3) Applicants are expected to know 3L Zones.

For SRO portion of Exam: N=15/25(64%), B=7/25(28%), M=3/25(12%): LOK: H=21/25(84%); F=4/25(16%); U=4/25(16%); LOD=2.44 Were any Bank questions from previous NRC exams? Identified during validation.

Remove handout reference from questions. Done during validation week. Make sure question numbers arent included on the Handout list to applicants during exam.

Submit original questions for Modified questions to verify they were modified IAW ES 1021. Only two questions submitted.

QA sheet listed 59 H (higher level, c/a) type questions for RO which exceeds the 50-60 percent allowed for RO exam.

11 of 11