ML22186A110
| ML22186A110 | |
| Person / Time | |
|---|---|
| Site: | Davis Besse  |
| Issue date: | 03/12/2018 |
| From: | Randy Baker NRC/RGN-III/DRS/OLB |
| To: | FirstEnergy Nuclear Operating Co |
| Baker R | |
| Shared Package | |
| ML17068A442 | List: |
| References | |
| Download: ML22186A110 (9) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist>: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only. (K/A and license-level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 3
N S
2 F
2 N
S 3
F 2
N S
4 F
2 N
S 5
H 2
N S
6 F
2 B
E S
NRC: If there were no signs of DH pump cavitation, then one would NOT vent the standby DH pump (and thus distractor D would also NOT be correct). How does one assume that there were signs of pump cavitation based on the running DH pump tripping on instantaneous overcurrent?
Response: Use the bank question as originally written. Question is SAT.
7 H
3 B
E S
NRC: Distractor A is NOT plausible. Suggest changing the distractor to Loss of power to 4160 V bus D1.
Response: Reworded distractor A to conform to the other distractors structure.
Question is SAT.
ES-401 2
Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 2 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 8
H 2
B S
9 F
2 N
S 10 H
2 M
S 11 H
3 N
S 12 F
2 N
S 13 H
3 B
S NRC 2013 Exam 14 H
2 B
S 15 F
3 N
S 16 H
3 B
S NRC 2015 Exam 17 F
2 N
S 18 H
2 N
S 19 H
3 M
S 20 F
2 N
E S
NRC: Need more information to verify that the 2nd part of distractors B & D is correct.
Response: Removed also from distractor B & D. Question is SAT.
21 F
2 N
S 22 H
3 B
S NRC 2015 Exam 23 H
2 M
E S
NRC: Instead of being Modified, this question should be called a Bank question, since there was no pertinent condition in the question stem that was changed and no distractors were changed.
Response: Stem conditions were clarified and all distractors added procedural selection. Question is SAT.
24 H
3 B
S 25 H
3 B
S NRC 2015 Exam 26 F
2 B
S 27 F
2 N
S 28 H
2 B
E S
NRC: In the Explanation/Justification for distractor A being correct, change if CCW and Seal Injection to all RCPs is confirmed to if CCW and Seal Injection to all RCPs is NOT available.
Response: Licensee agrees to update distractor analysis. Question is SAT.
ES-401 3
Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 3 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 29 H
3 X
B S
NRC: 1) With MU32 failed closed, FI MU34 (Train 2 Makeup Flow) should indicate 0 gpm instead of 15 gpm. This would mean that there is no correct answer.
- 2) Distractor A is NOT plausible that MU Pump #1 would have a sheared shaft, since FI MU34 indicates 15 gpm.
Response: Bypass flow around MU32, through FI MU34, is ~ 15 gpm. Question is SAT as written.
30 F
2 N
S 31 F
2 N
S 32 F
2 N
S 33 F
2 N
S 34 H
2 N
S 35 H
2 N
E S
NRC: Editorial: In the question stem, in the 2nd bulleted item for current plant conditions, change the word is to it (This is the lowest that it has been during the transient)
Response: Licensee agreed and changed. Question is SAT.
36 H
3 N
S 37 H
2 N
S NRC: In the distractors, delete the information in 1) after the Yes/No response.
Reason: The additional information after the Yes/No is not required by the question stem, and the additional information could assist the applicant in determining the correct answer.
Response: Licensee validated that the included information enhances plausibility of distractor. Question is SAT.
38 H
2 N
E S
NRC: 1) Distractors C & D are NOT plausible that the CAC fans would remain in fast speed after an SFAS actuation.
- 2) Editorial: In the last sentence in the question stem, add the word be (i.e., the CAC fans will be operating ).
Response: Plausible if the applicant believes the shift from fast speed may not occur until SFAS 3 signal. Added editorial comment. Question is SAT.
39 F
2 N
S 40 H
2 N
S 41 H
3 N
S
ES-401 4
Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 4 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 42 H
3 B
E S
NRC: NRC 2013 Exam. In distractor D (correct answer), change until SGs are at low level limits to until a SG is at low level limits, since either SG (not both SGs) is required to be at low level limits.
Response: Licensee agreed; distractor D was modified. Question is SAT.
43 F
2 N
S 44 F
2 B
S 45 H
2 N
S 46 H
2 B
S NRC: Why is distractor D also not correct, since SFAS Channel 1 is required to actuate to auto start CS Pump 1? Is it because power to SFAS Channel 1 is required to prevent CS Pump 1 from actuating?
Response: Loss of Channel 1 would provide an actuation signal to the other Channels. Question as written is SAT.
47 F
2 M
S NRC: Is there enough information in the stem to have a correct answer?
Response: Yes. Modified the second part of the stem for editorial fix. Question as written is SAT.
48 H
2 X
M S
NRC: Is a KIRK KEY interlock the same as a mechanical knife switch? Is this Bank question really Modified?
Response: No, but there is a mechanical knife switch. Reworded stem to include the Kirk Key Interlock. Question is SAT.
49 H
2 M
B E
S NRC: This is still a Bank question, since there was no pertinent condition in the question stem that was changed and no distractors were changed.
Response: Licensee agreed. Changed to bank. Question is SAT.
50 F
3 X
N U
S NRC: 1) Since Air Receiver 2-2 pressure is 235 psig and only one Air Receiver is required for Operability (per Tech Spec 3.8.3 Bases), in Question 1), it should say that there is enough air capacity for five EDG start attempts (not just one). Although there is only one correct answer, Question 1 is not a correct statement.
Response: Reworded the first part of the stem question to focus on Air Receiver 2-1 pressure from the TS bases. Question is now SAT.
51 F
3 N
E S
NRC: 1) In distractor B (correct answer), change item (1) to say Bus C2 is re-energized from Bus Tie XFMR AC through breaker AACC2. 2) In distractor D, the meaning is NOT clear, since the wording is NOT correct English.
Response: Licensee agreed. Changes incorporated. Question is SAT.
52 F
3 N
S 53 H
3 M
S
ES-401 5
Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 5 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 54 F
3 M
S 55 F
3 N
S 56 H
2 N
S 57 H
3 B
S NRC: NRC 2015 Exam.
58 H
2 B
S NRC 2013 Exam 59 F
3 X
N E
S NRC: Distractors B & D are NOT plausible that the maximum display on the in core T/Cs is only 920ºF. Suggest using 1800ºF instead.
Response: Changed temperature on distractors B & D to 1800F. Question is SAT.
60 H
3 N
S 61 H
2 N
S 62 H
2 B
S NRC 2013 Exam 63 H
2 N
S 64 F
2 N
S 65 F
3 B
S 66 F
3 N
S 67 F
2 N
E S
NRC: Distractor D is NOT plausible that Night Orders could be valid for one fuel cycle. Suggest changing distractor D to say 7 days, and reorder the distractors.
Response: Modified distractors to all reflect days and moved the label (days) to the stem. Question is now SAT.
68 H
2 X
N U
S NRC: Part 1 of distractors C & D is NOT plausible;..., withdrawing Control Rods will normally compensate for fuel depletion (instead of boron dilution).
Response: Reword the stem question to focus on specific time during the fuel cycle.
Question is now SAT.
69 H
2 B
S 70 F
2 N
E S
NRC: Add the word ONLY at the end of distractors A and C Response: Licensee agree. Modified distractors A & C. Question is SAT.
71 H
3 B
S NRC 2013 Exam 72 F
3 N
E S
NRC: In distractors B & D, change item 2) to the hydraulic shock of the transient causing the leak to become worse.
Response: Licensee agreed; incorporated changes. Question is SAT.
ES-401 6
Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 6 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 73 F
2 N
S 74 F
1 B
U S
NRC: LOD=1: For a loss of annunciators, it is pretty clear that the alarm response procedure for Annunciator System Trouble would be required to be referred to.
Response: Reselected a section 4 generic KA. Replacement question is SAT.
75 H
3 B
S 76 H
3 X
B E
S NRC: NRC 2013 Exam 1) Distractors A and C are NOT plausible that one would need to throttle the DH1A and DH1B injection valves, since the question stem states that LPI Train 1 & 2 motor amps are stable. Suggest changing the 2nd part of distractors A & C to say to Stop Containment Spray Pump 2.
- 2) Where does the procedure direct throttling the CS Discharge Valves?
- 3) Need DB-OP-02000 pages to verify that the 2nd part of distractors B & D is correct.
Response: Clarified which actions were in progress in the stem. Question is now SAT.
77 H
3 N
S 78 H
3 N
S 79 F
1 X
N E
S NRC: The 1st part of the question is at the RO level. The 2nd part of the question has a LOD = 1, since the auto start capability of a required Tech Spec component must be Operable for the component to be Operable. This makes the 2nd part of the distractors NOT plausible.
Response: Added conditions to stem to address the swing CCW pump to impact operability. Question is now SAT.
80 H
2 X
N E
S NRC: Question does not closely match the K/A. The stem should state that once the Aux Operator has taken the action to reset the Lockout, what will be the expected plant response.
Response: Reworded the second part of the stem to state that an aux operator resets the lockout. Question is SAT.
81 H
2 N
S 82 H
2 M
S 83 H
2 M
E S
NRC: 1) In the question stem, it provides a reference RA-EP-01500, Emergency Classification EAL Tables. Delete the words EAL Tables, since these words are NOT part of the title of procedure RA-EP-01500.
Response: Licensee corrected titles. Question is SAT.
84 H
2 N
S 85 H
2 N
S
ES-401 7
Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 7 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 86 H
3 X
N U
S NRC: 1) Distractors C & D are NOT plausible (for shutting down an RCP using the general system operating procedure DB-OP-06005) since the stem states that the AOP has been entered. Additionally, since the title of the procedure to shut down an RCP in distractors A & B is Reactor Coolant Pump Shutdown, and is part of the correct answer, this could be considered cueing.
Response: Yes, some Abnormal procedures reference the use of Operating Procedures. Removed the entry into the AOP from the stem and reworded the distractors to from the titles of the procedures. Question is now SAT.
87 F
3 N
E S
NRC: With the references provided, it is not clear if valve CC1328 (CCW to CRD Booster Pump Suction) is required for CCW system operability. Need more information.
Response: Provided additional Initial Conditions. Question is SAT.
88 H
2 N
E S
NRC: In the question stem, to make distractors B &D more plausible, change current plant conditions to indicate 15ºF subcooling margin instead of 7ºF subcooling margin.
Response: The question as written is still adequate. Question is SAT.
89 F
2 N
S 90 H
2 N
S 91 F
2 N
S 92 H
3 N
S 93 F
2 N
E S
NRC: For clarity, in the question stem for the 2nd part of the question, change the addition ____(2)___ to the addition to the Waste Gas Surge Tank ____(2)_____.
Response: Licensee agreed. Question is SAT.
94 H
3 B
S NRC 2015 Exam 95 H
2 B
S NRC 2013 Exam 96 F
2 N
S 97 F
2 N
S 98 H
3 N
S 99 H
3 X
N E
S NRC: In the question stem, add the word NEXT, i.e., will the crew perform NEXT? Reason: As stated in in the Explanation/Justification for distractor C, eventually the SGTR (Section 8.0) will become the priority after SCM is restored and the overcooling event is terminated.
Response: Licensee agreed and added to stem. Question is Sat.
ES-401 8
Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 8 of 9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 100 H
2 M
B E
S NRC: 1) Change from Modified to a Bank question, since there was no pertinent condition in the question stem that was changed. (Either SU7 or SU5.1 is still an NOUE.) (ES 401 D.2.f.) 2) In the question stem, it provides a reference RA-EP-01500, Emergency Classification EAL Tables. Delete the words EAL Tables, since these words are NOT part of the title of procedure RA-EP-01500. (Is the intent to provide all the EAL Tables?)
Response: Licensee agree-bank. Reworded reference title. Question is SAT.
ES-401 Written Examination Review Worksheet Form ES-401-9 Facility: DAVIS-BESSE NUCLEAR POWER STATION Exam Date: FEBRUARY 5 - 16, 2018 Page 9 of 9 RO TOTALS:
B= 24 F= 34 E= 14 Additional Notes:
M= 07 H= 41 U= 03 N= 44 SRO TOTALS:
B= 04 F= 07 E= 09 Additional Notes:
M= 02 H= 18 U= 01 N= 19
GENERAL COMMENT
S:
- 2.
Questions from the previous 2 NRC Exams: __4__(RO)/__1__(SRO)
- 3.
Average difficulty is _ 2.28_ on the RO exam and 2.32 on the SRO exam.
- 4. The answer distribution is:
/
SRO A = 22
/
7 B = 22
/
6 C = 16
/
8 D = 15
/
4