ML24045A084
| ML24045A084 | |
| Person / Time | |
|---|---|
| Site: | Davis Besse  |
| Issue date: | 01/29/2024 |
| From: | NRC/RGN-III/DORS/OB |
| To: | Energy Harbor Nuclear Corp |
| Greg Roach | |
| Shared Package | |
| ML22251A151 | List: |
| References | |
| L-24-029 | |
| Download: ML24045A084 (1) | |
Text
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution 1
H 2
M S
DB 2009 Q51 Add and slowly rising to RCS temp Add and slowly rising to SG2 pressure for proof no overcooling event in progress Distractor C reword to BOTH SGs 2
H 2
N S
Replaced question that is focused on diagnosis of event.
Swap around valve designator and nomenclature to align with remainder of the exam.
Re-arranged valve and descriptor to align with remainder of the exam.
3 F
3 M
S DB 2015 Q4 Modified question to eliminate the 2x2 format.
Modified Distractor D due to this also being a correct answer.
Modified question to change the initial conditions due to overlap from a scenario.
4 F
2 M
S TMI 2015 Q10 Reword Distractor C Changed lower bearing temp to 195F 5
H 2
N S
Added, All immediate actions have been completed to the stem.
Reworded Distractors B and corrected C to focus away from the EDG as it will still need to be shutdown REPLACED question to address overlap of a Bus C1 failure.
Added procedure the crew would be in given the conditions of the question.
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution 6
H 3
N S
Reworded stem regarding DH14 Replaced Distractor A as action to suspend refueling ops would probably be done to focus on the transient.
Removed 2nd reference to Steam Generator in the answer 7
F 3
B S
DB 2016 Q6 Re-wrote question to focus on plant integrated response (Tier 1)
Removed SW1434 from stem of question.
Added Level to SFAS 2 in question 8
F 3
N S
Editorial changes to question stem REPLACED Distractor B.
9 F
2 N
S K/A change Removed stem statement about control rods remaining in same location.
10 H
4 B
S DB 2016 Q9 Changed order of SGTR and LOOP 11 H
4 N
S Added, No personnel are in CTMT to the stem Choice C is correct answer. Adjusted analysis to correct for choices C and D.
Spelled out Control Room throughout question, capitalized SG Tube Rupture in Distractor A
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution 12 H
2 N
S Adjusted how FW isolation valves are written and removed start the from each answer choice and moved to the stem.
Minor editorial changes 13 F
2 N
S Reworded the stem to indicate an ELAP condition.
Added info to distractor analysis for Choice A.
Added note in stem that NO RCS leak exists and changed Distractor A to ensure that is absolutely wrong while remaining plausible.
Minor editorial changes 14 F
3 M
S DB 2020 Q57 Replaced original question because power supply for FW flow indication can be considered minutia knowledge.
Reworded the Stem Question to align to make a Tier 1 question.
15 H
3 N
S Stem adjusted. Distractors A and C changed, and answer B justification modified to include additional information.
Distractor D changed to be more like answer B.
Modified to eliminate 2x2 format Added to stem that the reactor is scrammed and how would the crew maintain level in SG1 Eliminated 2x2 format for this question.
16 H
3 N
S Made a 2x2 question. Reworded the stem.
Minor editorial changes 17 H
3 B
S DB 2016 Q14 Reworded Answer A and Distractor B for clarity and changed rupture to leak in the stem.
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution 18 H
3 N
S Stem editorial changes. Reworded Distractor C.
Reworded distractors to add procedure section references.
Added Both MUPs are in service 19 H
2 M
S DB 2022 Q82 Added time frame to stem.
Modified question to eliminate scenario overlap.
Modified question to more align with Tier 1 and modified distractors for clarification based on updated data from the plant reference simulator.
20 H
3 M
S DB 2015 Q19 Changed initial power level to 75%. Editorial changes to stem.
Added diagnosis of stuck rod to stem to remove misaligned rod confusion.
Added a in front of the 7% in the explanation.
Minor editorial changes 21 F
4 N
S Rewrote question to eliminate 2x2 format due to SRO level of knowledge and K/A concern.
Added procedure name to stem.
22 H
3 N
S Reworded stem to avoid overlap with Q14.
Changed Distractor A to eliminate subset with answer B. Changed Distractor D to comply with stem changes.
Changed distractor D to 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> Changed KA to 2.4.31 Knowledge of annunciator alarms, indications, or response procedures.
23 H
3 N
S Added condensate flow and S/G level info to stem to ensure Distractors B and C incorrect.
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution Typo in stem fixed to 320 Mwe.
Changed K/A to AA1.08 Ability to operate and/or monitor the following as they apply to PLANT RUNBACK: ICS 24 F
3 B
S Changed K/A and question as a loss of NNIX does NOT affect the MFW system.
PREVIOUS 2 EXAMS: DB 2020 Q22 Minor editorial changes 25 H
3 N
S Reworded question for better KA match and clarity of stem.
26 H
3 N
S Reworded stem and distractors to remove the 2x2 format and minimize overlap.
Added be fed to answer choices 27 F
2 N
S Changed Distractor A to make to better focus on RO level of knowledge.
Minor editorial changes 28 H
3 N
S Changed RCP lineup and second portion of 2x2 for better operational validity.
29 H
3 N
S Reworded for operational validity.
30 H
2 N
S Reworded stem to eliminate subjective words.
31 F
2 N
S Minor editorial changes 32 F
2 N
S Reworded answer and distractors to eliminate similarities between answer choices A and D.
Moved The QT Circ Pump _____ to the Stem of the question
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution Reworded distractors for clarity. I found the following paragraph in SD-039B, The Quench Tank Circulating Pump is a centrifugal pump used to circulate water between the Pressurizer Quench Tank and the Quench Tank Cooler and for pumping water out of the Quench Tank to the RC drain tank. This is not the normal method for lowering the QT level. The changes to the distractors make them undisputable 33 F
2 N
S 34 H
3 B
S Replaced K/A and question since air operated CCW TCVs do NOT exist at Davis Besse.
DB 2016 Q35 35 F
3 B
S Replaced question to avoid oversampling on NNI system based questions.
DB 2013 Q36 36 H
2 N
S Reworded Distractor A to eliminate partial correct answer concern.
37 F
2 N
S Enhanced Distractors.
Reworded Distractor C to remove the However, to be more consistent with the other choices Reworded Distracter A to align with plant nomenclature.
38 H
3 N
S Minor editorial changes 39 F
2 N
S 40 F
2 N
S Reworded stem and turned question into 2x2 for a better KA match.
41 H
2 B
S DB 2011 Q44
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution 42 F
4 N
S Changed from a 2x2 to a standard style question.
43 F
2 M
S DB 2008 Q61 Power changed to 79%.
44 H
2 B
S DB 2016 Q47 Added to Distractor A that SFRCS Initiation ONLY 45 H
2 B
S DB 2004 Q47 Distractor analyses corrected for SFRCS stpt.
46 H
2 M
S DB 2008 Q24 Adjusted suction pressure. Reworded answer choices and analyses.
Adjusted stem of the question to better align with K/A.
47 H
3 B
S Oconee 2018 Q49 Added info to the distractor analysis.
48 H
3 N
S 49 H
2 N
S Changed Distractors C and D to make them more plausible.
50 F
3 N
S Changed distractors to improve the LOD of the question. Distractor Analyses have been adjusted accordingly. Answer was correctly adjusted to C.
51 H
2 N
S Adjusted PSL pressure to correct for setpoint and removed unnecessary why in each answer choice.
52 F
2 N
S Adjusted malfunction to SAC 2 running unloaded.
Adjusted question format to 2x2.
Rewrote question and answer choices again to improve clarity of wording based on validation comments. Removed 2x2 format.
Changed the stem to remove SAC 1 from service instead of stating in the question 'if SAC 1 fails to auto start' 53 H
3 N
S Changed stem malfunction to loss of ALL air.
54 H
3 N
S Changed malfunction to mechanical penetration failure to remove overlap and account for system design.
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution Changed Distractor D to Fuel Handling Ventilation.
55 H
4 B
S PREVIOUS 2 EXAMS DB 2022 Q4 56 H
3 N
S Replaced question due to address technical errors and potential overlap covering loss of YBU.
57 H
3 N
S Editorial changes to stem and replaced Distractor B for better plausibility.
58 H
3 N
S Replaced question with a new one to minimize overlap with Q91.
Minor editorial changes 59 H
3 N
S Made question into a 2x2 format as original question could be answered by solely knowing what level SFP level lowered to.
Minor editorial changes 60 F
2 B
S DB 2011 Q62 Made train specific 61 H
2 N
S Editorial changes to stem.
Changed Distractor B and answer choice D.
Minor editorial changes 62 H
2 N
S Editorial changes and simplified wording for answer choices Reworded answer choices.
63 H
2 N
S Modified stem to align with Main Condenser (pressure rising vice vacuum degradaing) 64 F
2 N
S 65 F
2 B
S DB 2018 Q67 66 F
2 M
S DB 2011 Q68 Changed the stem from a NOT to a May and changed the answer to Distractor B. This becomes a modified bank question.
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution 67 F
2 N
S 68 F
3 B
S PREVIOUS 2 EXAMS DB 2022 Q50 69 F
3 B
S DB 2016 Q75 Made small change to Distractor A and added condensate lineup to stem and changed distractor C to account for the condensate pump.
This is now a modified bank question.
70 F
2 M
S GFES PWR Bank P59 71 H
2 B
S GFES PWR Bank November 2020 P3364 72 F
2 B
S GFES PWR Bank P848 73 H
4 B
S GFES PWR Bank P3939 REF PROVIDED: Steam Tables and Calculation Sheet Modified stem for actual DB plant conditions, made Distractor C the correct answer.
Changed K/A to K1.08 Define the term: Saturated Liquid 74 H
3 B
S GFES PWR Bank P76 REF PROVIDED: Steam Tables 75 F
2 B
S GFES PWR Bank P2800 Editorial changes to stem and Answer D.
76 H
3 M
S REF PROVIDED: Steam Tables Removed vent the pressurizer from Answer C Made the question a 2x2 to minimize overlap with Q29.
77 F
2 M
S DB 2008 Q45 Removed reference to MU Pumps and focused on HPI and LPI. Adjusted wording of answer choices.
78 H
2 M
S DB 2022 Q13 Modified question to better align with SRO only question criteria
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution Added from 100% power to stem 79 H
2 M
S DB 2005 Q79 (K/A change)
Replaced one annunciator which was determined not to alarm in the simulator.
Modified question from 2x2 format and clarified the question to meet 55.43 (SRO only) criteria Modified distractors to Waterfall format 80 F
2 N
S Changed Distractor C to DBP07.
81 H
3 B
S Oconee 2015 Q81 Ref Prov: DB-PF-06703, CC9.5 Adjusted stem to have Reactor power >40%.
82 F
3 B
S Editorial changes to stem and answer choices.
Changed Question to 2016 Q83 and changed K/A to minimize overlap in the area of releases.
DB 2016 Q83 83 H
2 N
S Ref Prov: Davis-Besse EAL Wall Board - Hot EALs 84 F
2 N
S Modified distractor C to align with SRO duties and responsibilities 85 H
2 N
S Changed stem to start with Breaker ABDC1 OOS and SBODG available and changed Distractor B to start SBODG in place of DG1.
Modified answer to distractor C. filling EDG 1 Day Tank using Diesel Fuel Oil Transfer Pump 2 to align with procedure section name Minor editorial changes 86 H
3 B
S PREVIOUS 2 EXAMS DB 2022 Q87 Modified Distractor A to make it significantly different from the other ones 87 H
2 N
S Editorial changes to stem and changed Distractor C to ensure it is an incorrect answer.
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution Modified stem to align with SRO duties and responsibilities. Spelled out Control Room throughout question 88 H
3 N
S Replaced question due to operational validity.
Crew would not pull rods during cooldown for reactivity.
Modified question and distractors to better align with K/A statement.
Changed question stem to make C correct answer and replaced D with a new distractor.
89 H
2 N
S Adjusted timeline in stem for clarity which changed correct answer.
added, 1100 on 1/8 is past the required action time to the distracter C analysis.
90 H
2 B
S DB 2018 Q77 Enhanced distractor analysis 91 H
3 M
S DB 2015 Q93 Changed stem by indicating lowering S/G level and changed distractors and answer.
92 F
3 B
S PREVIOUS 2 EXAMS DB 2021 SROQ18 93 H
3 N
S Adjusted question stem and answer/distractors due to a technical error.
Correct Distractor A plausibility analysis: L&P 2.1.6 not 2.1.7 Minor editorial changes 94 F
2 N
S Added CONDUCT OF OPERATIONS to the K/A statement.
Modified to read, you MUST direct the Bridge Operator to Added back into the to distractor D for psychometric issues
Form 2.3-5 Written Examination Review Worksheet Facility: DAVIS-BESSE Exam Date: JANUARY 2024
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Licensee Comments/Resolution 95 F
3 N
S Changed Distractors A and B to eliminate any possibility of these being correct.
96 H
2 B
S DB 2016 Q97 Ref Prov: T.S. Section 1.3, Completion Times Added T.S. Section 1.3 as a reference to this question 97 F
2 N
S Added EQUIPMENT CONTROL to the K/A statement.
98 F
2 N
S 99 H
3 B
S DB 2015 Q99 100 F
2 B
S Added info to Answer/Distractors to match procedure.
Modified distractors to alphabetical sequencing format
Form 2.3-5 Written Examination Review Worksheet Facility: Any Plant Exam Date: 12th of Never Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.
Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:
- 1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.
Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.
T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.
Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.
- 4. Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.
Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.
- 5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.
- 6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.
- 7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.
- 8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).
- 9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.