ML24011A182
| ML24011A182 | |
| Person / Time | |
|---|---|
| Site: | Limerick  |
| Issue date: | 12/14/2023 |
| From: | Thomas Setzer NRC/RGN-I/DORS/OB |
| To: | Constellation Energy Generation |
| References | |
| EPID L-2024-OLL-0000, 50-352/24-301, 50-353/24-301 50-352/OL-24, 50-353/OL-24 | |
| Download: ML24011A182 (1) | |
Text
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Facility: Limerick Exam Date: December 4-8, 2023 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL RO Admin 1 -
OVERTIME AVAILABILITY (2.1.5, Ability to use procedures related to shift staffing, such as minimum crew complement or overtime limitations IMP 2.9/3.9) (LOJPM6738)
A N
2 E
Typo fixed, initiating Cue Old:
- 1. Determine if you can work the entire requested shift (Sunday 12/7, 0100-0900), if not, list any and ALL restrictions that apply.
New:
Determine if you can work the entire requested shift (Sunday 12/17, 0100-0900), if not, list any and ALL restrictions that apply RO Admin 2 - MAIN GENERATOR VARS (2.1.25, Ability to interpret reference materials, such as graphs, curves, and tables, IMP 3.9/4.2)
(LOJPM6739)
A N
2 S
RO Admin 3 -
EVALUATE JET PUMP OPERABILITY (2.2.12, Knowledge of Surveillance Procedures, IMP 3.7)(LOJPM6717)
A N
2 S
RO Admin 4 - ACTIONS REQUIRED FOR SPIKING ARM (2.3.12, Knowledge of radiological safety principles and procedures pertaining to licensed operator duties, such as response to radiation monitor alarms, IMP 3.2)(LOJPM6742)
A N
2 E
Removed steps determined to be extraneous to JPM (step 4-10)
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Facility: Limerick Exam Date: December 4-8, 2023 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL SRO Admin 1 -
REACTIVATION OF SRO LICENSE (2.1.4, Knowledge of individual licensed operator responsibilities related to shift staffing, such as medical requirements, no-solo operation, maintenance of active license status, IMP SRO 3.8)
(LOJPM6710)
A N
3 E
PG 4 Initiating cue date typo corrected to match student copy (December 04 2023)
Step *6 modified:
OLD Text:
OP-AA-105-102, Attachment 2 Shift Managers review NOT signed-off, or all gaps requiring resolution listed NEW Text:
OP-AA-105-102, Attachment 2 Shift Managers review NOT signed-off, AND all gaps requiring resolution listed SRO Admin 2 - REVIEW AND APPROVE RMSI (2.1.37, Knowledge of procedures, guidelines, or limitations associated with reactivity management, IMP 4.6)
(LOJPM6727)
A N
2 E
Revised cues:
Old: You are directed to review the provided documents and sign for SRO AUTHORIZATION.
Identify if there are any discrepancies and whether they would prevent your authorization of the RMSI, cite each of them on Cue Sheet.
Form 2.3-3 Operating Test Review Worksheet (JPMs)
New:
You are directed to review the provided documents and sign for SRO AUTHORIZATION.
If there are any discrepancies that prevent your authorization of the RMSI Identify any / all the discrepancies on Cue Sheet.
SRO Admin 3 - REVIEW
/ CONTINUE PERFORMANCE OF MAIN TURBINE BYPASS VALVE EXERCISING SURVEILLANCE TEST (2.2.12, Knowledge of surveillance procedures, IMP 4.1)
(LOJPM6741)4.7)
A N
2 E
Steps and attachments corrected to match:
Initial conditions Cycle exposure Steps 4a, 8, 9, now LGS2 SRO Admin 4 -
TRANSPORTATION ACCIDENT INVOLVING RAD MATERIAL (2.3.14, Knowledge of radiation or contamination hazards that may arise during normal, abnormal, or emergency conditions or activities regulatory limits, IMP 3.8)
(LOJPM6123)
A N
3 E
Updated initial cue to add:
o There is no indication of water pollution Updated step 4f o No longer critical o Added cue that if truck driver contacted about pollution
Form 2.3-3 Operating Test Review Worksheet (JPMs)
SRO Admin 5 - ERP CLASSIFICATION AND REPORTING (2.4.41, Knowledge of the emergency action level thresholds and classifications IMP 4.6)
(LOJPM3145)
(LOJPM3166)
A N
3 E
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Facility: Limerick Exam Date: December 4-8, 2023 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL
- a. CONTROL ROD EXERCISE TEST (201002 A2.04, Reactor Manual Control System, IMP 3.5)(LOJPM3046)
S Y
2 E
Alternate Path deleted from cover sheet
- b. START A CONDENSATE PUMP (256000 A3.02/A4.03 Condensate System, IMP 3.4/3.3)
(LOJPM3530)
S Y
2 S
(LOJPM3086)
S N
2 E
Initial conditions updated, added:
3rd Offsite source is not connected
- d. TRANSFER HPCI FROM LEVEL TO PRESSURE CONTROL (206000 A2.10/A4.02, High-Pressure Coolant Injection System, IMP 4.7/4.3)
(LOJPM3164)
S Y
3 E
Steps 3 and 8 changes from critical to non-critical Step standards changed to reflect as found component position:
Old Step 3:
Examinee places flow controller in MANUAL New Step 3:
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Examinee ensures flow controller in MANUAL Old Step 8:
FIC-55-1R600 and/or HV-55-1F008 adjusted to raise speed and flow rate as required New Step 8:
If required: FIC-55-1R600 and/or HV-55-1F008 adjusted to raise speed and flow rate
- e. T-228 DRYWELL INERTING WITH NITROGEN (500000 EA1.07, High Containment Hydrogen Concentration, IMP 3.4) (LOJPM3763)
S N
3 E
Steps 14 and 15 deleted, termination point of JPM now step 13
- f. TRANSFER 101-D13 TO 201-D13 (264000 A4.04, Emergency Generators (Diesel/Jet) IMP 4.1)(LOJPM3041)
S N
3 E
Typo fixed, Step 3 outboard MSIV, corrected to read inboard
(LOJPM3165)
S N
3 S
Form 2.3-3 Operating Test Review Worksheet (JPMs)
- h. STANDBY GAS TREATMENT MANUAL STARTUP (261000 A4.03, Standby Gas Treat, IMP 3.7)
(LOJPM3528)
S Y
3 E
Step 4, Cue added: CUE: Another RO will acknowledge that alarm Step 9, Cue added: CUE: HP notified
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Facility: Limerick Exam Date: December 4-8, 2023 1
JPM # or title 2
Typ e
(S/
P/A
)
3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL
- i. ALTERNATE INJECTION FROM THE FIRE SYSTEM (295031 EA1.08, Rx Low Water Level, IMP 3.8)(LOJPM2260)
P N
3 S
- j. ALTERNATE DEPRESSURIZATION USING RHR SHUTDOWN COOLING (205000 A2.05, RHR SDC, 4.0)
(LOJPM2283)
P N
3 S
- k.
START ESW PUMP PER SE-1 (400000 A2.02, Comp Cooling Water, IMP 3.3)
(LOJPM2258)
P Y
3 S
Facility: Limerick Scenario: 1 SEG-2166E Exam Date: December 4-8, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
N-BOP N-SRO E
2 C-ATC C-SRO E
3 C-ATC MC-BOP TS-SRO E
CT TS changes noted below in edits 4
MC-BOP R-ATC C-SRO TS-SRO E
TS changes noted below in edits 5
C-All E
CT 6
M-All E
7 MC-BOP C-SRO E
8 MC-ATC E
9 C-BOP C-SRO E
SAT Edits:
- Changed the SEG to add a failure to automatically scram Added critical task T-101.1, Scram the Reactor Added E-D13, Attachment 3, page 1 and 2 to the SEG. This lists all impacted Tech Specs. 3 TSs were agreed to being the ones the SRO are required to identify - 3.8.1.1.a, 3.8.1.1.e, and 3.0.3 Removed assessment item for TS 3.4.6.2 and TS ref. in the event outline Removed ARC 107-G2, Rx Hi Press. Ref.
Removed Startup in Progress Added assessment item for TS 3.4.5.a
Facility: Limerick Scenario: 2 SEG-3160E Exam Date: December 4-8, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
R-ATC E
2 MC-ATC C-SRO E
3 C-BOP TS-SRO E
4 MC-BOP TS-SRO E
5 C-ATC C-BOP C-SRO E
6 M-ALL C-ATC E
7 M-ALL E
8 C-BOP E
9 MC-ATC E
10 C-BOP E
SAT Edits:
Added S73.0.D to expected action in the event that crew is in T-100 instead of T101 Pg. 15 evaluator note at top refers to Section 4.3 continuous withdraw. This section is not relevant to this scenario, removed Removed reference to 011-D3 Changed wording to match ARC to isolate RWCU per OT-200 App 7 Response time changed to 2 minutes Removed step to Ensure Turbine trip and
Generator Lockout, not applicable for a low power scenario Added assessment item to enter T-101 when Reactor level goes below 12.5 Added Simulator Operator instruction:
At time 3 min When requested to investigate panel 10C234:
report: RCIC Steam Flooding Dampers indicate closed
Facility: Limerick Scenario: 3 SEG-7018E Exam Date: December 4-8, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
C-BOP R-ATC C-SRO E
2 C-BOP MC-BOP TS-SRO E
3 C-ATC TS-SRO E
4 C-BOP MC-ATC C-SRO E
5 C-ATC C-SRO E
6 M-ALL E
7 C-ATC E
8 MC-ATC E
9 C-BOP E
SAT Edits:
Pg 12, listed procedure specific to this scenario (T-216 and/or T-216)
Pg 14, edited to: When Reactor Power is reduced to less than 90% with recirc flow then: DELETE PG 15, edited to: Lower Main Generator reactive load to 0 MVARs OR 20.9 KV Terminal Voltage, whichever is lower Pg 16, edited 3 minutes to 2 minutes
Tech Spec Added (T.S. 3.4.4.5.c.1)
Pg 32, Driver instructions added:
IF Reactor power lowers to less than 14% Insert RDCS INOP MRD024 WHEN Reactor level is below -161, with the concurrence of the lead evaluator report: The next step will cause Rod motion and DELETE malfunction MAD140C Evaluator note now reads Pg 35, EVALUATOR NOTE now reads: The scenario may be terminated at the discretion of the lead evaluator, or when the reactor level stabilized within required band, and all rod are in.
Facility: Limerick Scenario: 4 SEG-4165E (SPARE)
Exam Date: December 4-8, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
I-BOP TS-SRO E
2 R-ATC C-BOP MC-BOP TS-SRO E
3 MC-ATC TS-SRO E
4 C-ATC C-SRO E
5 M-ALL C-ALL E
6 M-ALL E
7 C-BOP E
8 MC-BOP E
9 C-BOP E
10 C-BOP SAT Edits:
Critical task T-101.5 Safety-significant boundary condition, only considered entry into T-112, modified to:
Entry and commencing execution of T-112 prior to RPV level reaching Minimum Steam Cooling RPV Water Level (MSCRWL) (-186)
Critical task T-112.2 Safety-significant boundary condition, did not address operator action, modified to:
Within 15 minutes of entering the restricted region, commence actions to exit restricted region
Added item:
Evaluate Technical Specification LCO 3.3.2.B.1 - determine placing the inoperable channel(s) in the tripped condition would cause an isolation, the inoperable channel(s) shall be restored to OPERABLE status within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> Pg 17 typo, modified to read critical parameter ARC-MCR-107 I3, text modified to include actions to reduce speed Pg 20 typo, corrected to read PIS-42-1N650D ARC ARC 118B-4 added Positions added on Page 23 Pg 25, Recognize unexpected/unexplained drop in RPV level - enter OT-100, Reactor Low Level, not applicable, Item deleted Page 33, 2 steps added:
SRO: When RPV level cannot be restored and maintained above -161 then direct BOP to line up for injection, start pumps and inject into the RPV from preferred Injection systems BOP: Line up for injection, start pumps and inject into the RPV from preferred Injection systems
Form 2.3-5 Written Examination Review Worksheet Limerick December 2023 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 1
H 3
B S
2 H
1 N
U LOD1, only 2 minutes has elapsed after the scram, which makes answer choice D an obvious correct answer.
Changed 2 minutes to 15 minutes.
3 F
2 N
E Typo in stem and added statement no operator action taken (OT-112) 4 H
2 B
S 5
H 3
N S
6 F
2 N
S 7
H 3
N S
8 H
2 N
E Stem typo corrected 9
F 2
B S
10*
H 3
X N
B U
U The new question as written is too plant specific for a Tier 3 question and doesnt meet the generic K/A (does not test ability to use radiation monitoring systems).
Rewritten question from Bank has subset issue, C is subset of correct answer A. Edited C to state frisk at current location 11 H
3 N
E Removed reference to hard failure and edited stem.
12 H
3 B
S 13 H
2 B
S 14 F
3 X
B U
This question is written at an SRO level (knowledge of implementation of emergency procedures)
Rewritten bank question is SAT 15 H
3 N
S 16 H
4 N
E Minor stem edit
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 17 H
1 B
U LOD1 with embedded reference (DLU)
Rewritten question SAT NOTE: Based on post exam comments, this question was deleted. It is therefore UNSAT and counts towards determining the overall quality of the exam 18 F
2 B
S 19 H
3 N
E Edited stem, moved, The 'D' IRM detector fails down scale before the IRM table 20*
H 3
N S
21 H
3 N
E Edited stem for better readability.
22 F
3 B
E Minor formatting changes made 23 H
3 X
B U
K/A mismatch, this question does not test knowledge of the operation of a flow dP cell flow detector.
Question rewritten SAT.
24 F
3 N
E Minor formatting changes made 25 F
2 X
N U
A and B implausible (heavy smoke). Also, should in the stem makes all part 1 distractors correct.
Edited question SAT with more edits to stem 26 F
3 N
S 27 H
3 B
E Modified stem to removed preferred source 28 H
1 X
B U
B2 and D2 are implausible - scram profile being activated on a scram (LOD 1).
Question stem edited to up level of difficulty.
Rewritten question SAT.
29 F
3 N
S 30 F
1 N
U LOD1. This question tests rote memorization of a power supply and low level of knowledge testing (low operational validity and low discriminatory value)
Rewritten question is SAT Question rewritten SAT 31 H
3 N
E Edited stem to remove continuously 32 F
3 X
N U
A and B are both correct due to subset (>100 psig).
Rewritten question SAT
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 33 H
2 X
N U
A and B not plausible (restart a recirc pump after it just tripped to raise flow in order to exit the restricted region). Also, there are potentially no correct answers as procedure says to insert control rods.
New question rewritten, edits made to punctuation, format and grammar.
34 F
3 N
E Minor typos corrected and formatting 35*
H 3
B S
36 F
2 B
E Minor font edits 37 F
3 X
N U
B2 and D2 are implausible, an operator would not reset an AE alarm (HI Rad) that is in excess of 100x normal rad levels Rewritten question SAT.
38 F
3 N
S 39 F
3 B
E Minor typos corrected 40 H
2 B
S 41 F
2 X
N U
C and D are implausible (42 degF) when determining TS cooldown rate.
Rewritten question is SAT 42 H
2 N
E Edited distractor A to a more plausible distractor.
43 F
2 X
X N
U The Auto Isolation noun name in the stem cues A1 and B1 as being incorrect and implausible. Also, air pressures of headers not plausible.
Rewritten question SAT 44 H
2 N
E Question overlaps with other questions on 10 or 20 aux bus. Edited stem to eliminate overlap.
45 F
3 N
S 46 F
2 N
U NOTE: Based on post exam comments, this question was determined to have 2 correct answers. It is therefore UNSAT and counts towards determining the overall quality of the exam 47 H
2 B
S 48 F
2 X
B U
B2 and D2 are implausible since protecting the turbine from overspeed is not a basis for RPS.
Rewritten question is SAT
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 49 F
3 B
E SGTS is overlapped, redrew K/A and rewrote.
Rewritten question is SAT 50 F
3 X
N U
Part 2 distractors test SRO level knowledge.
New question has 2 correct answers. D correct also because the should - a blowdown would mitigate the release. Question was edited to ask what is required, new question is SAT.
51 H
3 B
E Edited stem to remove to restore cooling 52 H
2 B
E Overlap with 44, edited question to remove overlap with 10 bus.
53 H
2 N
E Rewrote question due to low level of difficulty and distractors A and B not very plausible..New question SAT.
54 F
2 B
S 55 F
2 N
S 56 H
2 N
S 57 H
3 N
E Overlap, redrew K/A New question SAT 58 H
3 B
E Question is good but matches A1.09 better than A1.11.
59 H
2 N
S 60 H
2 B
S 61 F
2 X
N U
The stem asks the lowest pressure the PRO should close. 900 psig would also be correct when a PRO should close the SRV, therefore A and C are both correct.
Redrew K/A and rewrote question.
62 F
2 B
S 63*
F 3
N S
64 F
2 B
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 65 H
3 B
S 66 H
2 N
S 67 H
3 N
E Edited one distractor to add another safeguards bus (bus 2DA) 68 F
2 X
N U
Subset, B and D are correct.
Rewritten question SAT 69*
H 3
N S
70 F
3 N
S 71 H
2 N
S 72 H
3 B
S 73 H
3 N
E Minor formatting edits made 74 H
1 X
N E
Minor edits 75 H
2 X
N U
A1 and B1 are implausible. The stem states that the D13 bus has a lockout. It is not plausible that a candidate would think the D13 would power the D13 bus in a lockout condition.
Rewritten question SAT 76 H
3 N
S 77 H
2 X
N U
The stem asks whether the strainer is clogged.
Clogged is not defined in procedures. This is too subjective making no answer correct.
K/A was redrawn to A2.07. Rewritten question has no correct answer due to the subjective language in the stem asking for most limiting and highest frequency New question edited SAT 78 H
2 X
N U
The stem asks what is the highest action which is not defined in procedures. This is too subjective making no answer correct.
Rewritten question SAT 79 H
3 N
E K/A is not matched well. Questioned edited to test license limitations (safety limits). New question required edits to spelling and grammar. New question SAT 80 H
2 X
N U
Part 2 cues that part one should have been graphed on the inoperable side of the graph. Wording was edited in the stem to state if you were on the unsafe side.
Rewritten question SAT
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 81 H
2 X
N U
Another graphing question similar to 3 others.
Additionally, the graph in this stem informs the answer to question 65.
Rewritten question is SAT 82*
H 2
X N
U C is correct also because it asks what should be done. According to T-102, the candidate would spray the drywell prior to 340 degF. Question edited to ask what is required.
83 H
2 N
E This question as written overlaps with an Operating Test JPM. Rewrote question. New question SAT with minor edits.
84 F
3 N
S 85**
H 2
X B
U All Part 2 distractors are correct since you would enter them subsequently. Stem is also worded incorrectly, GP-4 is not required to prevent components failing open.
Removed subsequently, edited question, SAT Based on the results of an Administrative Review (informal staff review) requested by the candidate, this question was deleted. It is therefore UNSAT and counts towards determining the overall quality of the exam 86*
H 3
N S
87 H
3 N
S 88 H
1 X
N U
The stem states a loss of Division 3 DC occurs, which cues B and C as being correct (makes A and D implausible). The stem also states that the system realigns which cues B as correct. The stem provides all information with little analysis needed (LOD 1)
Rewritten question is SAT 89*
H 2
X X
N U
The stem states should be directed which is subjective. A and B are implausible since the stem asks for a basis for the action however A and B do not include a basis.
Rewritten question SAT 90 F
1 X
N U
Choice 1 in stem to verify mode switch in REFUEL is too obvious an answer (LOD1), which as a result makes B and C implausible since they dont include Choice 1 Rewritten question is SAT 91*
H 2
X X
N U
B is correct also since the candidate would manually initiate ARI (RRCS). Stem asks what should be directed in part 2 which also makes B correct. Also, this question informs question 61. This question was submitted as part of the initial 10 question submittal but since this informs another question (61) it is unsat for this specific exam.
Rewritten question is SAT
92 H
2 X
N U
The question is written too plant and system specific for a Tier 3 K/A Rewrote question, new question is SAT and was edited for grammar and spelling.
NOTE: Based on post exam comments, this question was determined to have 2 correct answers. It is therefore UNSAT and counts towards determining the overall quality of the exam 93 H
1 X
N U
This question is DLU based on the candidate being given the TS reference in the stem and having only to find temperature greater than 120 degF Rewritten question SAT 94 F
2 N
S 95 H
2 X
N U
B is correct also. TSs also may preclude all distractors from being correct.
Edited Distractors A and B to state no shutdown required, making B incorrect.
96 H
3 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 97 H
1 X
X N
U A and B are implausible unless a math error occurs.
The first part of the question tests arithmetic which is low level of difficulty.
Rewritten question edited for grammar and spelling.
98**
H 3
B S
99*
H 2
X X
N U
D is correct also since the stem stated that fuses have been removed. Also, stem asks what the CRS should do which is subjective.
Rewritten question still asks what should the CRS do and subset in part 1 makes all part 1 answers correct.
Rewritten question edited further and is SAT 100 F
2 N
S
- The questions marked with an asterisk were part of the licensees first 10 questions submittal for review and do not count towards determining the quality of the exam submittal unless noted.
- The questions marked with a double asterisk are questions from past NRC examinations and do not count towards determining the quality of the exam submittal unless noted.
Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.
Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:
- 1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.
Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.
T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.
Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.
- 4. Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.
Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.
- 5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.
- 6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.
- 7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.
- 8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).
- 9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.
marked by an (S) on this form.