ML21218A148
| ML21218A148 | |
| Person / Time | |
|---|---|
| Site: | Oconee  |
| Issue date: | 08/06/2021 |
| From: | Division of Reactor Safety II |
| To: | |
| References | |
| Download: ML21218A148 (28) | |
Text
ES-301 Administrative Topics Outline Form ES-301-1 Facility:
Oconee Nuclear Station Date of Examination:
06/07/21 Examination Level: RO SRO Operating Test Number:
2021-301 Administrative Topic (see Note)
Type Code*
Describe activity to be performed Conduct of Operations
[KA: G2.1.7 (4.4/4.7)]
(35 min)
D, R ADM-117, Perform a Reactor Power Imbalance Verification and Incore Detector Operability Conduct of Operations
[KA: G2.1.25 (3.9/4.2)]
(15 min)
N, R ADM-147, Perform a RCS Cooldown Calculation and Determine Status of RCP NPSH Equipment Control
[KA: G2.2.14 (3.9/4.3)]
(25 min)
D, R ADM-205, Determine LTOP Requirements Radiation Control
[KA: G2.3.4 (3.2/3.7)]
(10 min)
D, R ADM-302, Calculate the Maximum Permissible Stay Time Within Duke Power Basic Administrative Limits Emergency Plan N/A NOTE: All items (five total) are required for SROs. RO applicants require only four items unless they are retaking only the administrative topics (which would require all five items).
- Type Codes and Criteria:
(C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs and RO retakes)
(N)ew or (M)odified from bank ( 1)
(P)revious 2 exams ( 1, randomly selected)
ES-301 Administrative Topics Outline Form ES-301-1 Facility:
Oconee Nuclear Station Date of Examination:
06/07/21 Examination Level: RO SRO Operating Test Number:
2021-301 Administrative Topic (see Note)
Type Code*
Describe activity to be performed Conduct of Operations
[KA: Gen 2.1.7 (4.4/4.7)]
(35 min)
D, R ADM-S140, Perform a Power Imbalance Verification and Determine any Associated TS Requirements (SRO Only)
Conduct of Operations
[KA: G2.1.13 (2.5/3.2)]
(20 min)
D, R ADM-S106, Evaluate Items For Entry Into Containment (SRO Only)
Equipment Control
[KA: G2.2.18 (2.6/3.9)]
(27 min)
N, R ADM-S250, Determine Outage Risk Level Using S.D. 1.3.5 Defense in Depth Assessment Sheet (SRO Only)
Radiation Control
[KA: G2.3.13 (3.4/3.8)]
(10 min)
D, R ADM-S310, Select Individuals for Emergency Exposure (SRO Only)
[KA: G2.4.38 (2.4/4.4)]
(30 min)
D, R ADM-S404, Determine Emergency Classification and Protective Action Recommendations (Complete Emergency Notification Form)
(SRO Only)
NOTE: All items (five total) are required for SROs. RO applicants require only four items unless they are retaking only the administrative topics (which would require all five items).
- Type Codes and Criteria:
(C)ontrol room, (S)imulator, or Class(R)oom (D)irect from bank ( 3 for ROs; 4 for SROs and RO retakes)
(N)ew or (M)odified from bank ( 1)
(P)revious 2 exams ( 1, randomly selected)
ES-301 Control Room/In-Plant Systems Outline Form ES-301-2 ILT 21-1 NRC Exam Facility: Oconee Nuclear Station Date of Examination: 06/07/2021 Exam Level: RO X SRO-I SRO-U Operating Test No.: 1 Control Room Systems:* 8 for RO; 7 for SRO-I; 2 or 3 for SRO-U System / JPM Title Type Code*
Safety Function
- a. RO-506a, Establishing Containment Closure AP/1/A/1700/026, (Loss of Decay Heat Removal)
[KA: SYS103 EA2.03 (3.5*/3.8*)] (22 min)
D, A, S, L 5
[KA: SYS006 A4.08 (4.2/4.3)] (15 min)
D, S, EN, L 2
- c. RO-307a, Establish Continuous PZR Spray OP/1/A/1103/005 Enclosure 4.4, (Establishing Continuous PZR Spray)
[KA: SYS010 A4.01 (3.7/3.5)] (10 min)
A, N, S 3
[KA: SYS003 A4.03 (2.8/2.5)] (10 min)
D, A, L, S 4P
- e. RO-805a, Perform Required Actions for a Turbine Building Flood AP/1/A/1700/010, (Turbine Building Flood)
[KA: APE BW/A07 AA1.3 (3.3/3.5)] (15 min)
D, A, L, S 8
- f.
RO-601, Synchronization with the grid following a load rejection AP/1/A/1700/001, (Unit Runback)
[KA: SYS062 A4.07 (3.1*/3.1*)] (15 min)
D, S 6
- g. RO-702, Adjust Radiation Monitor Setpoints OP/1-2/A/1104/018 (GWD System) Enclosures 4.9 & 4.10
/ PT0/A/0230/01 (Radiation Monitor Check)
[KA: SYS073 A4.01 (3.9/3.9)] (20 min)
D, S 7
[KA: APE024 AA1.04 (3.6*/3.7)] (15 min)
D, S 1
In-Plant Systems* (3 for RO); (3 for SRO-I); (3 or 2 for SRO-U)
- i.
AO-S405, Perform EP/3/A/1800/001 Enclosure 5.17 Unit 3 Generator Emergency Hydrogen Purge EP/3/A/1800/001 Enclosure 5.17, (Generator Emergency Hydrogen Purge)
[KA: EPE055 EK3.02 (4.3/4.6)] (22 min)
N, E, L 4S
- j.
AO-101, Swap CRD Filters OP/1/A/1104/008 Enclosure 4.19, (Placing 1A or 1B CRD Filter in Service)
[KA: SYS001 A2.08 (2.9/3.3)] (15 min)
D, R, P 1
- k. AO-802a, Start Diesel Air Compressor and Align to Service Air Header AP/2/A/1700/022 (Loss of Instrument Air) Enclosures 5.4 (Emergency Start of the Diesel Air Compressor) and 5.7 (Manual Start of Diesel Air Compressors)
[KA: APE065 AA1.04 (3.5*/3.4*)] (20 min)
D, A, E 8
All RO and SRO-I control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.
- Type Codes Criteria for RO / SRO-I / SRO-U (A)lternate path (C)ontrol room (D)irect from bank (E)mergency or abnormal in-plant (EN)gineered safety feature (L)ow-Power / Shutdown (N)ew or (M)odified from bank including 1(A)
(P)revious 2 exams (R)CA (S)imulator 4-6 / 4-6 / 2-3 9 / 8 / 4 1 / 1 / 1 1/ 1/ 1 (control room system) 1 / 1 / 1 2 / 2 / 1 3 / 3 / 2 (randomly selected) 1 / 1 / 1
ES-301 Control Room/In-Plant Systems Outline Form ES-301-2 ILT 21-1 NRC Exam Facility: Oconee Nuclear Station Date of Examination: 06/07/2021 Exam Level: RO SRO-I SRO-U X Operating Test No.: 1 Control Room Systems:* 8 for RO; 7 for SRO-I; 2 or 3 for SRO-U System / JPM Title Type Code*
Safety Function
- a. RO-506a, Establishing Containment Closure AP/1/A/1700/026, (Loss of Decay Heat Removal)
[KA: SYS103 EA2.03 (3.5*/3.8*)] (22 min)
D, A, S, L 5
[KA: SYS006 A4.08 (4.2/4.3)] (15 min)
D, S, EN, L 2
- c. RO-307a, Establish Continuous PZR Spray OP/1/A/1103/005 Enclosure 4.4, (Establishing Continuous PZR Spray)
[KA: SYS010 A4.01 (3.7/3.5)] (10 min)
A, N, S 3
- d. N/A
- e. N/A
- f.
N/A
- g. N/A
- h. N/A
In-Plant Systems* (3 for RO); (3 for SRO-I); (3 or 2 for SRO-U)
- i.
AO-S405, Perform EP/3/A/1800/001 Enclosure 5.17 Unit 3 Generator Emergency Hydrogen Purge EP/3/A/1800/001 Enclosure 5.17, (Generator Emergency Hydrogen Purge)
[KA: EPE055 EK3.02 (4.3/4.6)] (22 min)
N, E, L 4S
- j.
AO-101, Swap CRD Filters OP/1/A/1104/008 Enclosure 4.19, (Placing 1A or 1B CRD Filter in Service)
[KA: SYS001 A2.08 (2.9/3.3)] (15 min)
D, R, P 1
- k. N/A All RO and SRO-I control room (and in-plant) systems must be different and serve different safety functions; all 5 SRO-U systems must serve different safety functions; in-plant systems and functions may overlap those tested in the control room.
- Type Codes Criteria for RO / SRO-I / SRO-U (A)lternate path (C)ontrol room (D)irect from bank (E)mergency or abnormal in-plant (EN)gineered safety feature (L)ow-Power / Shutdown (N)ew or (M)odified from bank including 1(A)
(P)revious 2 exams (R)CA (S)imulator 4-6 / 4-6 / 2-3 9 / 8 / 4 1 / 1 / 1 1/ 1/ 1 (control room system) 1 / 1 / 1 2 / 2 / 1 3 / 3 / 2 (randomly selected) 1 / 1 / 1
ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Oconee Exam Date: 2021-301 1
2 3
Attributes 4
Job Content 5
6 Admin JPMs ADMIN Topic and K/A LOD (1-5)
U/E/S Explanation I/C Cues Critical Scope Overlap Perf.
Key Minutia Job Link Focus Steps (N/B)
Std.
RO COO1 A1 2
S 2018 RO COO2 A1 2
S RO EC A2 2
S RO RC A3 2
S SRO COO1 A1 2
S 2018 SRO COO2 A1 2
S SRO EC A2 2
S SRO RC A3 2
S 1
Simulator/In-Plant Safety Function and K/A JPMs A
5 2
Comments provided separately.
B 2
2 C
3 2
D 4P 2
E 8
2 F
6 2
G 7
2 H
1 2
I 4S 2
J 1
2 2018 K
8 2
ES-301 2
Form ES-301-7 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1.
Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.
(ES-301, D.3 and D.4)
- 2.
Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)
- 3.
In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4.
For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6.
In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
ES-301 3
Form ES-301-7 Facility: Oconee Scenario: 1 Exam Date: 2021-301 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
S 2
S 3
X X
S 4
X S
5 S
6 X
S 7
X S
8 S
ES-301 4
Form ES-301-7 Instructions for Completing This Table:
Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
In column 1, sum the number of events.
In columns 2-4, record the total number of check marks for each column.
In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)
In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
ES-301 5
Form ES-301-7 Facility: Oconee Scenario: 2 Exam Date: 2021-301 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
S 2
X S
2020 S3/E5 3
X X
S 4
X X
S 2018 S4/E2 5
X S
2018 S2/E7; 2020 S2/E3 6
X S
7 X
S 8
X S
2020 S2E7 (as complicating malf during major)
ES-301 6
Form ES-301-7 Facility: Oconee Scenario: 3 Exam Date: 2021-301 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
S 2
X S
2020 S1/E2 3
X S
4 S
5 X
S 6
X S
7 X X S
8 S
9 S
ES-301 7
Form ES-301-7 Facility: Oconee Scenario: 4 Exam Date: 2021-301 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
X S
2018 S3/E1 2
X S
3 S
4 X
X S
2018 S4/E4 5
X S
2018 S2/E4 6
X S
2018 S4/E5 (Diff RCP, but same malf) 7 S
8 X X S
9 S
ES-301 8
Form ES-301-7 Facility: Oconee Exam Date: 2021-301 Scenario 1
2 3
4 5
6 7
8 11 Event Totals Events Unsat.
TS Total TS Unsat.
% Unsat.
Scenario Elements U/E/S Explanation 1
8 0
2 0
3 0
0 S
2 8
0 2
0 3
0 0
S 3
9 0
2 0
3 0
0 S
4 9
0 2
0 2
0 0
S Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a.
Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b.
TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c.
CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
7 In column 7, calculate the percentage of unsatisfactory scenario elements:
8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
2 + 4 + 6 1 + 3 + 5100%
ES-301 9
Form ES-301-7 Site name: Oconee Exam Date: 2021-301 OPERATING TEST TOTALS Total Total Unsat.
Total Total Unsat.
Explanation Edits Sat.
Admin.
JPMs 9
0 0
9 Sim./In-Plant JPMs 11 0
0 11 Scenarios 4
0 0
4 Op. Test Totals:
24 0
0 24 0
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1.
Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.
For scenarios, enter the total number of simulator scenarios.
- 2.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
- 3.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.
- 4.
Total each column and enter the amounts in the Op. Test Totals row.
- 5.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
- satisfactory, if the Op. Test Total % Unsat. is 20%
- unsatisfactory, if Op. Test Total % Unsat. is > 20%
- 6.
Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:
- The JPM performance standards were incorrect.
- The administrative JPM tasks/keys were incorrect.
- CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
- The EOP strategy was incorrect in a scenario(s).
- TS entries/actions were determined to be incorrect in a scenario(s).
Oconee 2021-301 Final Written Exam Sample Plan The final sample plan is documented with the combination of Form ES-401-4 and the Draft Sample Plan.
ES-401 Record of Rejected K/As Form ES-401-4 Tier /
Group Randomly Selected K/A Reason for Rejection T2G1 006 (ECCS) K4.19: Knowledge of ECCS design feature(s) and/or interlock(s) that provide for the following:
Interlocks to storage tank MAKEUP valve.
Q33: Reason: There are no BWST makeup valve interlocks at Oconee; there is an interlock for the letdown storage tank to the bleed valve, which has nothing to do with the BWST.
Replaced with K4.21 Bypassing/blocking ESF channels T1G2 068 (CR Evacuation)
G2.4.49: Ability to perform without reference to procedures those actions that require immediate operation of system components and controls.
Q83: Reason: Immediate operator actions are inherently NOT SRO; any attempt to write a question at the SRO level will likely include a second part tack-on that has no relationship with the K/A.
Replaced with 2.4.8 Knowledge of how abnormal operating procedures are used in conjunction with EOPs.
(CFR: 41.10 / 43.5 / 45.13) IR SRO 4.5 T2G2 002 (SF2; SF4P RCS) Reactor Coolant System Knowledge of the operational implications of the following concepts as they apply to the RCS: (CFR: 41.5 /
45.7)
K5.02 Purpose of vent flow path when draining Q57: Unable to write a question with plausible distractors that is not LOD=1 Replaced with:
K5.09 Relationship of pressure and temperature for water saturation and subcooling conditions.
T1G2 037 (APE 37) Steam Generator Tube Leak / 3 2.4.9 Knowledge of low power/shutdown implications in accident (e.g., loss of coolant accident or loss of residual heat removal) mitigation strategies.
(CFR: 41.10 / 43.5 / 45.13)
Q82: Not able to write an SRO level question about SGTL when on RHR or low power implications.
Replaced with 2.1.28 Knowledge of the purpose and function of major system components and controls. (CFR: 41.7)
T3 2.2.20 Knowledge of the process for managing troubleshooting activities.
(CFR: 41.10 / 43.5 / 45.13)
No Oconee procedures for managing troubleshooting. Fleet wide procedures are under major revision work for the time frame of exam development so no good SRO level questions can be developed.
Replaced with:
2.2.6 Knowledge of the process for making changes to procedures. (CFR: 41.10 / 43.3 / 45.13)
ES-401 Record of Rejected K/As Form ES-401-4 T1G1 007 (BW E02&E10) Reactor Trip, Stabilization, Recovery / 1 2.4.35 Knowledge of local auxiliary operator tasks during an emergency and the resultant operational effects Q 76: No valid SRO level question could be developed.
Replaced with:
077 (APE 77) Generator Voltage and Electrical Grid Disturbances / 6 2.2.22 Knowledge of limiting conditions for operations and safety limits. (CFR: 41.5 / 43.2 / 45.2) IR 4.0 T3 SRO G 2.4.30 Knowledge of events related to system operation/status that must be reported to internal organizations or external agencies, such as the State, the NRC, or the transmission system operator.
(CFR: 41.10 / 43.5 / 45.11)
Replaced due to potential exam security issue.
With:
G 2.4.32 Knowledge of operator response to loss of all annunciators.
(CFR: 41.10 / 43.5 / 45.13) IR SRO 4.0 T3 G 2.4.23 Knowledge of the bases for prioritizing emergency procedure implementation during emergency operations.
(CFR: 41.10 / 43.5 / 45.13)
Q73: No valid RO level question could be developed.
Replaced with:
G 2.4.2 Knowledge of system set points, interlocks and automatic actions associated with EOP entry conditions.
(CFR: 41.7 / 45.7 / 45.8) IR RO 4.5 T2G2 041 (SF4S SDS) Steam Dump/Turbine Bypass Control Replaced with:
034 (SF8 FHS) Fuel Handling Equipment Q93: No Valid SRO level question could be developed 041A2.02 Replaced with:
034 Ability to (a) predict the impacts of the following malfunctions or operations on the Fuel Handling System; and (b) based on those predictions, use procedures to correct, control, or mitigate the consequences of those malfunctions or operations:
(CFR: 41.5 / 43.5 / 45.3 / 45.13)
A2.03 Mispositioned fuel element T3 G 2.2.41 Ability to obtain and interpret station electrical and mechanical drawings.
(CFR: 41.10 / 45.12 / 45.13)
Q 70: No valid written exam Q could be developed.
Replaced with:
G 2.2.2 Ability to manipulate the console controls as required to operate the facility between shutdown and designated power levels.
(CFR: 41.6 / 41.7 / 45.2) IR RO 4.6
ES-401 Record of Rejected K/As Form ES-401-4 T1G1 065 (APE 65) Loss of Instrument Air / 8 Ability to determine and interpret the following as they apply to the Loss of Instrument Air:
AA2.06 When to trip reactor if instrument air pressure is de-creasing Q81: No valid question could be written that did not overlap with other Qs already submitted for this exam.
Replaced with:
APE008 Pressurizer (PZR) Vapor Space Accident (Relief Valve Stuck Open)
AA2.17 -Ability to determine and interpret the following as they apply to the Pressurizer Vapor Space Accident: (CFR: 43.5 /
45.13) Steam dump valve controller (position).
T2G1 064K1.05 Q49: Could not write a question for starting air pressure as it relates to starting the KHU. Replaced with 064K1.01 (IR=4.1).
Change made on 9MARCH2021).
ES-401 Written Examination Quality Checklist Form ES-401-6 Facility: Oconee Date of Exam: June 2021 Exam Level:
RO X SRO X Item Description Initial a
b*
c*#
- 1.
Questions and answers are technically accurate and applicable to the facility.
AG NA MB
- 2.
- a.
NRC K/As are referenced for all questions.
- b.
Facility learning objectives are referenced as available.
- c.
Correct answer explanation and distractor analysis provided (ES-401, D.2.g)
AG NA MB
- 3.
SRO questions are appropriate in accordance with Section D.2.d of ES-401 AG NA MB 4
The sampling process was random and systematic. (If more than four RO or two SRO questions were repeated from the last two NRC licensing exams, consult the NRR/NRO OL program office).
AG NA MB
- 5.
Question duplication from the licensee screening/audit exam was controlled as indicated below (check the item that applies) and appears appropriate.
__ The audit exam was systematically and randomly developed, or
__ the audit exam was completed before the license exam was started, or
__ the examinations were developed independently, or
_x_ the licensee certifies that there is no duplication, or
__ other (explain).
AG NA MB
- 6.
Bank use meets limits (no more than 75% from the bank, at least 10% new, and the rest new or modified); enter the actual RO/SRO-only question distribution(s) at right.
Bank Modified New 32/6 15/12 28/7 AG NA MB
- 7.
Between 38 and 45 questions of the questions on the RO exam and at least 13 questions of the questions on the SRO-only portion of the exam are written at the comprehension/analysis level (see ES-401, D.2.c); enter the actual RO/SRO-only question distribution(s) at right.
Memory C/A 33/11 42/14 AG NA MB
- 8.
References/handouts provided do not give away answers or aid in the elimination of distractors.
AG NA MB
- 9.
Question content conforms to specific K/A statements in the previously approved examination outline and is appropriate for the tier to which they are assigned; deviations are justified.
AG NA MB
- 10.
Question psychometric quality and format meet the guidelines in Appendix B.
AG NA MB
- 11.
The exam contains the required number of one-point, multiple-choice items; the total is correct and agrees with the value on the cover sheet.
AG NA MB Printed Name/Signature Date
- a.
Author Andreas Goldau / Andreas Goldau 5/27/21 _
- b.
Facility Reviewer (*)
N/A
- c.
NRC Chief Examiner (#)
Mark Bates / Mark Bates 5/27/21 _
- d.
NRC Regional Supervisor Gerald McCoy /
5/28/21 _
Note:
The facility reviewers initials or signature are not applicable for NRC-developed examinations.
Independent NRC reviewer initials items in Column c; chief examiner concurrence is required.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q -
K/A SRO Only 1
H 2-4 N
S NRC authored exam.
2 H
2-4 M
S 3
H 2-4 B
S 4
F 2-4 B
S 5
H 2-4 B
S 6
F 2-4 B
S 7
H 2-4 N
S 8
H 2-4 M
S Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q -
K/A SRO Only 9
H 2-4 B
S 10 H
2-4 B
S 11 F
2-4 B
S 12 H
2-4 B
S 13 H
2-4 N
S 14 F
2-4 N
S 15 H
2-4 B
S 16 H
2-4 M
S 17 F
2-4 B
S 18 H
2-4 M
S 19 H
2-4 B
S 20 H
2-4 N
S 21 F
2-4 B
S 22 H
2-4 N
S 23 F
2-4 M
S 24 H
2-4 N
S 25 H
2-4 M
S
ES-401 3
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q -
K/A SRO Only 26 H
2-4 M
S 27 H
2-4 M
S 28 H
2-4 B
S 29 F
2-4 B
S 30 H
2-4 N
S 31 H
2-4 N
S 32 F
2-4 N
S 33 H
2-4 M
S 34 F
2-4 B
S 35 H
2-4 M
S 36 F
2-4 B
S 37 H
2-4 N
S 38 H
2-4 N
S 39 H
2-4 B
S 40 F
2-4 N
S 41 H
2-4 N
S 42 H
2-4 M
S
ES-401 4
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q -
K/A SRO Only 43 H
2-4 N
S 44 H
2-4 N
S 45 H
2-4 B
S 46 F
2-4 B
S 47 F
2-4 N
S 48 F
2-4 N
S 49 H
2-4 B
S 50 H
2-4 M
S 51 H
2-4 M
S 52 H
2-4 B
S 53 F
2-4 B
S 54 F
2-4 B
S 55 F
2-4 B
S 56 F
2-4 M
S 57 H
2-4 B
S 58 F
2-4 N
S 59 H
2-4 B
S
ES-401 5
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q -
K/A SRO Only 60 F
2-4 N
S 61 F
2-4 B
S 62 H
2-4 N
S 63 F
2-4 B
S 64 F
2-4 B
S 65 F
2-4 N
S 66 F
2-4 N
S 67 F
2-4 N
S 68 F
2-4 M
S 69 F
2-4 B
S 70 F
2-4 B
S 71 H
2-4 N
S 72 F
2-4 N
S 73 F
2-4 B
S 74 H
2-4 N
S 75 F
2-4 N
S 76 H
2-4 B
S
ES-401 6
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q -
K/A SRO Only 77 H
2-4 M
S 78 H
2-4 B
S 79 H
2-4 N
S 80 F
2-4 M
S 81 H
2-4 M
S 82 H
2-4 M
S 83 H
2-4 N
S 84 F
2-4 N
S 85 F
2-4 M
S 86 F
2-4 B
S 87 H
2-4 B
S 88 H
2-4 M
S 89 H
2-4 N
S 90 H
2-4 M
S 91 F
2-4 M
S 92 F
2-4 M
S 93 F
2-4 M
S
ES-401 7
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q -
K/A SRO Only 94 F
2-4 N
S 95 F
2-4 B
S 96 F
2-4 N
S 97 H
2-4 M
S 98 H
2-4 M
S 99 F
2-4 B
S 10 0
H 2-4 N
S