ML22285A261
| ML22285A261 | |
| Person / Time | |
|---|---|
| Site: | Beaver Valley |
| Issue date: | 03/30/2022 |
| From: | Todd Fish NRC/RGN-I/DORS/OB |
| To: | Gaydosik T Energy Harbor Nuclear Corp |
| References | |
| EPID L-2022-OLL-003 | |
| Download: ML22285A261 (1) | |
Text
Form 2.3 -3 Operating Test Review Worksheet (JPMs)
Facility: Beaver Valley Unit 2 Exam Date: 4/18/22 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL S1 S
Y 3
E Modified initial VCT Level modified to 45%
S2 S
Y 4
E Classified step 5.12 as critical.
S3 S
Y 2
S S4 S
Y 2
S S5 S
Y 3
E Added cue: Additional operators will perform pump and valve stroke timing.
S6 S
N 2
S S7 S
Y 2
S S8 S
N 2
E Added tolerance bands to steps 7, 10, and 15.
P1 P
N 3
S P2 P
N 2
S P3 P
N 3
S RO A 1.1 A
N 3
E Added tolerance bands to steps 1.3, 12.1, 14.1, 17.1, 18.1, 20.1, 21.1 RO A 1.2 A
N 2
E Made step 4.1 non-critical.
RO A 2 A
N 2
S RO A 3 A
N 3
S SRO A 1.1 A
N 3
S SRO A 1.2 A
N 2
S SRO A 2 A
N 2
S SRO A 3 A
N 3
S SRO A 4 A
N 2
S
Form 2.3-3 Instructions for Completing the JPM Table
- 1. Enter the JPM number and/or title.
- 2. Enter the type of JPM(S)imulator, (P)lant, or (A)dministrative.
- 3. Enter (Y)es or (N)o for an Alternate Path JPM.
- 4. Rate the level of difficulty (LOD) of each JPM using a scale of 1-5 (easy-difficult). A JPM containing less than two critical steps, a JPM that tests solely for recall or memorization, or a JPM that involves directly looking up a single correct answer is likely LOD = 1 (too easy).
Conversely, a JPM with over 30 steps or a JPM that takes more than 45 minutes to complete is likely LOD = 5 (too difficult).
- 5. Check the appropriate block for each JPM error type, using the following criteria:
LOD = 1 or 5 is unsatisfactory (U).
REF: The JPM lacks required references, tools, or procedures (U).
IC: The JPM initial conditions are missing or the JPM lacks an adequate initial cue (U).
CUE: The JPM lacks adequate evaluator cues to allow the applicant to complete the task, or the evaluator cues are subjective or leading (U).
TSK: The JPM lacks a task standard or lacks completion criteria for a task standard (U).
CS: The JPM contains errors in designating critical steps, or the JPM lacks an adequate performance standard for a critical step (U).
TL: The JPM validation times are unreasonable, or a time-critical JPM lacks a completion time (U).
- 6. Mark the JPM as unsatisfactory (U), satisfactory (S), or needs enhancements (E). A JPM is (U) if it has one or more (U) errors as determined in step 5. Examples of enhancements include formatting, spelling, or other minor changes.
- 7. Briefly describe any JPM determined to be unsatisfactory (U) or needing enhancement (E). Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a satisfactory (S) resolution on this form.
Form 2.3-3 Operating Test Review Worksheet (Scenarios)
Facility: Beaver Valley Unit 2 Scenario: 1 Exam Date: 4/18/22 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
S 2
S 3
S 4
E Added field report that attempts to locally trip the Main Turbine were unsuccessful.
5 S
6 S
7 S
8 S
9 S
Form 2.3-3 Operating Test Review Worksheet (Scenarios)
Facility: Beaver Valley Unit 2 Scenario: 3 Exam Date: 4/18/22 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
S 2
S 3
S 4
S 5
S 6
S 7
S 8
E Clarified CT-11 to close CNMT valve/penetration before SI Flow terminated.
9 E
Clarified CT-3 extreme challenge to be Red Path in Containment.
10 S
11 S
Form 2.3-3 Operating Test Review Worksheet (Scenarios)
Facility: Beaver Valley Unit 2 Scenario: 4 Exam Date: 4/18/22 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
S 2
E Modified event to remove ambiguity of second PORV failure and modified T/S entry to T/S 3.4.11 Condition B.
3 S
4 S
5 S
6 S
7 S
8 S
Form 2.3-3 Instructions for Completing the Scenario Table
- 1. For each scenario, enter the scenario event names and descriptions.
- 2. Review the individual events contained in each scenario, and identify and mark event errors:
The scenario guide event description is not realistic/credibleunsatisfactory (U).
The scenario guide event description lacks adequate crew/operator performance standardsneeds enhancement (E).
The scenario guide event description lacks verifiable actions for a credited normal event, reactivity event instrument/component malfunction, or technical specification (TS) event (or a combination of these) (U).
The scenario guide event description incorrectly designates an event as a critical task (i.e., a noncritical task labeled as critical or a critical task labeled as noncritical). This includes critical tasks that do not meet the critical task criteria (i.e., the critical task does not have a measurable performance standard) (U).
The scenario guide event description incorrectly designates entry into TS actions when not required or does not designate entry into TS actions when required (U).
- 3. Based on the outcome in step 2, mark the scenario event as unsatisfactory (U), satisfactory (S), or needs enhancements (E). An event is (U) if it has one or more (U) errors as determined in step 2. Examples of enhancements include formatting, spelling, or other minor changes.
- 4. Briefly describe any scenario event determined to be unsatisfactory (U) or needing enhancement (E). Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario event is marked by a satisfactory (S) resolution on this form.
Form 2.3-5 Written Examination Review Worksheet - BV2 - 2022 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 1
H 2.3 Y
RO N
S 2
H 2.0 Y
RO B
S 3
H 3.3 Y
RO B
S 4
F 2.7 Y
RO B
S 5
H 2.3 Y
RO B
S 6
F 3.0 Y
RO N
S 7
F 3.0 Y
RO N
S 8
H 3.0 Y
RO N
S 9
F 2.7 Y
RO B
S 10 H
2.7 Y
RO B
S 11 H
3.0 Y
RO B
S 12 H
3.0 Y
RO B
S 13 H
4.3 Y
RO B
S 14 H
2.3 Y
RO N
S 15 F
2.7 Y
RO N
S 16 H
3.3 Y
RO B
S
Fo18rm 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:
- 1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.
Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.
T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.
Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.
- 4. Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.
Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.
- 5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.
- 6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.
- 7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.
- 8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).
- 9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.
Form 2.3-5 Written Examination Review Worksheet Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 17 H
3.0 Y
RO N
S 18 F
2.7 Y
RO N
S 19 H
3.5 Y
RO N
S 20 H
2.7 Y
RO B
S 21 H
3.0 Y
RO B
S 22 H
3.0 Y
RO N
S 23 F
2.3 Y
RO B
S 24 F
1.3 Y
RO N
S 25 H
3.0 Y
RO B
S 26 F
3.0 Y
RO N
S 27 F
2.3 Y
RO B
S 28 H
3.0 Y
RO B
S 29 H
3.3 Y
RO N
S 30 H
3.0 Y
RO B
S 31 H
2.7 Y
RO B
S 32 H
2.7 Y
RO N
S
Form 2.3-5 Written Examination Review Worksheet Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 33 H
2.5 Y
RO N
S 34 H
3.0 Y
RO B
S 35 H
3.0 Y
RO B
S 36 F
2.7 Y
RO N
S 37 H
3.7 Y
RO B
S 38 H
3.3 Y
RO B
S 39 F
3.0 Y
RO B
S 40 H
3.0 Y
RO B
S 41 H
3.0 Y
RO B
S 42 H
3.0 Y
RO B
S 43 F
2.3 Y
RO N
S 44 F
2.7 Y
RO N
S 45 H
2.3 Y
RO B
S 46 H
3.0 Y
RO B
S 47 H
2.3 Y
RO N
S 48 H
3.3 Y
RO B
S
Form 2.3-5 Written Examination Review Worksheet Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 49 F
2.7 Y
RO B
S 50 F
2.7 Y
RO N
S 51 H
2.7 Y
RO N
S 52 F
2.3 Y
RO N
S 53 F
3.0 Y
RO B
S 54 H
3.0 Y
RO B
S 55 H
2.3 Y
RO B
S 56 H
3.0 Y
RO B
S 57 F
3.3 Y
RO N
S 58 H
2.0 Y
RO N
S 59 H
3.7 Y
RO B
S 60 F
3.3 Y
RO N
S 61 F
3.3 Y
RO N
S 62 H
3.0 Y
RO N
S 63 H
3.0 Y
RO B
S 64 F
3.3 Y
RO B
S
Form 2.3-5 Written Examination Review Worksheet Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 65 F
3.3 Y
RO B
S 66 H
2.7 Y
RO B
S 67 F
2.3 Y
RO B
S 68 F
2.0 Y
RO B
S 69 F
3.0 Y
RO B
S 70 H
2.3 Y
RO B
S 71 F
2.0 Y
RO B
S 72 H
2.3 Y
RO B
S 73 H
3.0 Y
RO B
S 74 F
1.0 Y
RO B
U LOD=1. Selected new KA, and wrote new Q 75 F
2.0 Y
RO B
S
Form 2.3-5 Written Examination Review Worksheet
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 76 H
3.3 Y
SRO N
S 77 F
3.0 Y
SRO B
S 78 H
3.7 Y
SRO B
S 79 H
3.0 Y
SRO B
S 80 F
3.3 Y
SRO N
S 81 H
3.3 Y
SRO B
U Q contains two separate topics: first topic involves actions required for loss of heat sink; second topic involves TS related to SG OPERABILITY.
NUREG ES-4.2, section C.6; Limit the question to one concept or topic, unless the synthesis of the concepts is being tested. (NRC reviewer determined synthesis of the concepts was not being tested.)
Kept KA, randomly selected new system, and replaced Q.
82 H
2.7 Y
SRO N
S 83 H
3.7 Y
SRO B
S 84 H
3.3 Y
SRO B
S 85 F
3.0 Y
SRO N
S 86 H
3.5 Y
SRO B
U Not SRO-only. Question stem tells the SRO what AOP procedure should be implemented, i.e., the Q does not require the SRO to select appropriate procedures. (55.43.b(5), assess
facility condiions and selection of appropriate procedures Kept KA, randomly selected new system, replaced Q.
87 H
3.0 Y
SRO B
S 88 F
3.3 Y
SRO B
S 89 H
3.0 Y
SRO B
S 90 F
3.0 Y
SRO B
S 91 H
3.0 Y
SRO N
S
Form 2.3-5 Written Examination Review Worksheet Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 92 H
3.0 Y
SRO B
S 93 F
3.0 Y
SRO N
S 94 F
3.0 Y
SRO B
S 95 F
3.3 Y
SRO N
S 96 F
3.0 Y
SRO N
S 97 H
2.0 Y
SRO B
S 98 F
3.0 Y
SRO B
S 99 F
3.3 Y
SRO N
S 100 H
3.7 Y
SRO B
S
Beaver Valley Unit 2 2LOT22 Exam Page 1 of 2 Form 4.1-1 Record of Rejected Knowledge and Abilities Refer to Examination Standard (ES)-4.2, Developing Written Examinations, Section B.3, for deviations from the approved written examination outline.
Rev. AG Tier/Group Randomly Selected K/A Reason for Rejection 1/1 000038 EA1.25 Question 9; Unable to write a question which correlates CCW pump ammeter indications during a SGTR as there is no system response or actuations which would cause pump amperage to change with this event. K/A 000038 EA1.30 was randomly selected as a replacement.
2/1 005 K4.12 Question 31; Unable to write a question to meet K/A due to BV2 RHR system does not piggyback with the containment spray system. K/A 005 K4.03 was randomly selected as a replacement.
2/1 007 K5.02 Question 34; Unable to write a question because the Pressurizer Relief Tank is not used during the formation a steam bubble in the pressurizer at Beaver Valley. K/A 007 K3.01 was randomly selected as a replacement.
2/1 059 A1.07 Question 43; Unable to write a question to meet the K/A because BV2 Feed Pumps operate at one speed and do not have an integrated control system. K/A 059 A1.03 was randomly selected as a replacement.
2/1 103 A2.04 Question 54, Unable to write a discriminatory question regarding containment evacuation alarm due to BV2 does not have a separate evacuation alarm. K/A103 A2.03 was randomly selected as a replacement.
2/2 055 A3.03 Question 63; Unable to write a question to meet K/A because BV2 does not have an automatic diversion of CARS exhaust.
Due to the low importance ratings in the Condenser Air Removal System, a new system K/A was selected. K/A 075 K2.03 was randomly selected.
T3 G2.1.9 Question 64; Unable to write a discriminatory question at the RO level because directing personnel inside the control room is an SRO function. G2.1.1 03 was randomly selected as a replacement.
T3 G2.2.21 Question 66; Unable to write a discriminatory question at the RO level because pre-and post-maintenance operability requirements are an SRO task. G2.2.42 was randomly selected as a replacement.
Beaver Valley Unit 2 2LOT22 Exam Page 2 of 2 Form 4.1-1 Record of Rejected Knowledge and Abilities (Continued)
Rev. AG T4 193008 K1.05 Question 74; Question was unsat upon review. Requested new K/A. New system 193007 was selected. K/A is K1.04.
1/1 062 2.4.3 Question 78; Unable to write a question because there is no post-accident instrumentation associated with the Service Water system. K/A 062 2.4.11 was randomly selected as a replacement.
1/2 001 AA2.02 Question 82; Unable to write a question to meet the K/A because the emergency boration valve is not discussed in plant procedures for continuous rod withdrawal procedures.
K/A 001 AA2.03 was randomly selected as a replacement.
2/1 063 A2.02 Question 88; Unable to write a discriminatory question at the SRO level due to loss of battery room ventilation during battery charge is RO knowledge. Selected 063 A2.01 as replacement.
2/2 028 A2.03 Question 92; Unable to write question because the hydrogen recombiner and purge control system has been retired at BV2. K/A 015 A2.02 was randomly selected as a replacement.
1/1 WE05 EA2.1 Question 81; Question was unsat upon review. Requested new K/A selected. New system (WE04) was selected. K/A is WE04 EA2.1.
2/1 012 2.4.8 Question 86; Question was unsat upon review. Requested new K/A selected. New system (003) was selected. K/A is 003 2.4.8.