ML23179A128
| ML23179A128 | |
| Person / Time | |
|---|---|
| Site: | Calvert Cliffs  |
| Issue date: | 06/14/2023 |
| From: | Thomas Setzer NRC/RGN-I/DORS/OB |
| To: | Constellation Energy Company |
| References | |
| EPID L2023-OLL-0008 | |
| Download: ML23179A128 (1) | |
Text
Form 2.3-3 Operating Test Review Worksheet (JPMs) 2023 Calvert Cliffs Operating Exam - Administered week of June 5, 2023 Facility: Calvert Cliffs Exam Date: June 5, 2023 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL RO Admin 1 - Calculate readings from RPS A
N 2
E
- Changed tolerance for step 1 to 5 vice 10 RO Admin 2 - Calculate Main Generator Hydrogen Leak Rate A
N 2
E Are acceptable band tolerances for the calculations reasonable? NO
- Changed tolerance to 50
- Typo on the initial conditions - shown vs show
- Added more detail in task standard RO Admin 3 - Evaluate AFW Pump performance data during a Surveillance Test A
N 3
E Are acceptable band tolerances for the calculations reasonable? YES Added more detail to the task standard RO Admin 4 - Determine Response to a Security Event A
N 2
S To be filed in Non-Public ADAMS
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Facility: Calvert Cliffs Exam Date: June 5, 2023 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL SRO Admin 1 -
Reactivation of SRO License A
N 3
S SRO Admin 2 - Determine reportability requirements A
N 2
E Task Standard says form needs to be filled out.
To nonspecific and ambiguous. Added more detail to task standard SRO Admin 3 -
Evaluate and then independently verify AFW Pump performance data during a Surveillance Test, then determine Tech Spec actions A
N 2
E Changed validation time to 20 minutes SRO Admin 4 - Review and Authorize KI A
N 3
S SRO Admin 5 -
Determine Emergency Action Level A
N 3
S
- a. SIM-1 Dropped CEA during Startup S
Y 2
E Added group 5 to what can be selected on CEAPDS
Y 2
E
- Added cue earlier for location of the leak
- Minor typo on page 7
- Made isolating letdown not critical
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Facility: Calvert Cliffs Exam Date: June 5, 2023 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL
- c.
SIM-3 Prepare for RAS Actuation S
Y 2
E
- Created overrides for ECCS fans
- d. SIM-4 Restore Shutdown Cooling S
Y 2
E
- Changed the tolerance to 100 gpm.
- e. SIM-5 Respond to a condensate or feedwater rupture at power S
N 2
S
- f.
SIM-6 Bleed and Feed to Cool the Quench Tank S
N 3
E Task Standard lacks specificity - added more detail. Also corrected typo in cue.
- g. SIM-7 Respond to a loss of 11 4KV Bus S
Y 3
E Are all actions of AOP 7I critical? No, added more specificity to the task standard.
- Added an initial condition for grid stable.
- Deleted steps after 6.1.b
- h. SIM-8 Shift Control Room Ventilation S
N 3
E Are all actions of OI-22F critical? No, added specificity to the Task Standard.
Form 2.3-3 Operating Test Review Worksheet (JPMs)
Facility: Calvert Cliffs Exam Date: June 5, 2023 1
JPM # or title 2
Type (S/P/A) 3 ALT (Y/N) 4 LOD (1-5) 5 JPM Errors 6
U/E/S 7
Explanation LOD REF IC TSK CUE CS TL
- a. PLT-1 Bypass SIAS Sensor Modules P
N 3
S
- b. PLT-2 Startup Hydrogen Purge P
N 3
S
- c. PLT-3 Secure the MG Sets P
N 3
S
Form 2.3-3 Instructions for Completing the JPM Table
- 1. Enter the JPM number and/or title.
- 2. Enter the type of JPM(S)imulator, (P)lant, or (A)dministrative.
- 3. Enter (Y)es or (N)o for an Alternate Path JPM.
- 4. Rate the level of difficulty (LOD) of each JPM using a scale of 1-5 (easy-difficult). A JPM containing less than two critical steps, a JPM that tests solely for recall or memorization, or a JPM that involves directly looking up a single correct answer is likely LOD = 1 (too easy). Conversely, a JPM with over 30 steps or a JPM that takes more than 45 minutes to complete is likely LOD = 5 (too difficult).
- 5. Check the appropriate block for each JPM error type, using the following criteria:
LOD = 1 or 5 is unsatisfactory (U).
REF: The JPM lacks required references, tools, or procedures (U).
IC: The JPM initial conditions are missing or the JPM lacks an adequate initial cue (U).
CUE: The JPM lacks adequate evaluator cues to allow the applicant to complete the task, or the evaluator cues are subjective or leading (U).
TSK: The JPM lacks a task standard or lacks completion criteria for a task standard (U).
CS: The JPM contains errors in designating critical steps, or the JPM lacks an adequate performance standard for a critical step (U).
TL: The JPM validation times are unreasonable, or a time-critical JPM lacks a completion time (U).
- 6. Mark the JPM as unsatisfactory (U), satisfactory (S), or needs enhancements (E). A JPM is (U) if it has one or more (U) errors as determined in step 5. Examples of enhancements include formatting, spelling, or other minor changes.
- 7. Briefly describe any JPM determined to be unsatisfactory (U) or needing enhancement (E). Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a satisfactory (S) resolution on this form.
Form 2.3-3 Operating Test Review Worksheet (Scenarios)
Facility: Calvert Cliffs Scenario: 1 (contingency EOP)
Exam Date: June 5, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
N-SRO R-ATC/BOP S
2 C-BOP/SRO TS-SRO S
TS 3
C-BOP/SRO S
4 C-ALL MC-ATC TS-SRO S
TS 5
C-ATC/SRO S
6 C-BOP/SRO MC-BOP S
CT-1 7
C-ALL S
CT-2 8
M-ALL S
CT-3 SAT Other minor edits:
- Removed the if necessary for starting 11 AFW pump on page 16
Form 2.3-3 Operating Test Review Worksheet (Scenarios)
Facility: Calvert Cliffs Scenario: 2 (SPARE)
Exam Date: June 5, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
I-ATC/SRO MC-ATC S
2 C-BOP/SRO TS-SRO S
TS 3
C-ATC/SRO TS-SRO S
TS 4
R-ATC/BOP C-SRO S
5 C-ALL S
6 M-ALL S
CT-1 7
C-BOP/SRO MC-BOP S
CT-1 8
M-ALL S
CT-3 SAT Other minor edits:
- Added in RCP trip per the curves or CIS.
- Added AOP-2A entry for Event 3.
- Added in EOP-0 RLEC could be not met due to trends.
Form 2.3-3 Operating Test Review Worksheet (Scenarios)
Facility: Calvert Cliffs Scenario: 3 Exam Date: June 5, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
I-BOP/SRO TS-SRO S
TS 2
C-ATC/SRO TS-SRO S
TS 3
R-ATC/BOP C-SRO S
4 C-ALL S
5 MC-BOP C-BOP/SRO S
6 M-ALL S
CT-1, CT-2 7
MC-ATC I-ATC/SRO S
CT-3 SAT Other minor edits:
- Page 14, removed the bold for BD-4012
- Changed the wording of the 3rd Critical Task
Form 2.3-3 Operating Test Review Worksheet (Scenarios)
Facility: Calvert Cliffs Scenario: 4 (contingency EOP)
Exam Date: June 5, 2023 1
Scenario Event ID/Name:
2 Scenario event errors 3
U/E/S 4
Explanation Realism/
Credibility Performance Standards Verifiable Actions Critical Task TS 1
C-BOP/SRO MC-BOP S
TS 2
C-BOP/SRO TS-SRO S
TS 3
R-ATC C-BOP/SRO TS-SRO S
4 C-ALL S
5 C-ATC/SRO MC-ATC S
CT-1 6
C-ALL S
CT-2 7
M-ALL S
CT-3 SAT Other minor edits:
- Added in that RLEC could be not met due to loss of power to the main vent RMS pump.
`
Form 2.3-3 Instructions for Completing the Scenario Table
- 1. For each scenario, enter the scenario event names and descriptions.
- 2. Review the individual events contained in each scenario, and identify and mark event errors:
The scenario guide event description is not realistic/credibleunsatisfactory (U).
The scenario guide event description lacks adequate crew/operator performance standardsneeds enhancement (E).
The scenario guide event description lacks verifiable actions for a credited normal event, reactivity event instrument/component malfunction, or technical specification (TS) event (or a combination of these) (U).
The scenario guide event description incorrectly designates an event as a critical task (i.e., a noncritical task labeled as critical or a critical task labeled as noncritical). This includes critical tasks that do not meet the critical task criteria (i.e., the critical task does not have a measurable performance standard) (U).
The scenario guide event description incorrectly designates entry into TS actions when not required or does not designate entry into TS actions when required (U).
- 3. Based on the outcome in step 2, mark the scenario event as unsatisfactory (U), satisfactory (S), or needs enhancements (E). An event is (U) if it has one or more (U) errors as determined in step 2. Examples of enhancements include formatting, spelling, or other minor changes.
- 4. Briefly describe any scenario event determined to be unsatisfactory (U) or needing enhancement (E). Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario event is marked by a satisfactory (S) resolution on this form.
Form 2.3-5 Written Examination Review Worksheet 2023 Calvert Cliffs Written Exam - Administered June 14, 2023 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level 1
H 3
B S
2 F
2 B
E Dons comment on 2nd round was that this did not match the K/A. Calvert Cliffs defines SDC and RHR (ECCS) as the same system, so K/A is matched well enough 3
H 2
X X
B U
U E
Is part 1 SRO level (procedure selection)?
IS there any direction in AOP-2B for loss of 1 charging pump, where then B1 and C1 would be correct?
Plausibility discusses one CHG pump in operation showing signs of gas binding. In this case wouldnt you enter this AOP for 1 CHG pump in service? Stem doesnt say if other pumps are in operation.
Rewrote question, new question does not address loss of reactor coolant makeup per the K/A.
Changed AOP-2B to OI-2A-1and made 2nd bullet to describe trip of a charging pump. Reframed stem to describe a loss of reactor coolant makeup to better match the K/A 4
H 2
B S
Is IX ion exchanger? YES 5
F 1
B U
Question too easy for Tier 3, is GFE. Redraw K/A and rewrite.
Redrew K/A and rewrote. New question SAT.
6 H
2 B
E Part 2 seems trivial LOD1, only tests adding times accurately. Rewrote question to add a SFP event at 0800 7
F 3
X X
B U
K/A mismatch and True/False statements - only need to know valve lift setpoints and nothing about vapor space accidents. Need more detail on an accident instead of basic RCS overpressure event.
Rewrote question, SAT 8
H 3
B S
Form 2.3-5 Written Examination Review Worksheet 9
H 2
B E
Tier 4 - will edit question to have more operationally valid units and transmitters Made PORV downstream an actual gauge reading 10 F
2 X
N U
Why cant I sent the OSO? This seems subjectiveyou could send anyone to investigate.
Edited to discuss locations instead of who will be sent.
Rewrote question, SAT.
11 H
3 B
S 12 F
3 B
S 13 H
3 N
S Tier 4. Grammar in part 2 is questionable. OK as written.
14 H
3 B
S
`15 F
2 B
S 16 H
3 B
S 17 F
3 B
S 18 F
2 X
B U
This doesnt test automatic features of the CCWS.
What automatic features in CCWS would an RO monitor? K/A should have been eliminated on initial outline as there are no automatic features in CCWS to test.
Redrew K/A and rewrote question. New question SAT.
19 H
3 N
E Removed ONLY in part 1s 20 H
3 B
S 21 H
3 N
E Question seems easy but is not LOD1. A1 and B1 are not the best choices as distractors since the stem says SDC is in service, and you would not be at that high of a pressure. Changed A1 and B1 to 250 psia.
22 H
3 X
N U
K/A - is this a Loss of MFW? NO CC enters Loss of MFW AOP-3G for events that also include excessive feed. This doesnt seem to meet the intent of the K/A for Loss of MFW.
Rewrote question. New question SAT.
23 F
2 B
S Tier 4, very easy question but not LOD1 24 H
3 N
S
Form 2.3-5 Written Examination Review Worksheet 25 F
3 X
N U
C and D are implausible. These systems do not provide flow into the core.
Changed those systems to SIAS and RAS.
26 H
3 B
S 27 F
2 B
E Tier 4, edited to be more plant specific 28 H
3 B
E Edited B and C to eliminate the term available.
29 F
2 B
S Previous exam question (2021 NRC RO exam) 30 F
3 N
E K1 generic D reads like a none of the above. Need a distractor for D. Rewrote distractor D. Added image of containment dome and rx cavity temps.
`31 H
2 N
S 32 F
3 B
E Should D say 11 SFP cooler?
Changed D to HX
Form 2.3-5 Written Examination Review Worksheet 33 F
2 N
E Added on shift to part 2 to be within site procedures for the duties of the extra licensed RO. Fixed typos in A and B and made clear that this was IAW site procedures.
34 H
2 B
S 35 H
2 B
S 36 H
2 B
S 37 F
2 B
S 38 F
2 B
S 39 H
3 N
S 40 F
2 B
S 41 F
3 B
E Too plant specific for Tuer 3. Rewrote question 42 H
2 X
N U
K/A not met - A2how is procedures incorporated?
Need a part 2 where they use procedures to correct, control, mitigate. There is no procedure use for this as the actions occur automatically.
Added in accordance with the alarm manual and verify/ensure 43 H
3 N
S 44 H
3 B
S DLU? Worked thru reference with licensee, question is not DLU 45 F
2 B
S Easy question but not LOD1 46 H
3 N
E Need to state where must the breaker be locally operated. Also questioned if part (1) is procedurally driven (yes)
`47 F
1.5 B
S 48 H
3 B
S
Form 2.3-5 Written Examination Review Worksheet 49 H
3 B
S 50 H
3 B
E Eliminate 1st sentence and reword 2nd if needed. 1st sentence provides unneeded direction that the applicant should already know. Edited stem not to discuss the TBO exclusively.
51 H
3 B
S 52 F
3 N
S Easy question, but not LOD1 53 H
3 B
S 54 F
2 N
E Tagging K/A - rewrote distractors so 3 is not in 3 out of 4 distractors. 3 is the least likely right answer, so as written eliminating it arrives at the correct answer easily.
55 F
3 X
N U
K/A mismatch, backwards logic Rewrote question, new question sat 56 F
2 N
E Reworded part 2 to better grammar.
57 H
3 B
S 58 H
2 B
E Reworded A2 and C2. There is not a strong connection at Calvert Cliffs between subcooling and SDM.
59 H
3 B
S 60 H
2 B
S 61 H
3 B
S Previous exam question (2021 NRC RO exam) 62 H
2 N
S Does this test predicting impacts on AC system? Yes
`63 F
3 B
S 64 F
2 X
N U
A is none of the above. Tests rote memorization, rewrite.
Rewrote distractors 65 F
3 B
S 66 F
2 N
S
Form 2.3-5 Written Examination Review Worksheet 67 H
3 N
S 68 H
2 N
S 69 H
3 B
S 70 H
2 B
S 71 H
3 N
E A2 and C2 are not the best choices in a core uncover so are easily eliminated as wrong. Reworded A2 and C2.
72 F
2 N
S 73 F
3 B
E Tier 4 - Reworded question to be more challenging in defining a saturated liquid 74 F
3 B
S 75 F
3 X
N U
Rote memorization. Write a question that is more operationally valid, which in a diagnosis of plant parameters makes the candidate determine the power supply for RPS.
New question SAT 76 H
2 B
S Are we providing the entire procedure as a reference for this question? Is the alarm manual also being provided? If not, is this fair from memory? SAT 77 F
3 B
S 78 H
2 B
S 79 F
3 N
S 80 H
2 N
E Do you need to tell them Containment Barrier in the stem? Changed question to test what barrier is lost.
Rewritten question needs edit - removed bullet about the ABO hears a noise.
`81 F
3 B
S 82 H
2 B
S 83 H
3 B
S 84 H
2 N
S
Form 2.3-5 Written Examination Review Worksheet 85 H
3 B
E What are the bold strikethroughs on the reference TS for? Removed if any from stem.
Form 2.3-5 Written Examination Review Worksheet 86 F
3 X
N U
E Rewrite to SRO level where they select the procedure and direct activities, instead of giving the AOP in the stem.
Rewrote question to test procedure selection.
Rewritten question needs edit - change A1 and B1 to CO2 as Halon is not very plausible.
87 F
2 B
S 88 F
3 B
E D is not very plausible based on the reference provided, C stands out as correct based on 72 hrs + 6 hrs. Rewrote D to be plausible based on the TS reference.
89 H
2 X
N U
E Question is DLU, dont need to be an SRO to be able to navigate the reference.
Redrew K/A and rewrote question, new question needs enhancement, has grammar error in stem part
- 2. Edited stem to correct, SAT 90 H
3 B
S 91 F
3 X
N U
DLU Redrew K/A, new question SAT 92 H
3 N
S 93 F
2 N
S 94 H
3 B
S 95 H
2 N
S 96 F
3 N
S
Form 2.3-5 Written Examination Review Worksheet 97 H
3 N
S
`98 H
3 B
S 99 F
2 B
S 100 F
3 B
E Minor edit in stem