ML19240A515
| ML19240A515 | |
| Person / Time | |
|---|---|
| Site: | Monticello  |
| Issue date: | 12/06/2018 |
| From: | Joseph Demarshall Nuclear Management Co |
| To: | NRC/RGN-III/DRS/OLB |
| Demarshall J | |
| Shared Package | |
| ML17164A416 | List: |
| References | |
| Download: ML19240A515 (15) | |
Text
ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Monticello Exam Date: October 29 - November 8, 2018 1
2 3
Attributes 4
Job Content 5
6 Admin JPMs ADMIN Topic and K/A LOD (1-5)
U/E/S Explanation I/C Cues Critical Scope Overlap Perf.
Key Minutia Job Link Focus Steps (N/B)
Std.
JPM-OWI-01.08-001 RO Conduct of Ops - 2.1.4 3
S JPM-C.2-05.B.01-002 RO Conduct of Ops - 2.1.19 3
S JPM-4-AWI-04.04.02-004 RO Equipment Control - 2.2.15 2
S JPM-C.5-3403-002 RO Emergency Plan - 2.4.47 3
S JPM-FP-P-A-HU-06-001 SRO Conduct of Ops - 2.1.1 2
S 2016 NRC Exam.
JPM-0133-002 SRO Conduct of Ops
- 2.1.18 3
S JPM-OWI-03.05-001 SRO Equipment Control - 2.2.40 3
X X
E S
NRC: Step 6 - Why are we cueing the applicant to progress through the flowchart.
Recommend removing evaluator cue.
Step 7 should be marked Critical Licensee Response:
Cue removed from Step 6 and Step 7 marked as critical.
NRC: JPM satisfactory following enhancements.
JPM-ODCM-03.01-001 SRO Radiation Control - 2.3.15 2
S JPM-A.2-101-003 SRO Emergency Plan - 2.4.41 3
S Simulator /
In-Plant Safety Function and K/A JPMs
ES-301 Operating Test Review Worksheet Form ES-301-7 JPM-B.01.03-019 1 -201001 -
A4.01 3
S NRC: 2015 NRC Exam.
JPM-B.05.01.02-004 7 - 215005 -
A4.04 3
S NRC: 2015 NRC Exam JPM-B.03.02-005 2 - 206000 -
A4.04 1
2 X
S NRC: 2016 NRC Exam. How does this JPM allow for discriminating between competent and non-competent applicants? LOD=1.
(Note: this JPM is being categorized as E versus U due to prior use on an NRC exam.)
Licensee Response:
The HPCI suction valves do not automatically open when the Torus suctions are closed. If the examinee doesnt manually open the CST suction valve, the Engineered Safety System HPCI will be rendered inoperable. This JPM was a NEW CREATE for the 2016 NRC developed exam and is satisfactory as written.
NRC: JPM satisfactory as written.
JPM-C.4-B.04.01.C-001 5 - 223002 -
A4.02 3
S JPM-B.03.04-011 4 - 205000 -
A4.01 3
X E
S NRC: For consistency among JPMs, add procedure B.03.04-05.D.1 to the initiating cue.
Task standard of Step 3 does not match the procedure (i.e., task standard allows 30 seconds to establish flow while the procedure allows 10 seconds to establish minimum flow or secure the pump).
Terminology in task standards for performance steps 3 and 5 should match procedure terminology for starting/stopping RHR pumps (procedure and ARP do not use equipment part numbers for the handswitches whereas the task standard does).
Why would the applicant utilize Step 1.a of C.6-003-A-50 and Step 1.b? The 11 RHR pump had a lockout, why would the applicant follow a procedure path of the pump not tripping?
Licensee Response:
The procedure number has been added to the initiating cues.
The procedure requirement of 10 seconds is a conservative value. Conferred with
ES-301 Operating Test Review Worksheet Form ES-301-7 Engineering and they suggested that 30 seconds would be an appropriate amount of time to establish flow and not cause pump damage.
Time changed to 10 seconds.
The terminology in steps 3 and 5 has been revised to match.
The JPM does NOT call for the performance of step 1.b of C.6-003-A-50.
NRC: Steps to secure 11 RHR Pump in accordance with ARP C.6-003-A-50 have been removed from the JPM. JPM satisfactory following enhancements.
JPM-0112-001 3 - 239002 -
A2.03 3
S NRC: Provide marked up copy of Test No.
0112.
Licensee Response:
A marked up copy of Test 0112 will be provided.
JPM-E.4-12-002 6 - 262001 -
S4.01 3
S JPM-B.07.02.02-001 9 - 271000 -
A4.09 3
S NRC: Why is the applicant being given the tank to release in initial conditions rather than determining it from Form 2168? Why not let the applicant determine that from the Form, as they would normally do?
Licensee Response:
During validation, the LOR validators stated that Form 2168 would typically be reviewed by the CRS and they would direct which tanks to be released.
NRC: JPM satisfactory as written.
JPM-B.01.03-003 1 - 201003 -
A2.02 1
2 X
U S
NRC: How does this JPM allow for discriminating between competent and non-competent applicants? LOD=1.
Licensee Response:
This JPM is requiring the operator to take mitigating actions for an uncoupled control rod. If the control rod is not disarmed it could be withdrawn mistakenly potentially creating a CRDA situation. Locating the proper HCU, disconnecting 4 amphenol plugs and properly capping them appropriately does discriminate between a competent operator and one who is not. JPM is satisfactory as written.
ES-301 Operating Test Review Worksheet Form ES-301-7 Replaced this JPM with JPM-B.01.03-003 which involves HCU hydraulic isolation only.
NRC: JPM-B.01.03-002 replaced with JPM-B.01.03-003. Replacement JPM satisfactory.
JPM-C.5-3401-002 5 - 223001 -
A1.08 2
S JPM-B.08.05-003 8 - 286000 -
A1.01 3
X S
NRC: Step 6 states that the applicant should perform pump prestart checks per a checklist, but one is not found in procedure B.08.05-05.
How is this to be evaluated?
Licensee Response:
The pump prestart checklist is a hardcard posted locally to be used. JPM is satisfactory as written.
ES-301 5
Form ES-301-7 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1.
Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.
(ES-301, D.3 and D.4)
- 2.
Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)
- 3.
In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4.
For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6.
In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
ES-301 6
Form ES-301-7 Facility: Monticello Scenario: 1 Exam Date: October 29 - November 8, 2018 1
2 3
4 5
6 7
8 9
10 Event Realism
/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Place the ESOP in service S
2 - Control rod drifting in X
S 3 - RBCCW pump trip X
S 2015 NRC exam, scenario 4, event 2.
4 - Inadvertent initiation of RCIC X
S 5 - 15 bus lockout resulting in loss of CRD flow requiring Rx SCRAM X
E S
NRC: No bounding conditions for CT-1. See App D - D.1.c the performance standard for a CT includes two parts: 1) expected actions, 2) safety-significant boundary conditions that clearly identify at what point a CT must be accomplished Licensee Response: Bounding conditions added to body of ES-D-2 (Simulator Exercise Guide (SEG)). Added bounding statement of within 20 minutes of receiving the 2nd accumulator alarm.
NRC: Confirmed addition of bounding conditions to the ES-D-2.
6 - LOCA with loss of RFPs XX X
E S
2016 NRC exam, scenario 3, event 5.
NRC: No bounding conditions for CT-22. See App D - D.1.c the performance standard for a CT includes two parts: 1) expected actions, 2) safety-significant boundary conditions that clearly identify at what point a CT must be accomplished Licensee Response: Bounding conditions added to body of ES-D-2 (SEG). Prior to exceeding cooldown rate.
NRC: Confirmed addition of bounding conditions to the ES-D-2.
7 - Failure of 12 core spray pump to auto start S
7 Events 0
0 0
0 2
3 5
E S
NRC: Must define bounding conditions for CT-1 and CT-22 or the task cannot be considered critical and the minimum CTs for the scenario are not met.
Licensee Response: Bounding conditions added to body of ES-D-2 (SEG). See above.
NRC: Confirmed addition of bounding conditions to the ES-D-2.
ES-301 7
Form ES-301-7 Facility: Monticello Scenario: 2 Exam Date: October 29 - November 8, 2018 1
2 3
4 5
6 7
8 9
10 Event Realism/
Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Transfer RPV level control to first MFW Reg Valve S
2 - Withdraw control rods for startup X
S 2016 NRC exam, scenario 2, event 1 3 - Stuck control rod X
S 2016 NRC exam, scenario 2, event 1 4 - Core spray valve leakage X
S NRC: Would prefer to trigger event 4 on lead examiner direction, rather than trigger at a power level.
Licensee Response: This ensures all crews perform the same up-power reactivity manipulation. Event 4 changed to manual trigger.
NRC: Confirmed that the event will be triggered upon direction of the Lead Examiner.
5 - RBM upscale / INOP failure X
S 2015 NRC exam, scenario 2, event 2 6 - Drywell CAM Trouble / Faulty Sample Pump X
U S
MNGP: Added Drywell Cam Trouble to allow for LCO entry and TS call.
Previous TS for RBM failure (Event 5) was not an active LCO.
NRC: Confirmed that replacement TS requires LCO entry. Table line item revised to reflect the finalized ES-D-1 for S2, Event 6.
7 - Operational bases EQ with a trip of #11 service water pump S
NRC: Table line item revised to reflect the finalized ES-D-1 for S2, Event 7.
8 - Design bases EQ requiring SCRAM - Unisolable torus leak XX E
S NRC: No bounding conditions for CT-10. See App D - D.1.c the performance standard for a CT includes two parts: 1) expected actions, 2) safety-significant boundary conditions that clearly identify at what point a CT must be accomplished Licensee Response: Bounding conditions added to body of ES-D-2 (SEG).
NRC: Table line item revised to reflect the finalized ES-D-1 for S2, Event 8.
Confirmed addition of bounding conditions to the ES-D-2.
9 - LFRV fail to operate in automatic S
NRC: Table line item revised to reflect the finalized ES-D-1 for S2, Event 9.
9 Events 0
0 0
0 2
2 5
E S
NRC: Must define bounding conditions for CT-10 or the task cannot be considered critical and the minimum CTs for the scenario are not met.
Licensee Response: Bounding conditions added to body of ES-D-2 (SEG).
See above.
NRC: Confirmed that replacement TS (Event 6) requires LCO entry.
Confirmed addition of bounding conditions to the ES-D-2.
ES-301 8
Form ES-301-7 Facility: Monticello Scenario: 3 Exam Date: October 29 - November 8, 2018 1
2 3
4 5
6 7
8 9
10 Event Realism/
Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Test 0141 RB-Torus vacuum breaker check X
S 2 - MFRV A lockup S
3 - SJAE steam supply PCV fails closed X
S 2016 NRC exam, scenario 3, event 4.
4 - Rapid power reduction for degrading main condenser vacuum S
5 - Inadvertent ADS timer initiation X
X S
2015 NRC exam, scenario 3, event 4 2016 NRC exam, scenario 4, event 3 6 - Network load increase requiring a reactor SCRAM with Hydraulic ATWS XX E
S NRC: The List of Events under Scenario Overview on Page 6 of 29, and the detailed Event descriptions in the ES-D-2 provided with the submittal, do not accurately reflect the current ES-D-1 for Scenario 3 and is a change from what was originally discussed and agreed upon. There should only be 7 events total (as specified in the current ES-D-1), not 8 events. Events 6 and 7 together constitute the Major Event (i.e., Event 6 on the current ES-D-1). What is listed as Event 8 on Page 6 of 29 (RWCU Failure to Isolate) should actually be Event 7.
Licensee Response: Scenario overview and ES-D-2 have been corrected.
NRC: Table line item revised to reflect the finalized ES-D-1 for S3, Event 6.
Confirmed that the scenario overview and ES-D-2 have been revised to reflect the finalized ES-D-1.
7 - RWCU fails to isolate S
NRC: See Event 7 comments above regarding the correct number and assignment of events that comprise Scenario 3.
Licensee Response: Scenario overview and ES-D-2 have been corrected.
NRC: Table line item revised to reflect the finalized ES-D-1 for S3, Event 7.
7 Events 0
0 0
0 2
2 5
E S
ES-301 9
Form ES-301-7 Facility: Monticello Scenario: 4 Exam Date: October 29 - November 8, 2018 1
2 3
4 5
6 7
8 9
10 Event Realism/
Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Restore HPCI X
S 2016 NRC exam, scenario 4, event 1 2 - Inadvertent HPCI initiation X
X E
S 2016 NRC exam, scenario 3, event 3 NRC: The List of Events under Scenario Overview on Page 6 of 26, does not reflect either the current ES-D-1 for Scenario 4 or the actual Event listings in the ES-D-2 provided with the submittal. The Scenario Overview omits Event 2, Inadvertent HPCI Initiation. Accordingly, the Scenario 4 ES-301-7 Table has been constructed to reflect the current ES-D-1 for Scenario 4 and the actual Event listings in the ES-D-2 provided with the submittal.
Licensee Response: Scenario Overview has been corrected.
NRC: Confirmed that the scenario overview has been revised to reflect the finalized ES-D-1 and the Event listings in the ES-D-2.
3 - Runback of 11 recirc pump (Rapid power reduction)
X X
S 2015 NRC exam, scenario 4, event 4 (Rapid power reduction to exit the unanalyzed region of the P-F Map is the overlap item) 4 - RMCS normal insertion failure X
S 2015 NRC exam, scenario 4, event 4 5 - Runback of 12 recirc pump with neutron flux oscillations - SCRAM X
E S
NRC: No bounding conditions for this CT. See App D - D.1.c the performance standard for a CT includes two parts: 1) expected actions, 2) safety-significant boundary conditions that clearly identify at what point a CT must be accomplished.
Licensee Response: Bounding conditions added to body of ES-D-2 (SEG).
NRC: Confirmed addition of bounding conditions to the ES-D-2.
6 - SCRAM discharge volume (SDV) fails to isolate requiring ED X
X E
S 2015 NRC exam, scenario 2, event 6 NRC: No bounding conditions for this CT. See App D - D.1.c the performance standard for a CT includes two parts: 1) expected actions, 2) safety-significant boundary conditions that clearly identify at what point a CT must be accomplished.
Licensee Response: Bounding conditions added to body of ES-D-2 (SEG).
NRC: Confirmed addition of bounding conditions to the ES-D-2.
7 - SBGT fails to initiate S
NRC: Note that Scenario 4 on the ES-301-5 Forms for Crews 1, 3, and 4, each specify Event 8 instead of Event 7 for the BOP and SRO-I positions.
ES-301 10 Form ES-301-7 These forms will need to be revised to specify Event 7 (there are only 7 events total in Scenario 4).
Licensee Response: ES-301-5 has been corrected.
NRC: Confirmed the corrections to the ES-301-5 Forms.
7 Events 0
0 0
0 2
2 2
E S
NRC: Must define bounding conditions for CT-8 AND CT-33, or the task cannot be considered critical and the minimum CTs for the scenario are not met.
Licensee Response: Bounding conditions added to body of ES-D-2 (SEG).
NRC: Confirmed addition of bounding conditions to the ES-D-2.
ES-301 11 Form ES-301-7 Facility: Monticello Scenario: 5 (spare) Exam Date: October 29 - November 8, 2018 1
2 3
4 5
6 7
8 9
10 Event Realism/
Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation
ES-301 12 Form ES-301-7 Instructions for Completing This Table:
Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
In column 1, sum the number of events.
In columns 2-4, record the total number of check marks for each column.
In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)
In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
ES-301 13 Form ES-301-7 Facility: Monticello Exam Date: October 29 - November 8, 2018 Scenario 1
2 3
4 5
6 7
8 11 Event Totals Events Unsat.
TS Total TS Unsat.
% Unsat.
Scenario Elements U/E/S Explanation 1
7 0
2 0
3 0
0 E
S NRC: Must define bounding conditions for CT-1 and CT-22 or the t.ask cannot be considered critical and the minimum CTs for the scenario are not met (Enhancement)
NRC: CT bounding conditions defined. Scenario satisfactory.
2 9
0 2
1 2
0 7.7 E
S NRC: Must define bounding conditions for CT-10 or the task cannot be considered critical and the minimum CTs for the scenario are not met (Enhancement).
TS for RBM failure did not require LCO entry.
NRC: Confirmed that replacement TS requires LCO entry. CT bounding conditions defined.
Scenario satisfactory.
3 7
0 2
0 2
0 0
E S
NRC: Scenario overview and ES-D-2 do not reflect the current ES-D-1.
NRC: Scenario overview and ES-D-2 have been revised to reflect the finalized ES-D-1.
Scenario satisfactory.
4 7
0 2
0 2
0 0
E S
NRC: Must define bounding conditions for CT-8 AND CT-33, or the task cannot be considered critical and the minimum CTs for the scenario are not met (Enhancement).
NRC: CT bounding conditions defined. Scenario satisfactory.
5 Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a.
Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b.
TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c.
CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
7 In column 7, calculate the percentage of unsatisfactory scenario elements:
8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
2 + 4 + 6 1 + 3 + 5100%
ES-301 15 Form ES-301-7 Site name: Monticello Exam Date: October 29 - November 8, 2018 OPERATING TEST TOTALS Total Total Unsat.
Total Total Unsat.
Explanation Edits Sat.
Admin.
JPMs 9
0 1
8 Sim./In-Plant JPMs 11 1
1 9
Submitted In-plant JPM-B.01.03-002 was LOD=1.
JPM was replaced.
Scenarios 5
0 5
0 Op. Test Totals:
25 1
7 17 4
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1.
Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.
For scenarios, enter the total number of simulator scenarios.
- 2.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
- 3.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.
- 4.
Total each column and enter the amounts in the Op. Test Totals row.
- 5.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
- satisfactory, if the Op. Test Total % Unsat. is 20%
- unsatisfactory, if Op. Test Total % Unsat. is > 20%
- 6.
Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:
- The JPM performance standards were incorrect.
- The administrative JPM tasks/keys were incorrect.
- CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
- The EOP strategy was incorrect in a scenario(s).
- TS entries/actions were determined to be incorrect in a scenario(s).