ML20245E563
| ML20245E563 | |
| Person / Time | |
|---|---|
| Site: | Prairie Island  |
| Issue date: | 08/27/2020 |
| From: | Nuclear Management Co |
| To: | NRC/RGN-III/DRS/OLB |
| Callaway G | |
| Shared Package | |
| ML19127A300 | List: |
| References | |
| Download: ML20245E563 (15) | |
Text
ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Prairie Island Exam Date: Aug 17, 2020 1
2 3
Attributes 4
Job Content 5
6 Admin JPMs ADMIN Topic and K/A LOD (1-5)
U/E/S Explanation I/C Cues Critical Scope Overlap Perf.
Key Minutia Job Link Focus Steps (N/B)
Std.
A1A - RO Determine Turbine Startup and Load Time Conduct of OPS 2.1.25 2
E S
Justify the allowed range of values. The chart is in increments of 5 minutes. An accepted accuracy for interpreting analog indication is 1/2 of the smallest increment. The first allowed range matches this (2.5 minutes. The accumulated uncertainty for two readings would be 5 minutes. Explain why the final calculation has an allowed range of 10 minutes.
Facility agrees. Proposes to change final calculation allowed range to 87.5 - 92.5 minutes.
fc-6/25/2020µ SAT GWC 6/26/2020 A1B-RO Determine Blended Flow Concentration Conduct of Ops 2.1.25 2
E S
- 1. Justify why the RO would not be required to find the applicable procedure when directed to determine blended flow concentration.
- 2. Explain the basis for the allowed range (+/- 5%).
A straight line between 67 and 1500 is almost exactly in the center of 30 and 40 on a log scale (33.9).
Taking the calculated value of X and dividing by 15 (the scale of the meter) yields 34.3.
A range of +/-1% seem appropriate.
- 1. RO would be required to find applicable procedure, in the CR; however, low discriminatory value on an admin JPM for an operator to have to recall procedure name/number from memory, in a classroom setting.
- 2. Given previous comment of 1/2 smallest increment allowed accuracy, facility believes +/- 5% is appropriate. (copy of nomograph with potential accuracy error lines provided) fc-6/25/2020
- 1.
I am not asking that the applicant recall the procedure number or name from memory, but to be able to find the procedure from the provided references, just like in the CR. The current cue is not operationally oriented (i.e. not valid)
- 2.
I accept your range justification GWC 6/26/2929 After discussion, this meets the licensees expectations.
SAT GWC 7/29/2020 A2 -RO Equipment Control 2
U S
A daily surveillance is conduct of ops, not equipment control.
ES-301 Operating Test Review Worksheet Form ES-301-7 RWST Level Surveillance 2.2.40 NUREG 1021 gives examples of Equipment control as For example, have the applicant demonstrate how he or she would take a failed system or component out of service, initiate maintenance on the system, and test the system before placing it back in service.
Also, the level of difficulty is quite low.
Facility disagrees; NUREG 1021, ES-301, pg.2 Admin Topics under Equipment Control lists surveillance testing under examples. While this list is not all inclusive, Equipment Control is the only generic section that has a surveillance associated k/a.
fc-6/25/2020 I accept that surveillance testing is appropriate.
The level of difficulty is still unacceptable. Anyone who could meet site access requirements can meet this task. The only ability tested here is the ability to read an analog meter. This JPM does not discriminate between competent and less than competent applicants.
GWC 6/26/2020 Revised SAT GWC 7/29/2020 A3-RO Initiate a Radiation Monitor Out of Service Data Sheet A4 RO Determine Maximum RCS Venting Time Radiation Control 2.3.5 Emergency Plan 2.4.25 2
X U
E S
This does not meet the requirements for a radiation control RO JPM.
NUREG 1021 says, regarding Radiation Control JPMs, The ROs duties generally require knowledge of radiation worker responsibilities and operation of plant systems associated with liquid and gaseous waste releases.
In this JPM, the RO is given two forms and directed to transfer data from one form to another and then apply a sticker. It does not test knowledge of rad worker responsibilities or operation of plant systems associated with releases.
Would prefer (for schedule reasons) to do admin JPMs in classroom.
Facility disagrees. Facility believes this JPM to have high operational validity, high safety significance for CTMT radiation monitors, and high importance for E-Plan. Per previous response, NUREG 1021 ES-301 is a list of examples, but is not an exhaustive list.
We prefer classroom also, which is why we provide the procedure. Well check other on JPM page one if you need us to, no problem; however, there is a cue on last step allowing for performance in simulator or classroom.
Fc-6/25/2020 µ It is the level of difficulty that makes it unacceptable.
The task may be valid, safety-significant and important, but it still does not require knowledge of anything but the English language. Anyone who can
ES-301 Operating Test Review Worksheet Form ES-301-7 meet site access requirements can perform this task.
This JPM does not discriminate between competent and less than competent applicants.
GWC 6/26/2020 Replaced JPM SAT GWC 7/29/2020 A1A - SRO Verify Heavy Load Lift Requirements Conduct of Operations 2.1.23 2
S A1B - SRO Determine Containment CSFST Color Conduct of Operations 2.1.7 1
U S
- 2. Unit 1 has experienced a Large Break LOCA is a conclusion, not an observation. It adds nothing.
- 1. Facility agrees that orange and red paths are RO knowledge; however, this JPM does not test red/orange path entry conditions, but rather if red/orange path exists, which is SRO. With ERCS (plant computer) OOS, a licensed SRO will perform the CSFST. Normally it would be the STA, but we received feedback during previous validation about that and decided that it would raise fewer questions for an applicant if they were listed as extra.
- 2. Facility disagrees. The statement re LBLOCA is required to explain abnormal readings given to candidate; w/o that info we cant justify indications given to candidate; providing candidate with multiple indications of a LOCA and requiring them to first determine if a LOCA has occurred before proceeding with the JPM is not whats being tested; furthermore, NUREG 1021 tells us not to do it.
fc-6/25/2020
- 1.
Please explain which item in 10CFR55.43 applies to make this SRO knowledge. Procedure selection associated with red/orange conditions is specifically excluded in ES-401. If this is based on a site-specific learning objective that is unique to the SRO position, provide the facility learning objective that specifies this.
- 2.
The cause of the abnormal conditions in containment is not relevant to the task. The premise of critical safety functions is that they are not event based, they are safety function based.
GWC 6/26/2020 SRO only Objective provided Change initial assessment to SAT GWC 7/20/2020
ES-301 Operating Test Review Worksheet Form ES-301-7 A2 - SRO Perform Shutdown Safety Assessment Equipment Control 2.2.18 3
E S
In initiating cues, do not say Determine Total Points.
This is the only way to carry out the 2nd bullet. Saying determine current condition is enough.
Cues written this way to ensure examinee performs all aspects of JPM. Without the clear and specific direction, the potential exists for the examinee to count points in their head then simply circle the condition.
While they would satisfy the task standard, they would be unsat on multiple critical steps, steps which are required to determine if examinee fully grasps the knowledge.
Maybe theres an opportunity to re-word the task standard.
fc-6/25/2020 My concern is that the cue is not operationally valid.
What words would most likely be used to assign this task? Determine total points may be more detail than would be expected in assigning the task.
GWC 6/26/2020 Cue revised SAT 7/29/2020 A3 - SRO Authorize Waste Gas Release Radiation Control 2.3.6 3
E S
Change the cue step from Determine if to Complete step 7.11.
Theres a potential psychometric flaw in directing the candidate to complete a step in the ICs where they have to determine whether something is flawed.
Completing steps with no consequence is one thing; however, the candidate may inadvertently believe they must approve the release regardless.
Warrants further discussion perhaps.
fc-6/25/2020 My concern is that the cue is not operationally valid.
What words would most likely be used to assign this task? It seems that the expectation at this point is that the conditions are met and the assigned task would be to complete the procedure steps. determine if seems to be inappropriately leading the applicant.
GWC 6/26/2020 Revised cue SAT GWC 7/29/2020 A4 - SRO Determine Errors on Initial Classification Emergency Procedures
/ Plan 2.4.40 2
S
ES-301 Operating Test Review Worksheet Form ES-301-7 a
Malf of auto makeup during boration 1
004 A4.12 3
E S
The reference to C12.5, AOP 2, Section 2.4.1 seems to be limited to a listing of acceptable ways to stop the boration. None of the symptoms are present and the IFTHEN conditions of the step are not met.
Perhaps a note is appropriate that the applicant is not expected to enter the AOP.
Ill need to get additional clarification on phone call.
fc-6/25/2020 State the actual basis for this performance expectation. The reason the applicant takes these steps is not because AOP 2 says to do them. AOP 2 entry conditions are not met and the IFTHEN conditions of the step are not met.
Basis for action revised SAT GWC 7/29/2020 b
Periodic rotation of charging pumps 2
004 A4.08 3
S c
Energize Przr B/U Htrs and respond to PORV leakage 3
007 A4.10 3
X U
S Double jeopardy with written RO question 2. Both require the applicant to evaluate tailpipe temperature and determine that a PZR PORV is leaking.
Facility disagrees. JPM is testing PRZR PORV leakage response. Written exam question #2, while evaluating temperatures, is testing PRZR safeties, which would look different and requires a different response.
fc-6/25/2020 Agree with feedback. Change initial assessment to SAT GWC 6/26/2020 d
SWS/Loss of cooling water header pressure 4S 075 A4.01 3
S e
Alternate CFCUs w/ CFCLU hi temp 5
022 A4.01 3
S f
Manual start of D1 from CR 6
064 A3.06 2
E S
Add exercising the note prior to step 5.1.1.M (amber lights not lit, speed adjustment necessary)
Would like to compress time in step 5.1.1.I (i.e. not wait 3 minutes) if it has no consequence.
Facility agrees to add note to JPM prior to step 5.1.1.M.
Also, its not intended to wait the 3 minutesthat would be super uncomfortablefor everyone. We can clarify that cue or add a note for time compression if you desire.
ES-301 Operating Test Review Worksheet Form ES-301-7 fc-6/25/2020 µ I am not asking for a note that the applicant may adjust the speed setting. I am asking to change the simulator setup where the lights are not lit and make the step critical to properly adjust the speed setting.
After discussion, it is not practical to have the speed setting off.
SAT GWC 7/29/2020 g
NIS/PR Daily Calibration 7
015 A1.01 2
X E
S I suggest that recording initial and final pot settings are critical steps. If these values are incorrectly recorded, the task has not been properly completed, and it will have to be re-done. Also, this would test the applicants ability to read the POT setting.
Is average reactor thermal power calculated by the plant computer? I would like to require the applicant to obtain this data if it is practical instead of making it part of the cue.
- 1.
Facility agrees. SP1005 steps 8.6.4 & 8.6.6, Record INITIAL/FINAL GAIN SETTING R303 are the initial and final pot settings. These steps are already critical.
- 2.
Normally, yes, average thermal power is calculated by the plant computer (ERCS) calorimetric program (CALM); however, in this case, ERCS is OOS. SP 1005B is the calculation of power when ERCS is OOS, thus the basis for why its given in the cue. Just not practical in this case. SP 1005, NIS Daily Calibration, section 7 is for ERCS in service, 8 is ERCS OOS. (mark-up provided to examinee attached) fc-6/25/2020 µ Change initial assessment to SAT GWC 6/26/2020 Added expected initial reading to evaluator key GWC 7/29/2020 h
Shutdown SFP normal ventilation 8
2.1.20 3
S I
RCP Seal Isolation after loss of all AC 4P 028 AA1.03 3
E S
valves are closed is a conclusion, not an observation.. Give as a cue the indications that should lead the applicant to conclude that the valves are closed.
Facility amenable to making change; will evaluate on Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 j
4S 2.1.23 3
S Cue for closing MV-32031 is incorrect. MV-32031 is closed is a conclusion, not an observation. Give as a
ES-301 Operating Test Review Worksheet Form ES-301-7 Locally close 1 Turb Look a CLG wtr hdr MV cue the indications that should lead the applicant to conclude that the valve is closed.
Facility amenable to making change; will evaluate on Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 k
Restore instrument air compressors 8
068 AA1.21 3
S
ES-301 8
Form ES-301-7 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1.
Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1. (ES-301, D.3 and D.4)
- 2.
Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested.
Mark in column 2 (Appendix D, C.1.f)
- 3.
In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4.
For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6.
In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
ES-301 9
Form ES-301-7 Facility: Prairie Island Scenario: 1 Exam Date: Aug 17, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Secure 11 TD AFW Pump s
- 2. Raise power to POAH s
3- -12 Charging Pump Trip.
s 4 - 11 TD AFWP Accumulator Low Air Pressure X
s 5 - 11 CTMT Vac Bkr fails closed X
E S
Need a better bounding condition. 45 minutes is arbitrary. Let the event run with AFW flow and see if either containment or integrity goes orange.
Containment will go ORANGE path on sump level due to steam condensing. We already have it written into the CT plant conditions that a RED path on Integrity is likely and cannot be prevented by crew actions. In all of our CT testing, weve found that a RED path on Integrity during a faulted SG is inevitable from the cooldown and would not be fair to fail the crew on a CT they cant prevent.
We propose changing the bounding condition to ORANGE path on Containment CSFST.
Copy of proposed change attached fc-6/25/2020 µ SAT GWC 6/26/2020 7 - 12 MD AFW pump fails to auto start X
s 8 - SI to Feed/condensete relay failure s
8 2
2 7
S
ES-301 10 Form ES-301-7 Instructions for Completing This Table:
Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
In column 1, sum the number of events.
In columns 2-4, record the total number of check marks for each column.
In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)
In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
ES-301 11 Form ES-301-7 Facility: Prairie Island Scenario: 2 Exam Date: Aug 17, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Swap running EH oil pump X
S Repeat from 18-2 2 -
Controlling PRZR press chan fails low X
X S
Repeat from 18-4 Repeat from 16-3 3 - D1 Local Alarm X
S 4 - Turbine EH valve malfunction S
5 - Rapid downpower to 50%
S 6 - Loss of all AC X
E S
This bounding condition could not be credibly reached in the time frame of a scenario. If delayed restoration could lead to unnecessary E-Plan escalation, this may be a useful bounding condition.
- 1. We assume youre referring to the AFW restoration CT and SG dryout. If both SG wide range levels reach <50% and total AFW flow is <200 gpm, then RED path on Heat Sink; therefore potential loss of RCS and Fuel Clad barriers = SAE. The loss of offsite and onsite AC power initially would be an ALERT, assuming the crew restores power within 15 minutes. If power isnt restored to both safeguard buses in 15 minutes, then it would be a SAE. Since were not controlling how quickly the crew restores power, they may go over the 15 minutes which would invalidate an E-Plan escalation CT.
- 2. I think youll be more amenable to this (and Im gonna eat more crow for this as well):
- a.
We made a change somewhere along the linebefore the quarantine. When we ran this today, we noticed that this particular LOOP AFW CT doesnt actually apply because the 11 TD AFWP is OOS, sothe crew cant start it. Theyll have to restore power to Bus 16 (CT-6) in order to start 12 MD AFWP. I probably changed it due to overlap concerns with a previous exam but neglected to update the CT.
- b.
Good news is that we have CT that does apply that we didnt put in, again, probably a change and just didnt update it. Its in the body of the scenario, just not the CT table. CT-3A, manually trip the main turbine. Copy of proposed attached
ES-301 12 Form ES-301-7 fc-6/25/2020 µ The bounding condition I was referring to was to restore power prior to ORANGE or RED path on Core Cooling CSF. I think it takes hours to get an overheated core in a station blackout with no AFW. If it is not reasonable to expect the crew to restore power in 15 minutes, then the E-Plan escalation may not be a valid condition.
GWC 6/26/2020 Critical task revised SAT GWC 7/29/2020 7 - Turbine fails to auto trip S
8 - D2 fails to auto start X
S 9 - 12 MD AFW pump fails to auto start X
S 9
2 2
6 S
ES-301 13 Form ES-301-7 Facility: Prairie Island Scenario: 3 Exam Date: Aug 17, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Swap RMU pumps S
2 - 1st stage PI fails low X
X S
Repeat from 18-2 Repeat from 16-2 3 - Restore Tavg to Tref S
4 - PRZR level interlock chan fails low X
S 5 - 11 SGTR 4
S Four critical tasks are associated with a SGTR 6 - 11 & 12 RHR pumps fail to auto start S
7 - SI to cooling water relay signal failure S
S 7
2 4
S
ES-301 14 Form ES-301-7 Facility: Exam Date:
Scenario 1
2 3
4 5
6 7
8 11 Event Totals Events Unsat.
TS Total TS Unsat.
% Unsat.
Scenario Elements U/E/S Explanation 1
8 0
2 0
2 0
0 E
CT bounding condition improvement 2
9 0
2 0
2 0
0 E
CT bounding condition improvement 3
7 0
2 0
4 0
0 S
Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a.
Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b.
TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c.
CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
7 In column 7, calculate the percentage of unsatisfactory scenario elements:
8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
2 + 4 + 6 1 + 3 + 5100%
ES-301 15 Form ES-301-7 Site name: Exam Date:
OPERATING TEST TOTALS Total Total Unsat.
Total Total
% Unsat.
Explanation Edits Sat.
Admin.
JPMs 9
1 5
2 Sim./In-Plant JPMs 11 0
5 6
Scenarios 3
0 2
1 Op. Test Totals:
23 1
12 9
4.3%
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1.
Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs. For scenarios, enter the total number of simulator scenarios.
- 2.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
- 3.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.
- 4.
Total each column and enter the amounts in the Op. Test Totals row.
- 5.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
satisfactory, if the Op. Test Total % Unsat. is 20%
unsatisfactory, if Op. Test Total % Unsat. is > 20%
- 6.
Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:
The JPM performance standards were incorrect.
The administrative JPM tasks/keys were incorrect.
CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
The EOP strategy was incorrect in a scenario(s).
TS entries/actions were determined to be incorrect in a scenario(s).