ML20245E563: Difference between revisions

From kanterella
Jump to navigation Jump to search
(StriderTol Bot insert)
 
(StriderTol Bot change)
 
Line 16: Line 16:


=Text=
=Text=
{{#Wiki_filter:ES-301                                                             Operating Test Review Worksheet                                                       Form ES-301-7 Facility: Prairie Island       Exam Date: Aug 17, 2020 1         2                               3                               4       5                                 6 ADMIN                                    Attributes                    Job Content LOD Admin     JPMs       Topic and                                                                             U/E/S                         Explanation (1-5)  I/C         Critical Scope             Perf.
{{#Wiki_filter:ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Prairie Island Exam Date: Aug 17, 2020 1
K/A                                                                            Job Cues                    Overlap      Key Minutia Focus         Steps     (N/B)           Std.               Link Justify the allowed range of values. The chart is in increments of 5 minutes. An accepted accuracy for interpreting analog indication is 1/2 of the smallest increment. The first allowed range matches this (2.5 minutes. The accumulated uncertainty for two A1A - RO Conduct of                                                                                    readings would be 5 minutes. Explain why the final Determine                                                                                                    E OPS        2                                                                              calculation has an allowed range of 10 minutes.
2 3
Turbine Startup                                                                                                S 2.1.25 and Load Time Facility agrees. Proposes to change final calculation allowed range to 87.5 - 92.5 minutes.
Attributes 4
fc- 6/25/2020µ SAT GWC 6/26/2020
Job Content 5
6 Admin JPMs ADMIN Topic and K/A LOD (1-5)
U/E/S Explanation I/C Cues Critical Scope Overlap Perf.
Key Minutia Job Link Focus Steps (N/B)
Std.
A1A - RO Determine Turbine Startup and Load Time Conduct of OPS 2.1.25 2
E S
Justify the allowed range of values. The chart is in increments of 5 minutes. An accepted accuracy for interpreting analog indication is 1/2 of the smallest increment. The first allowed range matches this (2.5 minutes. The accumulated uncertainty for two readings would be 5 minutes. Explain why the final calculation has an allowed range of 10 minutes.
Facility agrees. Proposes to change final calculation allowed range to 87.5 - 92.5 minutes.
fc-6/25/2020µ SAT GWC 6/26/2020 A1B-RO Determine Blended Flow Concentration Conduct of Ops 2.1.25 2
E S
: 1. Justify why the RO would not be required to find the applicable procedure when directed to determine blended flow concentration.
: 1. Justify why the RO would not be required to find the applicable procedure when directed to determine blended flow concentration.
: 2. Explain the basis for the allowed range (+/- 5%).
: 2. Explain the basis for the allowed range (+/- 5%).
Line 25: Line 35:
Taking the calculated value of X and dividing by 15 (the scale of the meter) yields 34.3.
Taking the calculated value of X and dividing by 15 (the scale of the meter) yields 34.3.
A range of +/-1% seem appropriate.
A range of +/-1% seem appropriate.
: 1. RO would be required to find applicable procedure, in the CR; however, low discriminatory value on an admin JPM for an operator to have to recall A1B- RO Conduct of                                                                                      procedure name/number from memory, in a Determine                                                                                                    E Ops        2                                                                                classroom setting.
: 1. RO would be required to find applicable procedure, in the CR; however, low discriminatory value on an admin JPM for an operator to have to recall procedure name/number from memory, in a classroom setting.
Blended Flow                                                                                                  S 2.1.25                                                                                    2. Given previous comment of 1/2 smallest increment Concentration allowed accuracy, facility believes +/- 5% is appropriate. (copy of nomograph with potential accuracy error lines provided) fc-6/25/2020
: 2. Given previous comment of 1/2 smallest increment allowed accuracy, facility believes +/- 5% is appropriate. (copy of nomograph with potential accuracy error lines provided) fc-6/25/2020
: 1. I am not asking that the applicant recall the procedure number or name from memory, but to be able to find the procedure from the provided references, just like in the CR. The current cue is not operationally oriented (i.e. not valid)
: 1.
: 2. I accept your range justification GWC 6/26/2929 After discussion, this meets the licensees expectations.
I am not asking that the applicant recall the procedure number or name from memory, but to be able to find the procedure from the provided references, just like in the CR. The current cue is not operationally oriented (i.e. not valid)
SAT GWC 7/29/2020 Equipment                                                                               U   A daily surveillance is conduct of ops, not equipment A2 -RO                          2 Control                                                                                S    control.
: 2.
I accept your range justification GWC 6/26/2929 After discussion, this meets the licensees expectations.
SAT GWC 7/29/2020 A2 -RO Equipment Control 2
U S
A daily surveillance is conduct of ops, not equipment control.  


ES-301                         Operating Test Review Worksheet                                           Form ES-301-7 RWST Level       2.2.40                                         NUREG 1021 gives examples of Equipment control as Surveillance                                                    For example, have the applicant demonstrate how he or she would take a failed system or component out of service, initiate maintenance on the system, and test the system before placing it back in service.
ES-301 Operating Test Review Worksheet Form ES-301-7 RWST Level Surveillance 2.2.40 NUREG 1021 gives examples of Equipment control as For example, have the applicant demonstrate how he or she would take a failed system or component out of service, initiate maintenance on the system, and test the system before placing it back in service.
Also, the level of difficulty is quite low.
Also, the level of difficulty is quite low.
Facility disagrees; NUREG 1021, ES-301, pg.2 Admin Topics under Equipment Control lists surveillance testing under examples. While this list is not all inclusive, Equipment Control is the only generic section that has a surveillance associated k/a.
Facility disagrees; NUREG 1021, ES-301, pg.2 Admin Topics under Equipment Control lists surveillance testing under examples. While this list is not all inclusive, Equipment Control is the only generic section that has a surveillance associated k/a.
fc-6/25/2020 I accept that surveillance testing is appropriate.
fc-6/25/2020 I accept that surveillance testing is appropriate.
The level of difficulty is still unacceptable. Anyone who could meet site access requirements can meet this task. The only ability tested here is the ability to read an analog meter. This JPM does not discriminate between competent and less than competent applicants.
The level of difficulty is still unacceptable. Anyone who could meet site access requirements can meet this task. The only ability tested here is the ability to read an analog meter. This JPM does not discriminate between competent and less than competent applicants.
GWC 6/26/2020 Revised SAT GWC 7/29/2020 This does not meet the requirements for a radiation control RO JPM.
GWC 6/26/2020 Revised SAT GWC 7/29/2020 A3-RO Initiate a Radiation Monitor Out of Service Data Sheet A4 RO Determine Maximum RCS Venting Time Radiation Control 2.3.5 Emergency Plan 2.4.25 2
X U
E S
This does not meet the requirements for a radiation control RO JPM.
NUREG 1021 says, regarding Radiation Control JPMs, The ROs duties generally require knowledge of radiation worker responsibilities and operation of plant systems associated with liquid and gaseous waste releases.
NUREG 1021 says, regarding Radiation Control JPMs, The ROs duties generally require knowledge of radiation worker responsibilities and operation of plant systems associated with liquid and gaseous waste releases.
In this JPM, the RO is given two forms and directed to transfer data from one form to another and then apply a sticker. It does not test knowledge of rad worker responsibilities or operation of plant systems A3-RO                                                        associated with releases.
In this JPM, the RO is given two forms and directed to transfer data from one form to another and then apply a sticker. It does not test knowledge of rad worker responsibilities or operation of plant systems associated with releases.
Initiate a                                                    Would prefer (for schedule reasons) to do admin Radiation Radiation Monitor                                                  JPMs in classroom.
Would prefer (for schedule reasons) to do admin JPMs in classroom.
Control Out of Service                                                  U 2.3.5 Data Sheet                2                                X E Facility disagrees. Facility believes this JPM to have Emergency A4 RO                                                      S high operational validity, high safety significance for Plan Determine                                                      CTMT radiation monitors, and high importance for E-2.4.25 Maximum RCS                                                      Plan. Per previous response, NUREG 1021 ES-301 is Venting Time                                                    a list of examples, but is not an exhaustive list.
Facility disagrees. Facility believes this JPM to have high operational validity, high safety significance for CTMT radiation monitors, and high importance for E-Plan. Per previous response, NUREG 1021 ES-301 is a list of examples, but is not an exhaustive list.
We prefer classroom also, which is why we provide the procedure. Well check other on JPM page one if you need us to, no problem; however, there is a cue on last step allowing for performance in simulator or classroom.
We prefer classroom also, which is why we provide the procedure. Well check other on JPM page one if you need us to, no problem; however, there is a cue on last step allowing for performance in simulator or classroom.
Fc-6/25/2020 µ It is the level of difficulty that makes it unacceptable.
Fc-6/25/2020 µ It is the level of difficulty that makes it unacceptable.
The task may be valid, safety-significant and important, but it still does not require knowledge of anything but the English language. Anyone who can
The task may be valid, safety-significant and important, but it still does not require knowledge of anything but the English language. Anyone who can  


ES-301                     Operating Test Review Worksheet                                       Form ES-301-7 meet site access requirements can perform this task.
ES-301 Operating Test Review Worksheet Form ES-301-7 meet site access requirements can perform this task.
This JPM does not discriminate between competent and less than competent applicants.
This JPM does not discriminate between competent and less than competent applicants.
GWC 6/26/2020 Replaced JPM SAT GWC 7/29/2020 A1A - SRO Conduct of Verify Heavy Operations 2                                 S Load Lift 2.1.23 Requirements
GWC 6/26/2020 Replaced JPM SAT GWC 7/29/2020 A1A - SRO Verify Heavy Load Lift Requirements Conduct of Operations 2.1.23 2
S A1B - SRO Determine Containment CSFST Color Conduct of Operations 2.1.7 1
U S
: 1. Red and Orange path entry conditions are RO knowledge according to ES-401.
: 1. Red and Orange path entry conditions are RO knowledge according to ES-401.
: 2. Unit 1 has experienced a Large Break LOCA is a conclusion, not an observation. It adds nothing.
: 2. Unit 1 has experienced a Large Break LOCA is a conclusion, not an observation. It adds nothing.
: 3. You are an extra SRO in the control room adds nothing and does not make it an SRO JPM.
: 3. You are an extra SRO in the control room adds nothing and does not make it an SRO JPM.
: 1. Facility agrees that orange and red paths are RO knowledge; however, this JPM does not test red/orange path entry conditions, but rather if red/orange path exists, which is SRO. With ERCS (plant computer) OOS, a licensed SRO will perform the CSFST. Normally it would be the STA, but we received feedback during previous validation about that and decided that it would raise fewer questions for an applicant if they were listed as extra.
: 1. Facility agrees that orange and red paths are RO knowledge; however, this JPM does not test red/orange path entry conditions, but rather if red/orange path exists, which is SRO. With ERCS (plant computer) OOS, a licensed SRO will perform the CSFST. Normally it would be the STA, but we received feedback during previous validation about that and decided that it would raise fewer questions for an applicant if they were listed as extra.
: 2. Facility disagrees. The statement re LBLOCA is required to explain abnormal readings given to A1B - SRO                                                      candidate; w/o that info we cant justify indications Conduct of Determine                                                U    given to candidate; providing candidate with Operations 1 Containment                                              S    multiple indications of a LOCA and requiring them 2.1.7 CSFST Color                                                    to first determine if a LOCA has occurred before proceeding with the JPM is not whats being tested; furthermore, NUREG 1021 tells us not to do it.
: 2. Facility disagrees. The statement re LBLOCA is required to explain abnormal readings given to candidate; w/o that info we cant justify indications given to candidate; providing candidate with multiple indications of a LOCA and requiring them to first determine if a LOCA has occurred before proceeding with the JPM is not whats being tested; furthermore, NUREG 1021 tells us not to do it.
fc-6/25/2020
fc-6/25/2020
: 1. Please explain which item in 10CFR55.43 applies to make this SRO knowledge. Procedure selection associated with red/orange conditions is specifically excluded in ES-401. If this is based on a site-specific learning objective that is unique to the SRO position, provide the facility learning objective that specifies this.
: 1.
: 2. The cause of the abnormal conditions in containment is not relevant to the task. The premise of critical safety functions is that they are not event based, they are safety function based.
Please explain which item in 10CFR55.43 applies to make this SRO knowledge. Procedure selection associated with red/orange conditions is specifically excluded in ES-401. If this is based on a site-specific learning objective that is unique to the SRO position, provide the facility learning objective that specifies this.
GWC 6/26/2020 SRO only Objective provided Change initial assessment to SAT GWC 7/20/2020
: 2.
The cause of the abnormal conditions in containment is not relevant to the task. The premise of critical safety functions is that they are not event based, they are safety function based.
GWC 6/26/2020 SRO only Objective provided Change initial assessment to SAT GWC 7/20/2020  


ES-301                       Operating Test Review Worksheet                                       Form ES-301-7 In initiating cues, do not say Determine Total Points.
ES-301 Operating Test Review Worksheet Form ES-301-7 A2 - SRO Perform Shutdown Safety Assessment Equipment Control 2.2.18 3
E S
In initiating cues, do not say Determine Total Points.
This is the only way to carry out the 2nd bullet. Saying determine current condition is enough.
This is the only way to carry out the 2nd bullet. Saying determine current condition is enough.
Cues written this way to ensure examinee performs all aspects of JPM. Without the clear and specific direction, the potential exists for the examinee to count points in their head then simply circle the condition.
Cues written this way to ensure examinee performs all aspects of JPM. Without the clear and specific direction, the potential exists for the examinee to count points in their head then simply circle the condition.
While they would satisfy the task standard, they would A2 - SRO                                                    be unsat on multiple critical steps, steps which are Equipment Perform                                                  E required to determine if examinee fully grasps the Control  3 Shutdown Safety                                              S knowledge.
While they would satisfy the task standard, they would be unsat on multiple critical steps, steps which are required to determine if examinee fully grasps the knowledge.
2.2.18 Assessment                                                    Maybe theres an opportunity to re-word the task standard.
Maybe theres an opportunity to re-word the task standard.
fc-6/25/2020 My concern is that the cue is not operationally valid.
fc-6/25/2020 My concern is that the cue is not operationally valid.
What words would most likely be used to assign this task? Determine total points may be more detail than would be expected in assigning the task.
What words would most likely be used to assign this task? Determine total points may be more detail than would be expected in assigning the task.
GWC 6/26/2020 Cue revised SAT 7/29/2020 Change the cue step from Determine if to Complete step 7.11.
GWC 6/26/2020 Cue revised SAT 7/29/2020 A3 - SRO Authorize Waste Gas Release Radiation Control 2.3.6 3
E S
Change the cue step from Determine if to Complete step 7.11.
Theres a potential psychometric flaw in directing the candidate to complete a step in the ICs where they have to determine whether something is flawed.
Theres a potential psychometric flaw in directing the candidate to complete a step in the ICs where they have to determine whether something is flawed.
Completing steps with no consequence is one thing; however, the candidate may inadvertently believe they must approve the release regardless.
Completing steps with no consequence is one thing; however, the candidate may inadvertently believe they must approve the release regardless.
A3 - SRO      Radiation E Warrants further discussion perhaps.
Warrants further discussion perhaps.
Authorize Waste    Control  3 S fc-6/25/2020 Gas Release        2.3.6 My concern is that the cue is not operationally valid.
fc-6/25/2020 My concern is that the cue is not operationally valid.
What words would most likely be used to assign this task? It seems that the expectation at this point is that the conditions are met and the assigned task would be to complete the procedure steps. determine if seems to be inappropriately leading the applicant.
What words would most likely be used to assign this task? It seems that the expectation at this point is that the conditions are met and the assigned task would be to complete the procedure steps. determine if seems to be inappropriately leading the applicant.
GWC 6/26/2020 Revised cue SAT GWC 7/29/2020 A4 - SRO     Emergency Determine Errors Procedures 2                                S on Initial     / Plan Classification    2.4.40
GWC 6/26/2020 Revised cue SAT GWC 7/29/2020 A4 - SRO Determine Errors on Initial Classification Emergency Procedures
/ Plan 2.4.40 2
S


ES-301                         Operating Test Review Worksheet                                             Form ES-301-7 The reference to C12.5, AOP 2, Section 2.4.1 seems to be limited to a listing of acceptable ways to stop the boration. None of the symptoms are present and the IFTHEN conditions of the step are not met.
ES-301 Operating Test Review Worksheet Form ES-301-7 a
Malf of auto makeup during boration 1
004 A4.12 3
E S
The reference to C12.5, AOP 2, Section 2.4.1 seems to be limited to a listing of acceptable ways to stop the boration. None of the symptoms are present and the IFTHEN conditions of the step are not met.
Perhaps a note is appropriate that the applicant is not expected to enter the AOP.
Perhaps a note is appropriate that the applicant is not expected to enter the AOP.
a Malf of auto        1                                      E Ill need to get additional clarification on phone call.
Ill need to get additional clarification on phone call.
3 makeup during    004 A4.12                                    S fc-6/25/2020 boration State the actual basis for this performance expectation. The reason the applicant takes these steps is not because AOP 2 says to do them. AOP 2 entry conditions are not met and the IFTHEN conditions of the step are not met.
fc-6/25/2020 State the actual basis for this performance expectation. The reason the applicant takes these steps is not because AOP 2 says to do them. AOP 2 entry conditions are not met and the IFTHEN conditions of the step are not met.
Basis for action revised SAT GWC 7/29/2020 b
Basis for action revised SAT GWC 7/29/2020 b
Periodic rotation     2 3                                S of charging   004 A4.08 pumps Double jeopardy with written RO question 2. Both require the applicant to evaluate tailpipe temperature and determine that a PZR PORV is leaking.
Periodic rotation of charging pumps 2
c                                                      Facility disagrees. JPM is testing PRZR PORV Energize Przr                                                  leakage response. Written exam question #2, while 3                                      U B/U Htrs and              3          X                        evaluating temperatures, is testing PRZR safeties, 007 A4.10                                    S respond to PORV                                                  which would look different and requires a different leakage                                                    response.
004 A4.08 3
S c
Energize Przr B/U Htrs and respond to PORV leakage 3
007 A4.10 3
X U
S Double jeopardy with written RO question 2. Both require the applicant to evaluate tailpipe temperature and determine that a PZR PORV is leaking.
Facility disagrees. JPM is testing PRZR PORV leakage response. Written exam question #2, while evaluating temperatures, is testing PRZR safeties, which would look different and requires a different response.
fc-6/25/2020 Agree with feedback. Change initial assessment to SAT GWC 6/26/2020 d
fc-6/25/2020 Agree with feedback. Change initial assessment to SAT GWC 6/26/2020 d
SWS/Loss of         4S 3                                S cooling water   075 A4.01 header pressure e
SWS/Loss of cooling water header pressure 4S 075 A4.01 3
Alternate CFCUs         5 3                                  S w/ CFCLU hi     022 A4.01 temp Add exercising the note prior to step 5.1.1.M (amber lights not lit, speed adjustment necessary)
S e
Alternate CFCUs w/ CFCLU hi temp 5
022 A4.01 3
S f
Manual start of D1 from CR 6
064 A3.06 2
E S
Add exercising the note prior to step 5.1.1.M (amber lights not lit, speed adjustment necessary)
Would like to compress time in step 5.1.1.I (i.e. not wait 3 minutes) if it has no consequence.
Would like to compress time in step 5.1.1.I (i.e. not wait 3 minutes) if it has no consequence.
f 6                                      E Manual start of            2                                  Facility agrees to add note to JPM prior to step 064 A3.06                                    S D1 from CR                                                    5.1.1.M.
Facility agrees to add note to JPM prior to step 5.1.1.M.
Also, its not intended to wait the 3 minutesthat would be super uncomfortablefor everyone. We can clarify that cue or add a note for time compression if you desire.
Also, its not intended to wait the 3 minutesthat would be super uncomfortablefor everyone. We can clarify that cue or add a note for time compression if you desire.  


ES-301                           Operating Test Review Worksheet                                               Form ES-301-7 fc-6/25/2020 µ I am not asking for a note that the applicant may adjust the speed setting. I am asking to change the simulator setup where the lights are not lit and make the step critical to properly adjust the speed setting.
ES-301 Operating Test Review Worksheet Form ES-301-7 fc-6/25/2020 µ I am not asking for a note that the applicant may adjust the speed setting. I am asking to change the simulator setup where the lights are not lit and make the step critical to properly adjust the speed setting.
After discussion, it is not practical to have the speed setting off.
After discussion, it is not practical to have the speed setting off.
SAT GWC 7/29/2020 I suggest that recording initial and final pot settings are critical steps. If these values are incorrectly recorded, the task has not been properly completed, and it will have to be re-done. Also, this would test the applicants ability to read the POT setting.
SAT GWC 7/29/2020 g
NIS/PR Daily Calibration 7
015 A1.01 2
X E
S I suggest that recording initial and final pot settings are critical steps. If these values are incorrectly recorded, the task has not been properly completed, and it will have to be re-done. Also, this would test the applicants ability to read the POT setting.
Is average reactor thermal power calculated by the plant computer? I would like to require the applicant to obtain this data if it is practical instead of making it part of the cue.
Is average reactor thermal power calculated by the plant computer? I would like to require the applicant to obtain this data if it is practical instead of making it part of the cue.
: 1. Facility agrees. SP1005 steps 8.6.4 & 8.6.6, Record INITIAL/FINAL GAIN SETTING R303 are the initial and final pot settings. These steps are already critical.
: 1.
g 7                                        E  2. Normally, yes, average thermal power is NIS/PR Daily                  2 X 015 A1.01                                      S        calculated by the plant computer (ERCS)
Facility agrees. SP1005 steps 8.6.4 & 8.6.6, Record INITIAL/FINAL GAIN SETTING R303 are the initial and final pot settings. These steps are already critical.
Calibration calorimetric program (CALM); however, in this case, ERCS is OOS. SP 1005B is the calculation of power when ERCS is OOS, thus the basis for why its given in the cue. Just not practical in this case. SP 1005, NIS Daily Calibration, section 7 is for ERCS in service, 8 is ERCS OOS. (mark-up provided to examinee attached) fc-6/25/2020 µ Change initial assessment to SAT GWC 6/26/2020 Added expected initial reading to evaluator key GWC 7/29/2020 h
: 2.
8 Shutdown SFP                 3                                  S 2.1.20 normal ventilation valves are closed is a conclusion, not an observation. . Give as a cue the indications that should lead the applicant to conclude that the valves I
Normally, yes, average thermal power is calculated by the plant computer (ERCS) calorimetric program (CALM); however, in this case, ERCS is OOS. SP 1005B is the calculation of power when ERCS is OOS, thus the basis for why its given in the cue. Just not practical in this case. SP 1005, NIS Daily Calibration, section 7 is for ERCS in service, 8 is ERCS OOS. (mark-up provided to examinee attached) fc-6/25/2020 µ Change initial assessment to SAT GWC 6/26/2020 Added expected initial reading to evaluator key GWC 7/29/2020 h
4P                                          are closed.
Shutdown SFP normal ventilation 8
RCP Seal                                                      E 028    3 Isolation after                                                S AA1.03                                          Facility amenable to making change; will evaluate on loss of all AC Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 4S                                           Cue for closing MV-32031 is incorrect. MV-32031 is j                  3                                    S 2.1.23                                        closed is a conclusion, not an observation. Give as a
2.1.20 3
S I
RCP Seal Isolation after loss of all AC 4P 028 AA1.03 3
E S
valves are closed is a conclusion, not an observation.. Give as a cue the indications that should lead the applicant to conclude that the valves are closed.
Facility amenable to making change; will evaluate on Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 j
4S 2.1.23 3
S Cue for closing MV-32031 is incorrect. MV-32031 is closed is a conclusion, not an observation. Give as a  


ES-301                   Operating Test Review Worksheet                                         Form ES-301-7 Locally close 1                                           cue the indications that should lead the applicant to Turb Look a CLG                                            conclude that the valve is closed.
ES-301 Operating Test Review Worksheet Form ES-301-7 Locally close 1 Turb Look a CLG wtr hdr MV cue the indications that should lead the applicant to conclude that the valve is closed.
wtr hdr MV Facility amenable to making change; will evaluate on Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 k
Facility amenable to making change; will evaluate on Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 k
8 Restore 068  3                                S instrument air AA1.21 compressors
Restore instrument air compressors 8
068 AA1.21 3
S


ES-301                                                                                                 8                                                                               Form ES-301-7 Instructions for Completing This Table:
ES-301 8
Form ES-301-7 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
: 1. Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1. (ES-301, D.3 and D.4)
: 1.
: 2. Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested.
Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1. (ES-301, D.3 and D.4)
: 2.
Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested.
Mark in column 2 (Appendix D, C.1.f)
Mark in column 2 (Appendix D, C.1.f)
: 3. In column 3, Attributes, check the appropriate box when an attribute is not met:
: 3.
In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
Line 115: Line 176:
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
: 4. For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
: 4.
For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
: 5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
: 5.
: 6. In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
: 6.
In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.  


ES-301                                                                       9                                                                             Form ES-301-7 Facility: Prairie Island                               Scenario: 1       Exam Date: Aug 17, 2020 1                 2           3       4         5       6 7     8       9                                                   10 Required Verifiable                  Scen.
ES-301 9
Event         Realism/Cred.                       LOD     TS CTs         U/E/S                                             Explanation Actions  actions                    Overlap 1 - Secure 11 s
Form ES-301-7 Facility: Prairie Island Scenario: 1 Exam Date: Aug 17, 2020 1
TD AFW Pump
2 3
: 2. Raise power s
4 5
to POAH 3- -12 Charging s
6 7
Pump Trip.
8 9
4 - 11 TD AFWP Accumulator Low                                                   X                 s Air Pressure 5 - 11 CTMT Vac Bkr fails                                                     X                 s closed Need a better bounding condition. 45 minutes is arbitrary. Let the event run with AFW flow and see if either containment or integrity goes orange.
10 Event Realism/Cred.
Containment will go ORANGE path on sump level due to steam condensing. We already have it written into the CT plant conditions that a RED path on Integrity is likely and cannot be prevented by crew actions. In all of our CT testing, weve found that a RED path on E
Required Actions Verifiable actions LOD TS CTs Scen.
X                    Integrity during a faulted SG is inevitable from the cooldown and would not be fair to fail S
Overlap U/E/S Explanation 1 - Secure 11 TD AFW Pump s
the crew on a CT they cant prevent.
: 2. Raise power to POAH s
3- -12 Charging Pump Trip.
s 4 - 11 TD AFWP Accumulator Low Air Pressure X
s 5 - 11 CTMT Vac Bkr fails closed X
s 6 - Faulted 12 SG to CTMT X
E S
Need a better bounding condition. 45 minutes is arbitrary. Let the event run with AFW flow and see if either containment or integrity goes orange.
Containment will go ORANGE path on sump level due to steam condensing. We already have it written into the CT plant conditions that a RED path on Integrity is likely and cannot be prevented by crew actions. In all of our CT testing, weve found that a RED path on Integrity during a faulted SG is inevitable from the cooldown and would not be fair to fail the crew on a CT they cant prevent.
We propose changing the bounding condition to ORANGE path on Containment CSFST.
We propose changing the bounding condition to ORANGE path on Containment CSFST.
Copy of proposed change attached fc-6/25/2020 µ 6 - Faulted 12                                                                            SAT SG to CTMT                                                                                GWC 6/26/2020 7 - 12 MD AFW pump fails to                                                       X             s auto start 8 - SI to Feed/condensete                                                                     s relay failure 8                                     2 2     7       S
Copy of proposed change attached fc-6/25/2020 µ SAT GWC 6/26/2020 7 - 12 MD AFW pump fails to auto start X
s 8 - SI to Feed/condensete relay failure s
8 2
2 7
S  


ES-301                                                                                               10                                                                         Form ES-301-7 Instructions for Completing This Table:
ES-301 10 Form ES-301-7 Instructions for Completing This Table:
Use this table for each scenario for evaluation.
Use this table for each scenario for evaluation.
2     Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4   In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5     Check this box if the level of difficulty is not appropriate.
5 Check this box if the level of difficulty is not appropriate.
6     Check this box if the event has a TS.
6 Check this box if the event has a TS.
7     Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8     Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9     Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10     Record any explanations of the events here.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
In the shaded boxes, sum the number of check marks in each column.
In column 1, sum the number of events.
In column 1, sum the number of events.
In columns 2-4, record the total number of check marks for each column.
In columns 2-4, record the total number of check marks for each column.
In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)
In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)
In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.  


ES-301                                                                     11                                                                               Form ES-301-7 Facility: Prairie Island                           Scenario: 2           Exam Date: Aug 17, 2020 1               2         3       4       5       6   7     8       9                                                     10 Required Verifiable                    Scen.
ES-301 11 Form ES-301-7 Facility: Prairie Island Scenario: 2 Exam Date: Aug 17, 2020 1
Event       Realism/Cred.                     LOD     TS   CTs           U/E/S                                             Explanation Actions  actions                    Overlap 1 - Swap running EH                                                             X       S     Repeat from 18-2 oil pump 2-                                                                                     Repeat from 18-4 Controlling                                                                            Repeat from 16-3 PRZR press                                                  X          X        S chan fails low 3 - D1 X                   S Local Alarm 4 - Turbine EH valve                                                                       S malfunction 5 - Rapid downpower                                                                       S to 50%
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Swap running EH oil pump X
S Repeat from 18-2 2 -
Controlling PRZR press chan fails low X
X S
Repeat from 18-4 Repeat from 16-3 3 - D1 Local Alarm X
S 4 - Turbine EH valve malfunction S
5 - Rapid downpower to 50%
S 6 - Loss of all AC X
E S
This bounding condition could not be credibly reached in the time frame of a scenario. If delayed restoration could lead to unnecessary E-Plan escalation, this may be a useful bounding condition.
This bounding condition could not be credibly reached in the time frame of a scenario. If delayed restoration could lead to unnecessary E-Plan escalation, this may be a useful bounding condition.
: 1. We assume youre referring to the AFW restoration CT and SG dryout. If both SG wide range levels reach <50% and total AFW flow is <200 gpm, then RED path on Heat Sink; therefore potential loss of RCS and Fuel Clad barriers = SAE. The loss of offsite and onsite AC power initially would be an ALERT, assuming the crew restores power within 15 minutes. If power isnt restored to both safeguard buses in 15 minutes, then it would be a SAE. Since were not controlling how quickly the crew restores power, they may E        go over the 15 minutes which would invalidate an E-Plan escalation CT.
: 1. We assume youre referring to the AFW restoration CT and SG dryout. If both SG wide range levels reach <50% and total AFW flow is <200 gpm, then RED path on Heat Sink; therefore potential loss of RCS and Fuel Clad barriers = SAE. The loss of offsite and onsite AC power initially would be an ALERT, assuming the crew restores power within 15 minutes. If power isnt restored to both safeguard buses in 15 minutes, then it would be a SAE. Since were not controlling how quickly the crew restores power, they may go over the 15 minutes which would invalidate an E-Plan escalation CT.
X S    2. I think youll be more amenable to this (and Im gonna eat more crow for this as well):
: 2. I think youll be more amenable to this (and Im gonna eat more crow for this as well):
: a. We made a change somewhere along the linebefore the quarantine. When we ran this today, we noticed that this particular LOOP AFW CT doesnt actually apply because the 11 TD AFWP is OOS, sothe crew cant start it. Theyll have to restore power to Bus 16 (CT-6) in order to start 12 MD AFWP. I probably changed it due to overlap concerns with a previous exam but neglected to update the CT.
: a.
: b. Good news is that we have CT that does apply that we didnt put in, again, probably a change and just didnt update it. Its in the body of the scenario, just 6 - Loss of                                                                                    not the CT table. CT-3A, manually trip the main turbine. Copy of proposed all AC                                                                                          attached
We made a change somewhere along the linebefore the quarantine. When we ran this today, we noticed that this particular LOOP AFW CT doesnt actually apply because the 11 TD AFWP is OOS, sothe crew cant start it. Theyll have to restore power to Bus 16 (CT-6) in order to start 12 MD AFWP. I probably changed it due to overlap concerns with a previous exam but neglected to update the CT.
: b.
Good news is that we have CT that does apply that we didnt put in, again, probably a change and just didnt update it. Its in the body of the scenario, just not the CT table. CT-3A, manually trip the main turbine. Copy of proposed attached  


ES-301               12                                                                       Form ES-301-7 fc-6/25/2020 &#xb5; The bounding condition I was referring to was to restore power prior to ORANGE or RED path on Core Cooling CSF. I think it takes hours to get an overheated core in a station blackout with no AFW. If it is not reasonable to expect the crew to restore power in 15 minutes, then the E-Plan escalation may not be a valid condition.
ES-301 12 Form ES-301-7 fc-6/25/2020 &#xb5; The bounding condition I was referring to was to restore power prior to ORANGE or RED path on Core Cooling CSF. I think it takes hours to get an overheated core in a station blackout with no AFW. If it is not reasonable to expect the crew to restore power in 15 minutes, then the E-Plan escalation may not be a valid condition.
GWC 6/26/2020 Critical task revised SAT GWC 7/29/2020 7 - Turbine fails to auto           S trip 8 - D2 fails X    S to auto start 9 - 12 MD AFW pump X    S fails to auto start 9 2 2 6   S
GWC 6/26/2020 Critical task revised SAT GWC 7/29/2020 7 - Turbine fails to auto trip S
8 - D2 fails to auto start X
S 9 - 12 MD AFW pump fails to auto start X
S 9
2 2
6 S  


ES-301                                                                       13                                                             Form ES-301-7 Facility: Prairie Island                           Scenario:   3           Exam Date: Aug 17, 2020 1               2         3       4       5       6     7     8       9                                                   10 Required Verifiable                    Scen.
ES-301 13 Form ES-301-7 Facility: Prairie Island Scenario: 3 Exam Date: Aug 17, 2020 1
Event     Realism/Cred.                     LOD     TS   CTs         U/E/S                                             Explanation Actions  actions                      Overlap 1 - Swap S
2 3
RMU pumps 2 - 1st stage                                                                         Repeat from 18-2 X          X      S PI fails low                                                                          Repeat from 16-2 3 - Restore S
4 5
Tavg to Tref 4 - PRZR level interlock                                                   X                  S chan fails low 5 - 11                                                                                 Four critical tasks are associated with a SGTR 4            S SGTR 6 - 11 & 12 RHR pumps S
6 7
fail to auto start 7 - SI to cooling water relay                                                                     S signal failure S
8 9
7                                     2   4             S
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1 - Swap RMU pumps S
2 - 1st stage PI fails low X
X S
Repeat from 18-2 Repeat from 16-2 3 - Restore Tavg to Tref S
4 - PRZR level interlock chan fails low X
S 5 - 11 SGTR 4
S Four critical tasks are associated with a SGTR 6 - 11 & 12 RHR pumps fail to auto start S
7 - SI to cooling water relay signal failure S
S 7
2 4
S  


ES-301                                                                                                 14                                                                             Form ES-301-7 Facility:                                                                         Exam Date:
ES-301 14 Form ES-301-7 Facility: Exam Date:
1           2         3         4       5       6         7           8                                                       11 Scenario                                                                        % Unsat.                                                           Explanation Event      Events      TS         TS       CT      CT Scenario    U/E/S Totals      Unsat. Total     Unsat. Total  Unsat.
Scenario 1
Elements 1             8         0         2         0         2       0         0             E                                         CT bounding condition improvement 2             9         0         2         0         2       0         0           E                                         CT bounding condition improvement 3             7         0         2         0         4       0         0           S Instructions for Completing This Table:
2 3
4 5
6 7
8 11 Event Totals Events Unsat.
TS Total TS Unsat.
CT Total CT Unsat.  
% Unsat.
Scenario Elements U/E/S Explanation 1
8 0
2 0
2 0
0 E
CT bounding condition improvement 2
9 0
2 0
2 0
0 E
CT bounding condition improvement 3
7 0
2 0
4 0
0 S
Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5     For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6     For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
: a. Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
: a.
: b. TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
: c. CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
: b.
2+4+6 7           In column 7, calculate the percentage of unsatisfactory scenario elements:                           100%
TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
1+3+5 8           If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
: c.
9           In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
7 In column 7, calculate the percentage of unsatisfactory scenario elements:
8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
2 + 4 + 6 1 + 3 + 5100%


ES-301                                                                       15                                                   Form ES-301-7 Site name:                                                                         Exam Date:
ES-301 15 Form ES-301-7 Site name: Exam Date:
OPERATING TEST TOTALS Total           Total         Total Total                                                     % Unsat.                             Explanation Unsat.          Edits         Sat.
OPERATING TEST TOTALS Total Total Unsat.
Total Total  
% Unsat.
Explanation Edits Sat.
Admin.
Admin.
9             1               5             2 JPMs Sim./In-Plant 11             0               5             6 JPMs Scenarios           3             0               2             1 Op. Test 23             1               12           9             4.3%
JPMs 9
Totals:
1 5
2 Sim./In-Plant JPMs 11 0
5 6
Scenarios 3
0 2
1 Op. Test Totals:
23 1
12 9
4.3%
Instructions for Completing This Table:
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
1.
: 1.
Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs. For scenarios, enter the total number of simulator scenarios.
Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs. For scenarios, enter the total number of simulator scenarios.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the 2.
: 2.
previous tables. Provide an explanation in the space provided.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking 3.
: 3.
only.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.
: 4.             Total each column and enter the amounts in the Op. Test Totals row.
: 4.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in 5.
Total each column and enter the amounts in the Op. Test Totals row.
the bolded % Unsat. cell.
: 5.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
satisfactory, if the Op. Test Total % Unsat. is 20%
satisfactory, if the Op. Test Total % Unsat. is 20%
unsatisfactory, if Op. Test Total % Unsat. is > 20%
unsatisfactory, if Op. Test Total % Unsat. is > 20%
Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, 6.
: 6.
including the following:
Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:
The JPM performance standards were incorrect.
The JPM performance standards were incorrect.
The administrative JPM tasks/keys were incorrect.
The administrative JPM tasks/keys were incorrect.

Latest revision as of 02:45, 2 December 2024

ES-301-7
ML20245E563
Person / Time
Site: Prairie Island  Xcel Energy icon.png
Issue date: 08/27/2020
From:
Nuclear Management Co
To:
NRC/RGN-III/DRS/OLB
Callaway G
Shared Package
ML19127A300 List:
References
Download: ML20245E563 (15)


Text

ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Prairie Island Exam Date: Aug 17, 2020 1

2 3

Attributes 4

Job Content 5

6 Admin JPMs ADMIN Topic and K/A LOD (1-5)

U/E/S Explanation I/C Cues Critical Scope Overlap Perf.

Key Minutia Job Link Focus Steps (N/B)

Std.

A1A - RO Determine Turbine Startup and Load Time Conduct of OPS 2.1.25 2

E S

Justify the allowed range of values. The chart is in increments of 5 minutes. An accepted accuracy for interpreting analog indication is 1/2 of the smallest increment. The first allowed range matches this (2.5 minutes. The accumulated uncertainty for two readings would be 5 minutes. Explain why the final calculation has an allowed range of 10 minutes.

Facility agrees. Proposes to change final calculation allowed range to 87.5 - 92.5 minutes.

fc-6/25/2020µ SAT GWC 6/26/2020 A1B-RO Determine Blended Flow Concentration Conduct of Ops 2.1.25 2

E S

1. Justify why the RO would not be required to find the applicable procedure when directed to determine blended flow concentration.
2. Explain the basis for the allowed range (+/- 5%).

A straight line between 67 and 1500 is almost exactly in the center of 30 and 40 on a log scale (33.9).

Taking the calculated value of X and dividing by 15 (the scale of the meter) yields 34.3.

A range of +/-1% seem appropriate.

1. RO would be required to find applicable procedure, in the CR; however, low discriminatory value on an admin JPM for an operator to have to recall procedure name/number from memory, in a classroom setting.
2. Given previous comment of 1/2 smallest increment allowed accuracy, facility believes +/- 5% is appropriate. (copy of nomograph with potential accuracy error lines provided) fc-6/25/2020
1.

I am not asking that the applicant recall the procedure number or name from memory, but to be able to find the procedure from the provided references, just like in the CR. The current cue is not operationally oriented (i.e. not valid)

2.

I accept your range justification GWC 6/26/2929 After discussion, this meets the licensees expectations.

SAT GWC 7/29/2020 A2 -RO Equipment Control 2

U S

A daily surveillance is conduct of ops, not equipment control.

ES-301 Operating Test Review Worksheet Form ES-301-7 RWST Level Surveillance 2.2.40 NUREG 1021 gives examples of Equipment control as For example, have the applicant demonstrate how he or she would take a failed system or component out of service, initiate maintenance on the system, and test the system before placing it back in service.

Also, the level of difficulty is quite low.

Facility disagrees; NUREG 1021, ES-301, pg.2 Admin Topics under Equipment Control lists surveillance testing under examples. While this list is not all inclusive, Equipment Control is the only generic section that has a surveillance associated k/a.

fc-6/25/2020 I accept that surveillance testing is appropriate.

The level of difficulty is still unacceptable. Anyone who could meet site access requirements can meet this task. The only ability tested here is the ability to read an analog meter. This JPM does not discriminate between competent and less than competent applicants.

GWC 6/26/2020 Revised SAT GWC 7/29/2020 A3-RO Initiate a Radiation Monitor Out of Service Data Sheet A4 RO Determine Maximum RCS Venting Time Radiation Control 2.3.5 Emergency Plan 2.4.25 2

X U

E S

This does not meet the requirements for a radiation control RO JPM.

NUREG 1021 says, regarding Radiation Control JPMs, The ROs duties generally require knowledge of radiation worker responsibilities and operation of plant systems associated with liquid and gaseous waste releases.

In this JPM, the RO is given two forms and directed to transfer data from one form to another and then apply a sticker. It does not test knowledge of rad worker responsibilities or operation of plant systems associated with releases.

Would prefer (for schedule reasons) to do admin JPMs in classroom.

Facility disagrees. Facility believes this JPM to have high operational validity, high safety significance for CTMT radiation monitors, and high importance for E-Plan. Per previous response, NUREG 1021 ES-301 is a list of examples, but is not an exhaustive list.

We prefer classroom also, which is why we provide the procedure. Well check other on JPM page one if you need us to, no problem; however, there is a cue on last step allowing for performance in simulator or classroom.

Fc-6/25/2020 µ It is the level of difficulty that makes it unacceptable.

The task may be valid, safety-significant and important, but it still does not require knowledge of anything but the English language. Anyone who can

ES-301 Operating Test Review Worksheet Form ES-301-7 meet site access requirements can perform this task.

This JPM does not discriminate between competent and less than competent applicants.

GWC 6/26/2020 Replaced JPM SAT GWC 7/29/2020 A1A - SRO Verify Heavy Load Lift Requirements Conduct of Operations 2.1.23 2

S A1B - SRO Determine Containment CSFST Color Conduct of Operations 2.1.7 1

U S

1. Red and Orange path entry conditions are RO knowledge according to ES-401.
2. Unit 1 has experienced a Large Break LOCA is a conclusion, not an observation. It adds nothing.
3. You are an extra SRO in the control room adds nothing and does not make it an SRO JPM.
1. Facility agrees that orange and red paths are RO knowledge; however, this JPM does not test red/orange path entry conditions, but rather if red/orange path exists, which is SRO. With ERCS (plant computer) OOS, a licensed SRO will perform the CSFST. Normally it would be the STA, but we received feedback during previous validation about that and decided that it would raise fewer questions for an applicant if they were listed as extra.
2. Facility disagrees. The statement re LBLOCA is required to explain abnormal readings given to candidate; w/o that info we cant justify indications given to candidate; providing candidate with multiple indications of a LOCA and requiring them to first determine if a LOCA has occurred before proceeding with the JPM is not whats being tested; furthermore, NUREG 1021 tells us not to do it.

fc-6/25/2020

1.

Please explain which item in 10CFR55.43 applies to make this SRO knowledge. Procedure selection associated with red/orange conditions is specifically excluded in ES-401. If this is based on a site-specific learning objective that is unique to the SRO position, provide the facility learning objective that specifies this.

2.

The cause of the abnormal conditions in containment is not relevant to the task. The premise of critical safety functions is that they are not event based, they are safety function based.

GWC 6/26/2020 SRO only Objective provided Change initial assessment to SAT GWC 7/20/2020

ES-301 Operating Test Review Worksheet Form ES-301-7 A2 - SRO Perform Shutdown Safety Assessment Equipment Control 2.2.18 3

E S

In initiating cues, do not say Determine Total Points.

This is the only way to carry out the 2nd bullet. Saying determine current condition is enough.

Cues written this way to ensure examinee performs all aspects of JPM. Without the clear and specific direction, the potential exists for the examinee to count points in their head then simply circle the condition.

While they would satisfy the task standard, they would be unsat on multiple critical steps, steps which are required to determine if examinee fully grasps the knowledge.

Maybe theres an opportunity to re-word the task standard.

fc-6/25/2020 My concern is that the cue is not operationally valid.

What words would most likely be used to assign this task? Determine total points may be more detail than would be expected in assigning the task.

GWC 6/26/2020 Cue revised SAT 7/29/2020 A3 - SRO Authorize Waste Gas Release Radiation Control 2.3.6 3

E S

Change the cue step from Determine if to Complete step 7.11.

Theres a potential psychometric flaw in directing the candidate to complete a step in the ICs where they have to determine whether something is flawed.

Completing steps with no consequence is one thing; however, the candidate may inadvertently believe they must approve the release regardless.

Warrants further discussion perhaps.

fc-6/25/2020 My concern is that the cue is not operationally valid.

What words would most likely be used to assign this task? It seems that the expectation at this point is that the conditions are met and the assigned task would be to complete the procedure steps. determine if seems to be inappropriately leading the applicant.

GWC 6/26/2020 Revised cue SAT GWC 7/29/2020 A4 - SRO Determine Errors on Initial Classification Emergency Procedures

/ Plan 2.4.40 2

S

ES-301 Operating Test Review Worksheet Form ES-301-7 a

Malf of auto makeup during boration 1

004 A4.12 3

E S

The reference to C12.5, AOP 2, Section 2.4.1 seems to be limited to a listing of acceptable ways to stop the boration. None of the symptoms are present and the IFTHEN conditions of the step are not met.

Perhaps a note is appropriate that the applicant is not expected to enter the AOP.

Ill need to get additional clarification on phone call.

fc-6/25/2020 State the actual basis for this performance expectation. The reason the applicant takes these steps is not because AOP 2 says to do them. AOP 2 entry conditions are not met and the IFTHEN conditions of the step are not met.

Basis for action revised SAT GWC 7/29/2020 b

Periodic rotation of charging pumps 2

004 A4.08 3

S c

Energize Przr B/U Htrs and respond to PORV leakage 3

007 A4.10 3

X U

S Double jeopardy with written RO question 2. Both require the applicant to evaluate tailpipe temperature and determine that a PZR PORV is leaking.

Facility disagrees. JPM is testing PRZR PORV leakage response. Written exam question #2, while evaluating temperatures, is testing PRZR safeties, which would look different and requires a different response.

fc-6/25/2020 Agree with feedback. Change initial assessment to SAT GWC 6/26/2020 d

SWS/Loss of cooling water header pressure 4S 075 A4.01 3

S e

Alternate CFCUs w/ CFCLU hi temp 5

022 A4.01 3

S f

Manual start of D1 from CR 6

064 A3.06 2

E S

Add exercising the note prior to step 5.1.1.M (amber lights not lit, speed adjustment necessary)

Would like to compress time in step 5.1.1.I (i.e. not wait 3 minutes) if it has no consequence.

Facility agrees to add note to JPM prior to step 5.1.1.M.

Also, its not intended to wait the 3 minutesthat would be super uncomfortablefor everyone. We can clarify that cue or add a note for time compression if you desire.

ES-301 Operating Test Review Worksheet Form ES-301-7 fc-6/25/2020 µ I am not asking for a note that the applicant may adjust the speed setting. I am asking to change the simulator setup where the lights are not lit and make the step critical to properly adjust the speed setting.

After discussion, it is not practical to have the speed setting off.

SAT GWC 7/29/2020 g

NIS/PR Daily Calibration 7

015 A1.01 2

X E

S I suggest that recording initial and final pot settings are critical steps. If these values are incorrectly recorded, the task has not been properly completed, and it will have to be re-done. Also, this would test the applicants ability to read the POT setting.

Is average reactor thermal power calculated by the plant computer? I would like to require the applicant to obtain this data if it is practical instead of making it part of the cue.

1.

Facility agrees. SP1005 steps 8.6.4 & 8.6.6, Record INITIAL/FINAL GAIN SETTING R303 are the initial and final pot settings. These steps are already critical.

2.

Normally, yes, average thermal power is calculated by the plant computer (ERCS) calorimetric program (CALM); however, in this case, ERCS is OOS. SP 1005B is the calculation of power when ERCS is OOS, thus the basis for why its given in the cue. Just not practical in this case. SP 1005, NIS Daily Calibration, section 7 is for ERCS in service, 8 is ERCS OOS. (mark-up provided to examinee attached) fc-6/25/2020 µ Change initial assessment to SAT GWC 6/26/2020 Added expected initial reading to evaluator key GWC 7/29/2020 h

Shutdown SFP normal ventilation 8

2.1.20 3

S I

RCP Seal Isolation after loss of all AC 4P 028 AA1.03 3

E S

valves are closed is a conclusion, not an observation.. Give as a cue the indications that should lead the applicant to conclude that the valves are closed.

Facility amenable to making change; will evaluate on Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 j

4S 2.1.23 3

S Cue for closing MV-32031 is incorrect. MV-32031 is closed is a conclusion, not an observation. Give as a

ES-301 Operating Test Review Worksheet Form ES-301-7 Locally close 1 Turb Look a CLG wtr hdr MV cue the indications that should lead the applicant to conclude that the valve is closed.

Facility amenable to making change; will evaluate on Monday and provide proposed update next week fc-6/25/2020 µ Cue revised SAT 7/29/2020 k

Restore instrument air compressors 8

068 AA1.21 3

S

ES-301 8

Form ES-301-7 Instructions for Completing This Table:

Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.

1.

Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1. (ES-301, D.3 and D.4)

2.

Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested.

Mark in column 2 (Appendix D, C.1.f)

3.

In column 3, Attributes, check the appropriate box when an attribute is not met:

The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)

The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)

All critical steps (elements) are properly identified.

The scope of the task is not too narrow (N) or too broad (B).

Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)

The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.

A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).

4.

For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:

Topics are linked to the job content (e.g., not a disguised task, task required in real job).

The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)

5.

Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.

6.

In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.

Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.

ES-301 9

Form ES-301-7 Facility: Prairie Island Scenario: 1 Exam Date: Aug 17, 2020 1

2 3

4 5

6 7

8 9

10 Event Realism/Cred.

Required Actions Verifiable actions LOD TS CTs Scen.

Overlap U/E/S Explanation 1 - Secure 11 TD AFW Pump s

2. Raise power to POAH s

3- -12 Charging Pump Trip.

s 4 - 11 TD AFWP Accumulator Low Air Pressure X

s 5 - 11 CTMT Vac Bkr fails closed X

s 6 - Faulted 12 SG to CTMT X

E S

Need a better bounding condition. 45 minutes is arbitrary. Let the event run with AFW flow and see if either containment or integrity goes orange.

Containment will go ORANGE path on sump level due to steam condensing. We already have it written into the CT plant conditions that a RED path on Integrity is likely and cannot be prevented by crew actions. In all of our CT testing, weve found that a RED path on Integrity during a faulted SG is inevitable from the cooldown and would not be fair to fail the crew on a CT they cant prevent.

We propose changing the bounding condition to ORANGE path on Containment CSFST.

Copy of proposed change attached fc-6/25/2020 µ SAT GWC 6/26/2020 7 - 12 MD AFW pump fails to auto start X

s 8 - SI to Feed/condensete relay failure s

8 2

2 7

S

ES-301 10 Form ES-301-7 Instructions for Completing This Table:

Use this table for each scenario for evaluation.

2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.

3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)

5 Check this box if the level of difficulty is not appropriate.

6 Check this box if the event has a TS.

7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.

8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.

10 Record any explanations of the events here.

In the shaded boxes, sum the number of check marks in each column.

In column 1, sum the number of events.

In columns 2-4, record the total number of check marks for each column.

In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.

In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)

In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)

In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)

In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.

ES-301 11 Form ES-301-7 Facility: Prairie Island Scenario: 2 Exam Date: Aug 17, 2020 1

2 3

4 5

6 7

8 9

10 Event Realism/Cred.

Required Actions Verifiable actions LOD TS CTs Scen.

Overlap U/E/S Explanation 1 - Swap running EH oil pump X

S Repeat from 18-2 2 -

Controlling PRZR press chan fails low X

X S

Repeat from 18-4 Repeat from 16-3 3 - D1 Local Alarm X

S 4 - Turbine EH valve malfunction S

5 - Rapid downpower to 50%

S 6 - Loss of all AC X

E S

This bounding condition could not be credibly reached in the time frame of a scenario. If delayed restoration could lead to unnecessary E-Plan escalation, this may be a useful bounding condition.

1. We assume youre referring to the AFW restoration CT and SG dryout. If both SG wide range levels reach <50% and total AFW flow is <200 gpm, then RED path on Heat Sink; therefore potential loss of RCS and Fuel Clad barriers = SAE. The loss of offsite and onsite AC power initially would be an ALERT, assuming the crew restores power within 15 minutes. If power isnt restored to both safeguard buses in 15 minutes, then it would be a SAE. Since were not controlling how quickly the crew restores power, they may go over the 15 minutes which would invalidate an E-Plan escalation CT.
2. I think youll be more amenable to this (and Im gonna eat more crow for this as well):
a.

We made a change somewhere along the linebefore the quarantine. When we ran this today, we noticed that this particular LOOP AFW CT doesnt actually apply because the 11 TD AFWP is OOS, sothe crew cant start it. Theyll have to restore power to Bus 16 (CT-6) in order to start 12 MD AFWP. I probably changed it due to overlap concerns with a previous exam but neglected to update the CT.

b.

Good news is that we have CT that does apply that we didnt put in, again, probably a change and just didnt update it. Its in the body of the scenario, just not the CT table. CT-3A, manually trip the main turbine. Copy of proposed attached

ES-301 12 Form ES-301-7 fc-6/25/2020 µ The bounding condition I was referring to was to restore power prior to ORANGE or RED path on Core Cooling CSF. I think it takes hours to get an overheated core in a station blackout with no AFW. If it is not reasonable to expect the crew to restore power in 15 minutes, then the E-Plan escalation may not be a valid condition.

GWC 6/26/2020 Critical task revised SAT GWC 7/29/2020 7 - Turbine fails to auto trip S

8 - D2 fails to auto start X

S 9 - 12 MD AFW pump fails to auto start X

S 9

2 2

6 S

ES-301 13 Form ES-301-7 Facility: Prairie Island Scenario: 3 Exam Date: Aug 17, 2020 1

2 3

4 5

6 7

8 9

10 Event Realism/Cred.

Required Actions Verifiable actions LOD TS CTs Scen.

Overlap U/E/S Explanation 1 - Swap RMU pumps S

2 - 1st stage PI fails low X

X S

Repeat from 18-2 Repeat from 16-2 3 - Restore Tavg to Tref S

4 - PRZR level interlock chan fails low X

S 5 - 11 SGTR 4

S Four critical tasks are associated with a SGTR 6 - 11 & 12 RHR pumps fail to auto start S

7 - SI to cooling water relay signal failure S

S 7

2 4

S

ES-301 14 Form ES-301-7 Facility: Exam Date:

Scenario 1

2 3

4 5

6 7

8 11 Event Totals Events Unsat.

TS Total TS Unsat.

CT Total CT Unsat.

% Unsat.

Scenario Elements U/E/S Explanation 1

8 0

2 0

2 0

0 E

CT bounding condition improvement 2

9 0

2 0

2 0

0 E

CT bounding condition improvement 3

7 0

2 0

4 0

0 S

Instructions for Completing This Table:

Check or mark any item(s) requiring comment and explain the issue in the space provided.

1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).

This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).

2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:

a.

Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.

b.

TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)

c.

CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.

7 In column 7, calculate the percentage of unsatisfactory scenario elements:

8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.

9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.

Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.

2 + 4 + 6 1 + 3 + 5100%

ES-301 15 Form ES-301-7 Site name: Exam Date:

OPERATING TEST TOTALS Total Total Unsat.

Total Total

% Unsat.

Explanation Edits Sat.

Admin.

JPMs 9

1 5

2 Sim./In-Plant JPMs 11 0

5 6

Scenarios 3

0 2

1 Op. Test Totals:

23 1

12 9

4.3%

Instructions for Completing This Table:

Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.

1.

Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs. For scenarios, enter the total number of simulator scenarios.

2.

Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.

3.

Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.

4.

Total each column and enter the amounts in the Op. Test Totals row.

5.

Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.

Refer to ES-501, E.3.a, to rate the overall operating test as follows:

satisfactory, if the Op. Test Total % Unsat. is 20%

unsatisfactory, if Op. Test Total % Unsat. is > 20%

6.

Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:

The JPM performance standards were incorrect.

The administrative JPM tasks/keys were incorrect.

CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).

The EOP strategy was incorrect in a scenario(s).

TS entries/actions were determined to be incorrect in a scenario(s).