ML17270A166
| ML17270A166 | |
| Person / Time | |
|---|---|
| Site: | Callaway  |
| Issue date: | 09/20/2017 |
| From: | Vincent Gaddy Operations Branch IV |
| To: | Union Electric Co |
| References | |
| Download: ML17270A166 (11) | |
Text
ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Callaway Exam Date: September 11, 2017 1 2 3 4 5 6 ADMIN Attributes Job Content LOD Admin JPMs Topic U/E/S Explanation (1-5) I/C Critical Scope Perf.
and K/A Job Cues Overlap Key Minutia Focus Steps (N/B) Std. Link Did not review RO Admin JPMs, or JPM S7. Remaining RO applicant removed from exam prior to Op Test Exam Review.
The initial conditions should state that the plant has been at 100% power the entire month, so that none of the outage rules would apply.
The initiating cue should also require informing the shift manager of WHY any individual is not eligible.
Critical steps should be generated for documenting why A.1.a (SRO) 2.1.5 3 ES an individual is not eligible.
The task standard should not say Note since it will be re required to inform why they are not eligible.
Could possible make a chart with the 5 individuals names, and the applicant fills it out with Eligible / Not Eligible & Why.
Corrections made Now SAT I will need someone to explain how control bank C at 46 A.1.b (SRO) 2.1.37 2 ES steps and control bank D at 199 steps are plausible Task standard (and initiating cues) should indicate the job should be screened as priority 1 and why. (It is possible to screen as priority 1 for the wrong reason.)
A2 (SRO) 2.2.17 2 ES Should also indicate the risk color and why.
Is it cueing to say risk color, since it is possible to land in Not allowed by tech specs or other columns.
Rewrote task standard. Now SAT This is a direct lookup JPM. No discriminatory value to A3 (SRO) 2.3.4 1 US simply look up the values in a procedure. New JPM Now SAT This JPM is overlap of question 94 on the written. Even though the written question asks about rapidly progressing events, they are too close, and either this or the written question must be replaced. Replaced question 94 on the written exam.
Task standard should state marked the PAR A4 (SRO) 2.4.44 2 X S flowchart in accordance with the attached key.
Corrected JPM step 3 (which is critical) states the task element is to include detailed justifications and basis for the PAR selected, but there is no information on the key. Rewrote.
Now SAT Rev. 11
ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Callaway Exam Date: September 11, 2017 1 2 3 4 5 6 Safety Attributes Job Content Simulator/In- LOD Function and U/E/S Explanation Plant JPMs (1-5) I/C Critical Scope Perf.
K/A Job Cues Overlap Key Minutia Focus Steps (N/B) Std. Link Task standard should be started the TDAFP and raised speed to 3850 rpm. Should probably remove the 4S P1 3 ES statement in the task standard of notifying the EC, unless 054 AA1.02 you want to make Step 15 a critical step. Rewrote task standard. Now SAT The task standard should be written to describe what the applicant will do. NG01 and NK21 are already energized, so they should not be part of the task 6
P2 2 ES standard. Rewrote task standard 055 EA2.03 JPM steps 3, 4, and 5 should be split into separate steps for each breaker, with the appropriate steps listed as critical steps. Split steps. Now SAT 8 JPM step 6 says Step B10.a.4. Should be B10.a.5.
P3 2 ES 008 A2.02 Corrected step number. Now SAT This overlaps with emergency boration in both scenarios 1 1 and 2. Even though this JPM has you emergency S1 2 X US 004 A4.18 borate from OTO-ZZ-00003, the flowpaths are identical.
Replaced with another JPM. Now SAT JPM step 15 could be cleaned up a little bit. (The standard states that the candidate performs the RNO 2 because pressure has been above 27 psig, but the RNO S2 3 ES 013 A4.01 should come into play because at step b, the pumps are not running. Added clarifying statement in step 15. Now SAT JPM step 12 performance standard, last sentence should 3
S3 2 ES say, Candidate checked BB HIS-456A closed. Added 010 A4.03 statement to step 12. Now SAT Task standard should state something like, will have started either RCP A or B after RCP D lift pump fails to start, and make step 12 critical. This is to ensure that the applicant follows the cue, and attempts to start RCP D first. Revised task standard.
4P S4 2 ES Should JPM step 26 be N/A, since the applicant should 003 A4.06 not be trying to start RCP C? Made step N/A.
Should JPM step 27 be critical? i.e., what happens if oil lift pump does not run for greater than 2 minutes? Yes.
Should JPM step 30 be critical? Changed step to be not critical. Now SAT As-written, this is an LOD=1 JPM, as it is not much more 5 than operating a stopwatch. This JPM could be S5 1 US 026 A4.01 salvageable if one of the stroke times was unacceptable, and the applicant had to perform other tasks (also would Rev. 11
ES-301 Operating Test Review Worksheet Form ES-301-7 make it an alternate path JPM.) Changed so that one stroke time is unacceptable. Now SAT 6
S6 2 S 062 A4.01 8
S8 3 S 029 A2.03 Rev. 11
ES-301 4 Form ES-301-7 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1. Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.
(ES-301, D.3 and D.4)
- 2. Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)
- 3. In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4. For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6. In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
Rev. 11
ES-301 5 Form ES-301-7 Facility: Callaway Scenario: 1 Exam Date: September 11, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions Actions Overlap General Comments: Add a number to critical tasks (i.e., CT-1). Add event number to event description page.
SRO gets Credit for Component failure (page 1). Add per turnover, B CCP likely to be 1 ES started (page 7). Add the step 27 substeps (page 8). Corrections made. Now SAT Add steps for Attachment 8 (page 12). Insert note from procedure between steps A7 2 ES and A8 (page 13). Corrections made. Now SAT Add to step A3 that permissives will be LIT (page 15) Add cabinet number to step A5 3 X ES (page 16). Corrections made. Now SAT Add Tech Spec 3.4.17 (pages 1 and 19). Add steps for Attachment 8 (page 21).
4 X 2014/3 ES Corrections made. Now SAT 5 (major) 2014/3 S 6 XX S 2 2 2 ES Rev. 11
ES-301 6 Form ES-301-7 Facility: Callaway Scenario: 2 Exam Date: September 11, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions Actions Overlap General Comments: Add a number to critical tasks (i.e., CT-1). Add event number to event description page.
1 X ES There are no verifiable actions for the BOP (page 1). Corrections made. Now SAT Add step 1 is immediate action (page 10). BB-LS-459D should indicate that you are 2 X ES removing BB-461 from control (page 11). Corrections made. Now SAT 3 ES Missing step 5.1.7 (page 13). Corrections made. Now SAT 4 S 5 (major) X ES Need to add addendum 11 (page 29). Corrections made. Now SAT 6 X S 2 2 0 ES Rev. 11
ES-301 7 Form ES-301-7 Facility: Callaway Scenario: 3 Exam Date: September 11, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions Actions Overlap General Comments: Add a number to critical tasks (i.e., CT-1). Add event number to event description page. Change Initial condition of simulator to ensure that the cooling tower level alarm does not come in. Change initial condition to remove SFP cleanup from being in service.
1 S 2 X 2016/4 ES The title for OTO-SE-00001 is incorrect (page 8). Corrections made. Now SAT Add addendum 2 and steps 5.3 of OTN-EG-00001 as possible alternatives that may be 3 ES used to shutdown and startup SFP pumps. (page 11). Corrections made. Now SAT 4 X ES Add title for OTO-AB-00001 (page 14). Corrections made. Now SAT 5 X 2016/1 S Add noun name for EFHV0037 (page 21). Add step 13 steps for RCPs not running, and 6 (major) X 2016/1 ES the RNO actions (page 25). Corrections made. Now SAT 7 X S 3 2 3 ES Rev. 11
ES-301 8 Form ES-301-7 Facility: Callaway Scenario: 4 Exam Date: September 11, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions Actions Overlap General Comments: Add a number to critical tasks (i.e., CT-1). Add event number to event description page. Swap order of events 3 and 4. Change simulator so there NB01 remains energized for 10 minutes.
1 X S 2 X S 3 S In final scenario, this is event 4.
Swap events 3 and 4. Add conditional tech spec 3.4.1, depending on how low pressure 4 (X) ES gets (page 12). Corrections made. (In final scenario, this is event 3.) Now SAT 5 (major) X ES Provide addendum 18 (page 15). Corrections made. Now SAT 6 X 2014/2 S 2 (3) 2 1 ES Rev. 11
ES-301 9 Form ES-301-7 Instructions for Completing This Table:
1 Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f)
- opening, closing, and throttling valves
- starting and stopping equipment
- raising and lowering level, flow, and pressure
- making decisions and giving directions
- acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
- In column 1, sum the number of events.
- In columns 2-4, record the total number of check marks for each column.
- In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
- In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
- In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
- In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
Rev. 11
ES-301 10 Form ES-301-7 Facility: Callaway Exam Date: September 11, 2017 1 2 3 4 5 6 7 8 11 Scenario % Unsat. Explanation Event Events TS TS CT CT Scenario U/E/S Totals Unsat. Total Unsat. Total Unsat.
Elements 1 6 2 2 ES All corrections made. Scenario now SAT 2 6 2 2 ES All corrections made. Scenario now SAT 3 7 3 2 ES All corrections made. Scenario now SAT 4 6 2 2 ES All corrections made. Scenario now SAT Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a. Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b. TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c. CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
2+4+6 7 In column 7, calculate the percentage of unsatisfactory scenario elements: 100%
1+3+5 8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
Rev. 11
ES-301 11 Form ES-301-7 Site name: Callaway Exam Date: September 11, 2017 OPERATING TEST TOTALS Total Total Total %
Total Explanation Unsat. Edits Sat. Unsat.
Admin.
5 1 4 0 JPMs Sim./In-Plant 10 2 6 2 JPMs Scenarios 4 0 4 0 Op. Test 19 3 14 2 15.7%
Totals:
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1. Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.
For scenarios, enter the total number of simulator scenarios.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and 2.
simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous 3.
tables. This task is for tracking only.
- 4. Total each column and enter the amounts in the Op. Test Totals row.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test 5.
Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
- satisfactory, if the Op. Test Total % Unsat. is 20%
- unsatisfactory, if Op. Test Total % Unsat. is > 20%
Update this table and the tables above with post-exam changes if the as-administered operating test 6.
required content changes, including the following:
- The JPM performance standards were incorrect.
- The administrative JPM tasks/keys were incorrect.
- CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
- The EOP strategy was incorrect in a scenario(s).
- TS entries/actions were determined to be incorrect in a scenario(s).
Rev. 11