ML17319A703
| ML17319A703 | |
| Person / Time | |
|---|---|
| Site: | South Texas  |
| Issue date: | 10/09/2017 |
| From: | Vincent Gaddy Operations Branch IV |
| To: | South Texas |
| References | |
| Download: ML17319A703 (15) | |
Text
ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: South Texas Project Exam Date: Sept 25-29, 2017 1 2 3 4 5 6 Attributes Job Content ADMIN Topic and LOD Admin JPMs U/E/S Explanation K/A (1-5) I/C Critical Scope Perf. Job Cues Overlap Key Minutia Focus Steps (N/B) Std. Link MT OPS with RA1 degraded vac: 3 S 051 G2.1.32 This was unsat because it was a direct look U
up JPM as written. For the task standard, 1
either state the volume (45827 gal) or state RCS Refill: 002:
RA2 S per the key.
2.1.25 STP made requested changes for draft 2
submittal and from validation week and this JPM is now Sat.
Task number and title on page 7 are Det H2 recomb E incorrect as a 96200 If calc includes a range, ensure to include this range RA3 settings: 028: 3 in the key. STP made requested changes for 2.1.23 S draft submittal and from validation week and this JPM is now Sat.
Offsite notif RA4 3 S form: 2.4.39 Title is wrong on cover page (not about the condenser). Task standard is missing H7 fuel move that the key includes. Also, is case 1 B4, case 2 H8, then case 3 H7? If so the JPM standard is wrong for this. Also it is not in the U
standard steps that the moves must be done in order as indicated on the key. The task Fuel movement:
SA5 3 X standard needs to state per the attached key 2.1.42 if you expect them to do this in that order.
Task standard was missing many aspects S
and therefore makes JPM unsat as submitted.
STP made requested changes for draft submittal and from validation week and this JPM is now Sat.
Review ESF SA6 3 S pwr avail: 2.1.20 Review faulted SA7 ECO (AFW) 3 S 2.2.13 Initiate a dose E Task number and title on page 8 is incorrect.
SA8 3 S STP made requested changes for draft extension:2.3.13 Rev. 11
ES-301 Operating Test Review Worksheet Form ES-301-7 submittal and from validation week and this JPM is now Sat.
Task number and title on page 8 is incorrect.
E Outline was not updated for PAR JPM vs EAL Determine PAR:
SA9 3 call (ES-301-1). STP made requested 2.4.44 S changes for draft submittal and from validation week and this JPM is now Sat.
1 2 5
3a 3b 3c 3d 3e 3f 3h 4a 4b 6 Simulator/In-Plant Safety Function JPMs and K/A LOD Job U/E/S (1-5) I/C Cues Critical Scope Overlap Perf. Key Minutia Link Explanation
- 1. Task standard should also specify all critical steps completed for all JPMs.
- 2. Please gray highlight the NOTE line item where the Alt path begins for easier identification by examiner General for ALL Alternate path JPMs.
- 3. I dont understand the completion comments for time in 15 minutes near the end of all JPMS some of the JPMs.
- 4. The task title (usually around page 8 of JPM standard where the task number is grouped with it) doesnt seem to match the actual task being done for a few JPMs. I think this is a cut/paste error.
S1 5 3 S For initiating cue, should specify that CRS wants 1B in standby once 1C is started, such that at step 13 in the JPM (step 10.17 of E procedure) the applicant performs it without cue (if asked, examiner should state S2 8 2 directions already given). This would also apply for step 14 of the JPM (give it in the init S cue) for the mode sel switch. STP made requested changes for draft submittal and from validation week and this JPM is now Sat.
S3 3 3 S S4 6 3 S Rev. 11
ES-301 Operating Test Review Worksheet Form ES-301-7 S5 4S 3 S S6 2 3 S S7 7 3 S S8 1 3 S P1 2 3 S P2 9 3 S P3 6 3 S Rev. 11
ES-301 4 Form ES-301-7 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1. Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.
(ES-301, D.3 and D.4)
- 2. Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)
- 3. In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4. For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6. In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
Rev. 11
ES-301 5 Form ES-301-7 Facility: South Texas Project General comments on scenarios Exam Date: Sept 25-29, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap General Comments for all scenarios Please remove the critical parameters section at the beginning of the D-1 for crew performance on all scenarios.
For parameters to record for grading purposes, we will need to work thru the required parameters to capture for scenarios for grading during validation week, but it is a much longer list and is required for each event.
The nureg requires every switch that is manipulated to be listed in the guide. As an example in this scenario, in CT-17, list the valve by name/number or the switch name/number for each item manipulated. You can un-bold the verify steps or other aspects of the CT since there is nothing actually done with these from a switch manip perspective. The bolded items are important for the actual switch manipulations necessary to complete the CT. Each item manipulated for the CT should be annotated in some way with its name/number so that it can be quickly marked when going thru the list of items for this CT.
Please insert the booth communications from the booth comm file into the D-2 by the step where it is performed. All of these items need to be in the guide at its point in time when given to ensure no cueing can occur with booth and applicant crews.
There are no scenario objectives in the D-2 forms as required by the NUREG.
CT table needs post scenario CT statement added. Sent via e-mail. The station needs to develop proper bases for each CT also (future for items not done yet, for items where it already exists, please put in the CT table).
The NOTES column is made for examiners to TAKE notes in the D-2 during administration so items you have in those blocks are good but really belong underneath the step in req actions column and listed as a note or comment, maybe in parenthesis or something like that.
Rev. 11
ES-301 6 Form ES-301-7 Facility: South Texas Project Scenario: 1 Exam Date: Sept 25-29, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap 1 S Procedure not highlighted in gray for this event Where is the actual TS number and LCO (ie TS 3.6.1, LCO condition A)
E S
for the D-2. STP made requested changes for draft submittal and from 2 validation week and this scenario event is now Sat.
LCO letter is missing (ie condition A in this case I think). Since the Alarm response procedure is used it should be grayed out and not use words E
refer to since actions are taken from it and performed. Also need to list the function number in the guide (ie function 5e for this event I think).
S STP made requested changes for draft submittal and from validation 3 week and this scenario event is now Sat.
Need all annunciators for each event. Reason is that during evaluations if someone doesnt understand that an alarm is expected for this event E then they will sometimes take different actions and it then affects the grading. Is there are required DP we can put in the guide for FW/steam S hdr DP so examiner doesnt have to look over applicants shoulder for Addendum 3? STP made requested changes for draft submittal and from 4 validation week and this scenario event is now Sat.
D-1 and D-2 needs to have ATWS on it for this event. This is a major also. Please put the the rx trip switches step in the D-2 before de-E energizing the LCs since they have to try this first even though it doesnt work. Step 3 of event 5 page 13 should be verifies AFW flow, not Feedwater flow. Bounding criteria are not in the D-2 with CT-10 on page
- 13. It needs to be put here (to align with content on CT table).
S STP made requested changes for draft submittal and from validation 5 week and this scenario event is now Sat.
Need to properly bound CT for this event in the D-2.
E S
STP made requested changes for draft submittal and from validation 6 week and this scenario event is now Sat.
7 S 8 S Rev. 11
ES-301 7 Form ES-301-7 Rev. 11
ES-301 8 Form ES-301-7 Facility: South Texas Project Scenario: 2 Exam Date: Sept 25-29, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap Need all alarms at top for SFP pump, not just trouble alarm. Put the alarm E response procedure in the D-2 for the actions for this event.
1 STP made requested changes for draft submittal and from validation S week and this scenario event is now Sat.
Put all alarms in the D-2, also listed the alarm response procedure used, The noted field is really for examiner notes and on this page in particular it is heavily populated with things that should be a note underneath the required op action E list as a note. Keep the step numbers from the actual procedures (that is a nice feature for the examiner) in the notes but clear some space for examiners to write. For example, the next page (9) has some space in it even though there are a few notes in the notes block.
For the SRO TS call block on page 9, we need the TS number (which you have) the LCO letter (condition A, B, etc) and the statement in that order, then the S function lost or inop, which I think in this case is e and f with the low action 20 (which you have).
STP made requested changes for draft submittal and from validation 2 week and this scenario event is now Sat.
D1 is needs enhancement for event 3, after dropped rod, then downpower is required as part of this event so they go together, and therefore should be part E of event 3 (not in event 4 or 5)
Put annunciator response procedure in the D-2 for the major alarm window (I guess it would be rod bottom in this case) even if not used but referred to. Need DNBR TS LCO condition letter here also. Gray the downpower procedure on S page 12 0POP03-ZG-0008.
STP made requested changes for draft submittal and from validation 3 week and this scenario event is now Sat.
This event is actually a LOOP, and is the major on event 4 (D1 is needs editing). Need to put the MAJOR alarms at the top of this event in D-2 (such as E
what power alarms come in, rx trip, turbine trip, etc) I would pick the top 5 or six that indicate the story for this event, including the ECW 1A failure alarm.
Need bounding criteria for CT-9 in the D-2 guide on page 14 and page 16.
S STP made requested changes for draft submittal and from validation 4 week and this scenario event is now Sat.
5 S Rev. 11
ES-301 9 Form ES-301-7 Need alarms at top of this event. Again, any switch manipulations done by the E crew here (in addition to what you already have in the guide) need to be in the D-2 guide. Need to put bounding criteria on the CT-EC00-H in the D-2 on page
- 17. CT-24 is not annotated in the D-2 guide nor bolded.
S STP made requested changes for draft submittal and from validation 6 week and this scenario event is now Sat.
Rev. 11
ES-301 10 Form ES-301-7 Facility: South Texas Project Scenario: 3 Exam Date: Sept 25-29, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap Put annunciator response in D-2 for event 1. Also, bullet the alarm window. Place any other applicable alarms here also. Highlight in gray the procedures used. Actual crew E actions on switches belong in required operator actions column not in notes section (I think the only one in notes that is not in the correct column is the mode select switch). Put TS 1 that are applicable with LOCA letters and statement for most limiting on the guide (which you have) I am not sure which LCO goes with this most limiting action though.
S STP made requested changes for draft submittal and from validation week and this scenario event is now Sat.
Other alarms, any annunc response procedure? TS LCO format section E similar to above comments on other TS events. Full action is on same page (page 7) so this note is wrong. Not tied to the particular LCO S though. STP made requested changes for draft submittal and from 2 validation week and this scenario event is now Sat.
Not sure how they assess the leak for either the downpower or rx trip. If E they trip right away then no reactivity event can be credited. Are there no criteria for this? STP changed event during validation to ISO-phase fan trip, still requires downpower.
S STP made requested changes for draft submittal and from validation 3 week and this scenario event is now Sat.
The procedure and diagnostic aspects of the SBLOCA should be in the guide. How the crew works through the decision that they have a SBLOCA (should start with a leak, an Annun response procedure, or E
something, right)? At least need alarms and indications at the top of this event. What is actual leak size? Put that in guide for examiner. CT-16 not bounded in D-2. Page 19 need to highlight the ESF sequencer step and corresponding note so examiner knows that the LOCA increases in S
size at this point to a LBLOCA.
STP made requested changes for draft submittal and from validation 4 week and this scenario event is now Sat.
E Properly bound CT-8 in the D-2.
STP made requested changes for draft submittal and from validation 5 S week and this scenario event is now Sat.
6 E Properly bound CT-3 in D-2.
Rev. 11
ES-301 11 Form ES-301-7 STP made requested changes for draft submittal and from validation S week and this scenario event is now Sat.
Rev. 11
ES-301 12 Form ES-301-7 Facility: South Texas Project Scenario: 4 Exam Date: Sept 25-29, 2017 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap 1 S Put alarm response procedures in guide, gray highlight all used E
procedures.
STP made requested changes for draft submittal and from validation S
2 week and this scenario event is now Sat.
3 S Good TS write-up here.
Put alarm response procedures in guide, gray highlight all used E
procedures.
STP made requested changes for draft submittal and from validation S
4 week and this scenario event is now Sat.
Put alarm response procedures in guide, gray highlight all used E
procedures.
STP made requested changes for draft submittal and from validation S
5 week and this scenario event is now Sat.
Properly bound all CTs in D-2. Need to have specific items that are E required to be manipulated in bold and verify items can be un-bolded within the CT so focus is on the actions to complete the CT.
S STP made requested changes for draft submittal and from validation 6 week and this scenario event is now Sat.
Shouldnt isolation of SG 1D (CT-18) include the failed FWIV-7144 if they miss it in addendum 5? NO-other valves close (MFRV and LPFRV E get an auto close signal so there is no feed path even if this valve is missed). STP added notes to scenario guide for examiners for this S aspect.
7 For CT-20, how long would it take for a SG PORV to open? If it is not E within the timeframe of the scenario then it is not a critical task. STP modified bounding criteria after val week.
S STP made requested changes for draft submittal and from validation 8 week and this scenario event is now Sat.
Rev. 11
ES-301 13 Form ES-301-7 Instructions for Completing This Table:
1 Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f)
- opening, closing, and throttling valves
- starting and stopping equipment
- raising and lowering level, flow, and pressure
- making decisions and giving directions
- acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
- In column 1, sum the number of events.
- In columns 2-4, record the total number of check marks for each column.
- In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
- In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
- In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
- In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
Rev. 11
ES-301 14 Form ES-301-7 Facility: South Texas Project Exam Date: Sept 25-29, 2017 1 2 3 4 5 6 7 8 11 Scenario % Unsat. Explanation Event Events TS TS CT CT Scenario U/E/S Totals Unsat. Total Unsat. Total Unsat.
Elements All comments resolved as shown above, this is only the score sheet for the 1 8 0 E draft submittal.
2 6 0 E 3 6 0 E 4 8 0 E Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a. Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b. TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c. CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
2+4+6 7 In column 7, calculate the percentage of unsatisfactory scenario elements: 100%
1+3+5 8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
Rev. 11
ES-301 15 Form ES-301-7 Site name: Exam Date:
OPERATING TEST TOTALS Total Total Total %
Total Explanation Unsat. Edits Sat. Unsat.
Admin.
9 2 3 4 JPMs Sim./In-Plant 11 0 1 10 JPMs Scenarios 4 0 4 0 Op. Test 24 2 8 14 8 Satisfactory submittal.
Totals:
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1. Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.
For scenarios, enter the total number of simulator scenarios.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and 2.
simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous 3.
tables. This task is for tracking only.
- 4. Total each column and enter the amounts in the Op. Test Totals row.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test 5.
Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
- satisfactory, if the Op. Test Total % Unsat. is 20%
- unsatisfactory, if Op. Test Total % Unsat. is > 20%
Update this table and the tables above with post-exam changes if the as-administered operating test 6.
required content changes, including the following:
- The JPM performance standards were incorrect.
- The administrative JPM tasks/keys were incorrect.
- CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
- The EOP strategy was incorrect in a scenario(s).
- TS entries/actions were determined to be incorrect in a scenario(s).
Rev. 11