ML20301A384
| ML20301A384 | |
| Person / Time | |
|---|---|
| Site: | South Texas  |
| Issue date: | 07/29/2020 |
| From: | Greg Werner Operations Branch IV |
| To: | South Texas |
| References | |
| Download: ML20301A384 (16) | |
Text
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 Facility: South Texas Exam Date: July 13, 2020 1
2 3
Attributes 4
Job Content 5
6 Admin JPMs ADMIN Topic and K/A LOD (1-5)
U/E/S Explanation I/C Cues Critical Scope Overlap Perf.
Key Minutia Job Link Focus Steps (N/B)
Std.
A1 Calculate SCM (2.1.19) 3 S
A2 Determine TTB on Loss RHR during mid-loop (2.1.25) 3 S
A3 Prepare ECO for Main LO cooler (2.2.13) 3 S
A4 Determine RWP requirements for RRA-SFP (2.3.7) 3 S
A5 (SRO)
Calc SCM and apply TS (2.1.19) 3 X
E S
On handout dont cue to RCS subcooling for location of TS. Just ask what TS (if any apply). I think LOD = 3 (not 4 as you have on the worksheet).
After validation comments were included and the JPM is now Sat.
A6 (SRO)
Review RCS inventory and determine TS applicability (2.1.7) 3 X
X E
On the Key at bottom is states that 1gpm wasnt exceeded and this is probably a grouping issue with the brackets since it was exceeded at 1.057 gpm. You probably meant to put this after the acceptance criteria in parenthesis. Also, shouldnt they have to check the unacceptable block for exceeding the TS limits on the form? Put on cue sheet to report all conclusions from the review and determine applicable TS LCOs (if any) based on your review. This
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 S
takes the follow-up question out of the end of the JPM.
Licensee corrected the issues and the JPM is now Sat.
A7 (SRO)
Review ECO tagout for Main LO cooler - faults
( 2.2.13) 4 S
A8 (SRO)
Determine personnel exposure limits E-plan
( 2.3.4) 3 S
A9 (SRO)
EAL (2.4.41) 4 S
1 2
LOD (1-5) 3a I/C 3b Cues 3c Critical 3d Scope 3e Overlap 3f Perf.
3h Key 4a Minutia 4b Job Link 5
U/E/S 6
Explanation Simulator/In-Plant Safety Function and K/A JPMs General comments for all JPMS
- 1.
Some JPMS do not have procedure section submitted with the JPM. You should always send the procedure section to be used for the JPM in the submittal (examples include S2 and S3).
- 2.
It is a good practice to put the step in the JPM or scenario, but it needs to be clear what items they have to manipulate. It is easier to bold the items required to be manipulated in a list of items that are checked in a
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 step but if you choose not to do something like this then you have to put that they Closed valve x, or started pump Y in the expected standard field.
S1 1 (01 A4.06) 3 S
Cant have an M and D for S1. If it is direct from bank it is D, if it was from the bank then you modified it, it is M for modified. Please update the ES-301-2 for this. Same for S5 JPM (has D and M on 301-2).
Completed changes here and in 301-2. It is now Sat.
S2 6 (064 A4.06) 3 E
S Could we have at one more load added before the alt path so we get one more switch manipulation in the JPM? Right now we have a total of two switches manipulated (CCW pump start and diesel trip).
Added a load and JPM is Sat.
S3 3 (010 A2.02) 3 E
S This topic also is a written question (Q42) so one item would need to be changed. The spray valve 655C fails here and on written spray 0655B is the failure. Actions are still the same.
Licensee changed Q42 and left the JPM the same and it is Sat.
S4 8 (075 A2.02) 3 E
S If they can walk down the board and see that the valve is still open for the tripped CW pump then this would not be alt path because they can see the problem before they ever start the JPM. Diagnosis and alt path need to occur once the JPM has started.
Licensee shears shaft now after start of JPM so nothing is given away at beginning and is Now Sat.
S5 5 (026 A2.08) 3 S
S6 4S (059 A4.08) 3 S
S7 7 (073 A4.01) 3 E
S Sometimes you bold the items that must be manipulated and sometimes not. Either do that or place the ones that must be manipulated below the list or state that these valves must be closed (opened, whatever is the required position)
Licensee fixed the bolding issues and JPM is now Sat.
S8 2 (006 A1.13) 3 S
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 P1 6 (062 A4.04) 2 E
S All plant JPMs need to have the examiner use a pen to indicate the switch or button the applicant is intending to manipulate, not actually read the quoted cue for most of these items because this is not allowed per the NUREG-1021. A note on each step to point to the light and state that this light is lit or the breaker is up or the breaker is down, the breaker is to the right position, or to the left position is what is required. True for all in-plant JPMs.
The initiating cue seems odd - is the candidate the Unit Sup or the operator?
Seems like the candidate is the operator.
Why are they ONLY doing train B? From procedure, should do all 3 trains, so from a JPM standpoint, need to include in cue, that fire was in train B.
Need to discuss this procedure during validation week. I think the procedure is lacking - for Step 45, the procedure states to Open Control Power Disconnect Switches, but the action is to remove control power fuses -which are not the same if I read this correctly. Is this like a knife switch or a traditional fuse? If so, how is removing one end so it is not making contact ok? What if a seismic event occurs and fuse moves and re-establishes contact or falls out and contacts other electrical connections? Also, good procedure writing practices would have ONE ACTION per step, not multiple. Step 47 requires placing transfer switches in LOCAL and then START fans, assuming the transfer switches and the control switches are labeled, but not identified in the procedure.
Licensee included pictures to show how the disconnects work (and each has a fuse inside the switch). Gave them comments on procedure as well. JPM is Sat.
P2 2 (006 A4.02) 3 S
P3 5 (103 A2.03) 3 E
S All plant JPMs need to have the examiner use a pen to indicate the switch or button the applicant is intending to manipulate, not actually read the quoted cue for most of these items because this is not allowed per the NUREG-1021. A note on each step to point to the light and state that this light is lit or the breaker is up or the breaker is down, the breaker is to the right position, or to the left position is what is required. True for all in-plant JPMs.
JPM is now Sat.
ES-301 5
Form ES-301-7 Rev. 11 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1.
Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.
(ES-301, D.3 and D.4)
- 2.
Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)
- 3.
In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4.
For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6.
In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
ES-301 6
Form ES-301-7 Rev. 11 Facility: South Texas General comments on scenarios Exam Date: July 13, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation General Comments for all scenarios
- 1.
It is a good practice to put the step in the JPM or scenario, but it needs to be clear what items they have to manipulate. It is easier to bold the items required to be manipulated in a list of items that are checked in a step but if you choose not to do something like this then you have to put that they Closed valve x, or started pump Y in the expected standard field.
- 2.
Measurable Performance indicators for CT table and in the D-2 body of the guide should have the actual switches manipulated for success of the CT. As an example, lets say that a NSSS Group isolation fails to isolate AFW as expected in scenario x, so the applicant must manually close the inbd and outbd isolation valves for AFW. Both of these valves should be listed in the CT table.
See examples below for each scenario but here are the attributes for a CT a) the expected actions and b) the boundary conditions for completing the expected actions.
c)
Performance feedback
- 3.
For parameters to record for grading purposes, we will need to work thru the required parameters to capture for scenarios for grading during validation week.
ES-301 7
Form ES-301-7 Rev. 11 Facility: South Texas Scenario: 1 Exam Date: July 13, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
S 2
2 S
3 X
3 E
S Hard to tell what items get manipulated (if any) by the BOP for this event. It would help if you bolded the items that must be manipulated or state the they must be opened or closed (as appropriate). Is it the RO or BOP that even does this-check the outline from page to make sure who gets the bean is correct on D_1 page.
Licensee made the suggested corrections. Also validation comments were incorporated for pages 1, 2, 3, 6, 9, 10, 15, 16, and 30 and this scenario is now Sat.
4 3
S On pg 16 of 39, Take Appropriate Actions Per Technical Specifications
..., labeled as BOP, but is this directed from the SRO to the BOP (if not then should be assigned to SRO. The next two steps also appear to be SRO, but are labeled as BOP.
Licensee made the suggested corrections. Also validation comments were incorporated for pages 1, 2, 3, 6, 9, 10, 15, 16, and 30 and this scenario is now Sat.
5 3
S Add a note for lead examiner to ok triggering Event 6 (check this for each event as well on all scenarios if missing).
ES-301 8
Form ES-301-7 Rev. 11 Licensee made the suggested corrections. Also validation comments were incorporated for pages 1, 2, 3, 6, 9, 10, 15, 16, and 30 and this scenario is now Sat.
6 3
E S
It makes it hard to grade or see the graded actions for the final three events (due to being lumped together) when there is no clear distinction for actions on what occurs for Event 6, Event 7, and Event 8. Try to separate these events out so the examiner can track what actions are associated with each event and can give proper credit for beans after the major (if needed), especially for component failure for Event 8 -
Train B 4160 lockout.
The expected actions are in the CT table but the boundary conditions for CT-13 appear to be incorrect. Dont you have to complete the immediate action steps before proceeding to anything else? So the bounding conditions would be that (before going to step 5 in EOP?), not what you have in the table. It would also help to have those elements in the D-2 guide for grading considerations. Also, the Performance Feedback is not included in the table or the D-2 guide. This would be something like verifying turbine trip by observing trip breakers open once EHC pumps have tripped or something like that.
The CT table should have all of the attributes of the CT in it to properly define the CT. These include the following items:
a) the expected actions and b) the boundary conditions for completing the expected actions.
c) Performance feedback from the actions taken Licensee made the suggested corrections. Also validation comments were incorporated for pages 1, 2, 3, 6, 9, 10, 15, 16, and 30 and this scenario is now Sat.
ES-301 9
Form ES-301-7 Rev. 11 7
3 E
S The expected actions are in the CT description in the D-2 guide for CT-46 but the boundary conditions are in the table only and need more detail for grading. The pressure that HHI starts to feed is the parameter value to monitor for perf. Feedback that the task has been completed successfully.
Also, for this event, you need to put what equipment is failed into the guide so the examiner will know what is broken (I see AFW #13 fails, AFW #11 out of service and I guess AFW #12 is lost with the safety bus loss. It would help the examiner to have all of that together in the D-2 guide before the feed and bleed starts. The reason for this is that an applicant might try to restart something or something like that for deranged equipment.
In reading C-46, appears that another failure criteria would be the failure to trip the RCPs in a timely manner. If this is correct, what is a timely manner - needs to be defined.
The event has been corrected and the RCP trip criteria are bounded by the WCAP for feed and bleed and occurs prior to Orange path on heat sink. Licensee made the suggested corrections. Also validation comments were incorporated for scenario 1 and this scenario is now Sat.
8 3
S
ES-301 10 Form ES-301-7 Rev. 11 Facility: South Texas Scenario: 2 Exam Date: July 13, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
3 S
2 3
S 3
X 3
S 4
X 3
S 5
3 S
6 3
S 7
3 E
S It isnt clear if all the steps for CT-17 are located only on page 23 of the D-2 or on other pages as well. All the items necessary to complete CT-17 and all other CTs for that matter should be placed in the CT table for clarity. This means a list of valves, sequencers, pumps, whatever is required to meet the CT.
Licensee made the suggested corrections. Also validation comments were incorporated into scenario 2 and this scenario is now Sat.
8 3
S
ES-301 11 Form ES-301-7 Rev. 11 Facility: South Texas Scenario: 3 (spare) Exam Date: July 13, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation Review all comments on CTs from above for remaining scenario CTs to ensure all aspects are included in the scenario guides (bounded properly, proper feedback)-Licensee completed this action.
1 3
S 2
3 S
3 3
E S
I dont see any actions for the RO or BOP for the DRPI failure. If so then this cant be counted as an I failure for either applicant.
Licensee made the corrections and also incorporated validation comments for scenario 3 and this scenario is now Sat.
4 3
S 5
3 S
6 3
S 7
3 S
8 3
S The CT for the SG PORVs is located at the end of event 8 so on the D-1 form it should reflect CT #CTEO10DSTP on Event 8.
Licensee made the corrections and also incorporated validation comments for scenario 3 and this scenario is now Sat.
ES-301 12 Form ES-301-7 Rev. 11 Facility: South Texas Scenario: 4 Exam Date: July 13, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
3 S
2 2
S 3
3 S
4 4
S 5
3 S
6 3
S 7
3 S
Is there any time requirement after you go below 32,500 gal that is needed to prevent pump damage? In other words, the CT is split here so how much below 32,500 is okay or how long after reaching this point before gas intrusion occurs?
Licensee made the corrections above (before you lose flow is answer and these bounding criteria are in the CT task table) and also incorporated validation comments for scenario 4 and this scenario is now Sat.
ES-301 13 Form ES-301-7 Rev. 11 Facility: South Texas Scenario: 5 Exam Date: July 13, 2020 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation 1
3 S
Licensee incorporated validation comments for scenario 5 and this scenario is now Sat.
2 3
S 3
3 S
4 3
S 5
4 S
6 3
S 7
3 S
ES-301 14 Form ES-301-7 Rev. 11 Instructions for Completing This Table:
1 Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
In column 1, sum the number of events.
In columns 2-4, record the total number of check marks for each column.
In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)
In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
ES-301 15 Form ES-301-7 Rev. 11 Facility: South Texas Exam Date: July 13, 2020 Scenario 1
2 3
4 5
6 7
8 11 Event Totals Events Unsat.
TS Total TS Unsat.
% Unsat.
Scenario Elements U/E/S Explanation 1
8 0
2 0
2 0
0 E
S See above comments for edits All edits completed for post val submittal.
2 8
0 2
0 2
0 0
E S
See above comments for edits All edits completed for post val submittal.
3 8
0 2
0 2
0 0
E S
See above comments for edits All edits completed for post val submittal.
4 7
0 2
0 2
0 0
S All edits completed for post val submittal.
5 7
0 2
0 2
0 0
S All edits completed for post val submittal.
Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a.
Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b.
TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c.
CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
7 In column 7, calculate the percentage of unsatisfactory scenario elements:
8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
2 + 4 + 6 1 + 3 + 5100%
ES-301 16 Form ES-301-7 Rev. 11 Site name: South Texas Exam Date: July 13, 2020 OPERATING TEST TOTALS Total Total Unsat.
Total Total Unsat.
Explanation Edits Sat.
Admin.
JPMs 9
0 2
7 Sim./In-Plant JPMs 11 0
6 5
Scenarios 5
0 3
2 Op. Test Totals:
25 0
11 14 0.0%
Satisfactory submittal.
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1.
Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.
For scenarios, enter the total number of simulator scenarios.
- 2.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
- 3.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.
- 4.
Total each column and enter the amounts in the Op. Test Totals row.
- 5.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
- satisfactory, if the Op. Test Total % Unsat. is 20%
- unsatisfactory, if Op. Test Total % Unsat. is > 20%
- 6.
Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:
- The JPM performance standards were incorrect.
- The administrative JPM tasks/keys were incorrect.
- CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
- The EOP strategy was incorrect in a scenario(s).
- TS entries/actions were determined to be incorrect in a scenario(s).