ML20303A032
| ML20303A032 | |
| Person / Time | |
|---|---|
| Site: | Millstone  |
| Issue date: | 09/18/2020 |
| From: | David Silk NRC Region 1 |
| To: | |
| Jackson D | |
| Shared Package | |
| ML19105A205 | List: |
| References | |
| 50-336/20-301, CAC 000500 | |
| Download: ML20303A032 (13) | |
Text
ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Millstone 2 (RO JPMs - See Draft Op Test Outlines for Titles) Exam Date: 9/21/20 - 9/30/20 1 2 3 4 5 6 Attributes Job Content ADMIN Topic LOD Admin JPMs U/E/S Explanation and K/A (1-5) I/C Critical Scope Perf. Job Cues Overlap Key Minutia Focus Steps (N/B) Std. Link A1R (1 CO) 2.1.25 3 S A2R (1 CO) 2.1.25 3 S A3R (2 EC) 2.2.44 3 X E Move cue in step 5 to initiating cue.
A4R (1 RC) 2.3.11 3 X E Add specificity to task standard (expected flow rate) 1 Simulator/In-Plant Safety Function and JPMs K/A Overlap with Scenario 3 event 7 - Modify one of S1 SF 1 029 EA1.12 3 X U these items to avoid overlap.
S2 SF 2 006 A1.13 3 E Specify target band for SIT level & pressure Verif. steps (procedure steps 2.d - 2.1, & 2.k)are not S3 SF 3 074 EA1.05 3 X E critical. Change to non-critical. Remove step 6 cue.
S4 SF 4P 010 A2.02 3 X E Change initiate cue to say You are the ATC Modify task standard to state task to be performed IAW the procedure. Verification steps (procedure S5 SF 5 022 A4.01 3 X X E steps 4.4.2 a & b and 4.2.2 a-c) are not critical.
Change to non-critical. Fix task standard typo.
Move third bullet in the initiating cue to the initial conditions. Change task standard to state that task S6 SF 6 062 A4.01 3 X X X E is to be done IAW the procedure. Make all of JPM step 4 critical. Swap critical steps in JPM step 13.
S7 SF 7 013 A4.02 3 X E Make all of JPM step 10 critical.
S8 SF 8 008 A2.01 3 X E Make JPM step 2 critical.
P1 SF 4S 059A2.11 3 X E Fix initial conditions & cue.
Specify task standard (done IAW xxx). Shorten P2 SF 6 062A3.04 3 X ? E Initiating cue. Are JPM steps 2 and 12 critical?
P3 SF 8 064 K1.02 3 E Specify task standard (done IAW xxx).
ES-301 2 Form ES-301-7 Facility: Millstone 2 (SROI/SROU JPMs - See Draft Op Test Outlines for Titles) Exam Date: 9/21/20 - 9/30/20 1 2 3 4 5 6 Attributes Job Content ADMIN Topic LOD Admin JPMs U/E/S Explanation and K/A (1-5) I/C Critical Scope Perf. Job Cues Overlap Key Minutia Focus Steps (N/B) Std. Link A1S (CO) 2.1.27 3 X E Delete cue about AOP2588.
Narrow and adjust initiating cue to focus applicant A2S (CO) 2.1.36 3 X E and to eliminate need for examiner cues.
A3S (EC) 2.2.44 3 X E Update and enlarge photo of C08. Add TS to step 2.
Change task standard to state reasons for non-A4S (RC) 2.3.6 3 X E approval. Add details in step 2.
A5S (EP) 2.4.41 3 X E Specify EAL & PAR in task standard.
1 Simulator/In-Plant Safety Function and JPMs K/A S1 SF 1 029 EA1.12 3 See comments on RO JPMs above.
S2 SF 2 006 A1.13 3 See comments on RO JPMs above.
S3 SF 3 074 EA1.05 3 See comments on RO JPMs above.
S4 SF 4P 010 A2.02 3 See comments on RO JPMs above.
S5 SF 5 022 A4.01 3 See comments on RO JPMs above.
S7 SF 7* 013 A4.02 3 See comments on RO JPMs above.
S8 SF 8* 008 A2.01 3 See comments on RO JPMs above.
P1 SF 4S 059A2.11 3 See comments on RO JPMs above.
P2 SF 6 062A3.04 3 See comments on RO JPMs above.
P3 SF 8 064 K1.02 3 See comments on RO JPMs above.
Note: * - indicates SROU JPM
ES-301 3 Form ES-301-7 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1. Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.
(ES-301, D.3 and D.4)
- 2. Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)
- 3. In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4. For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6. In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
ES-301 4 Form ES-301-7 Facility: Millstone 2 (See D-1 Forms for Details) Scenario: 1 Exam Date: 9/21/20 - 9/30/20 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap 1 S 2 S 3 X S 4 X E Swap order of events 4 and5.
5 X E 6 XXX U CT about tripping RCPs is not critical.
7 X S 7 0 0 0 0 3 3 0 E
ES-301 5 Form ES-301-7 Facility: Millstone 2 (See D-1 Forms for Details) Scenario: 2 Exam Date: 0/21/20 - 09/30/20 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap 1 X S 2 X X E Add actions and cue for bypassing channel.
3 X E Add details for cross-tying flow thru water box.
4 X X E Change description of event to controller failure vs PT failure. Add BOP may lower power.
5 X E Add details for ATC borating from the RWST.
6 S Insert event 6 after BOP lowers turbine load.
7 XX X S 7 0 4 0 0 2 2 2 E
ES-301 6 Form ES-301-7 Facility: Millstone 2 (See D-1 Forms for Details) Scenario: 3 Exam Date: 9/21/20 - 9/30/20 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap 1 X E Add switch numbers for manipulations.
2 X E Typo-Replace TS 3.7.1.6.A with B.
3 X E SGSV leakage and down power were combined.
4 XXX S 5 X X U CT not critical because on pre-emptive manual trip.
6 6 0 0 0 0 2 3 1 E
ES-301 7 Form ES-301-7 Facility: Millstone 2 (See D-1 Forms for Details) Scenario: 4 Exam Date: 9/21/20 - 9/30/20 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap 1 E Put PZR level controller in Manual vs Auto.
2 X E Fix typo wrt alarms (D vs B). Add TS info for 3.3.1.1.
3 E Fix typo wrt B vs A SGFP in auto.
4 X X S 5 XX E Clarify CT1 (prior to 165).Delete 2nd CT. Bold step to establish condensate flow to SG1.
6 X S 7 XX X U Clarify CT 3 (prior to exceeding 200F subcooling. CT 4 is redundant - delete.
7 0 0 0 0 2 3 3 E
ES-301 8 Form ES-301-7 Facility: Millstone 2 (See D-1 Forms for Details) Scenario: 5 Exam Date: 9/21/20 - 9/30/20 1 2 3 4 5 6 7 8 9 10 Required Verifiable Scen.
Event Realism/Cred. LOD TS CTs U/E/S Explanation Actions actions Overlap 1 S 2 S 3 X S 4 X X X E Add details about borating form the BAST.
5 X S Add details about emergency boration. Script C/D and X XX XX E 6 depressurization details.
6 0 0 0 0 2 2 4 E
ES-301 9 Form ES-301-7 Instructions for Completing This Table:
Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f)
- opening, closing, and throttling valves
- starting and stopping equipment
- raising and lowering level, flow, and pressure
- making decisions and giving directions
- acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
- In column 1, sum the number of events.
- In columns 2-4, record the total number of check marks for each column.
- In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
- In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
- In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
- In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
ES-301 10 Form ES-301-7 Facility: Millstone Unit 2 Exam Date: 9/21/20 - 9/30/20 1 2 3 4 5 6 7 8 11 Scenario % Unsat. Explanation Event Events TS TS CT CT Scenario U/E/S Totals Unsat. Total Unsat. Total Unsat.
Elements 1 7 1 3 0 3 1 E Scenario still had minimum required CTs 2 7 0 2 0 3 0 E 3 6 1 2 0 2 1 E Scenario still had minimum required CTs 4 7 1 2 0 3 1 E Scenario still had minimum required CTs 5 6 0 2? 0? 2 0 E Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a. Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b. TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c. CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
2+4+6 7 In column 7, calculate the percentage of unsatisfactory scenario elements: 100%
1+3+5 8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
ES-301 11 Form ES-301-7 Site name: Millstone Unit 2 Exam Date: 9/21/20 - 9/30/20 OPERATING TEST TOTALS Total Total Total %
Total Explanation Unsat. Edits Sat. Unsat.
Admin.
9 0 7 9 JPMs Sim./In-Plant JPM was unsat because it overlapped with a 11 1 10 11 JPMs scenario event and needed to be changed.
Scenarios 5 0 5 5 Op. Test 25 0 22 24 4.0% Licensee made the necessary changes.
Totals:
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1. Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.
For scenarios, enter the total number of simulator scenarios.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and 2.
simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous 3.
tables. This task is for tracking only.
- 4. Total each column and enter the amounts in the Op. Test Totals row.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test 5.
Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
- satisfactory, if the Op. Test Total % Unsat. is 20%
- unsatisfactory, if Op. Test Total % Unsat. is > 20%
Update this table and the tables above with post-exam changes if the as-administered operating test 6.
required content changes, including the following:
- The JPM performance standards were incorrect.
- The administrative JPM tasks/keys were incorrect.
- CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
- The EOP strategy was incorrect in a scenario(s).
- TS entries/actions were determined to be incorrect in a scenario(s).
Millstone 2 Operating Test Outline Comments (Rev 0):
A1R and A1S were designated by the licensee to be COO admin JPMs. However, because the content of these JPMs dealt with assessing the plant electrical system in light of an out-of-service EDG, the examiner concluded that this was an EC admin JPM. The examiner directed the license to designate these JPMs as EC admin JPMs (because they were New JPMS) and to eliminate both the RO and SRO admin JPMs that were proposed as EC admin JPMs for this exam.
The examiner recommended that the Spare RO admin JPM be used as a COO admin JPM.
The examiner directed that the licensee find a replacement COO admin JPM for the SRO admin JPM section.
The examiner noted that no in-plant JPMs were designated as E for emergency or abnormal actions. The licensee had forgotten to designate JPM P3 as an E and will make the correction on the final outline submittal.
The examiner informed the licensee that only one scenario (#4) used an EOP contingency procedure. The examiner explained that per NUREG-1021 Appendix D page D-10 that each scenario set needed an event such that every applicant was in at least one scenario that required implementation of an EOP contingency procedure. Therefore, due to the size and make up of the class, another scenario was needed that required usage of an EOP contingency procedure. There licensee will modify another scenario so that an EOP contingency procedure will be implemented.
The examiner informed the licensee that one scenario needed to be a low power because there were no low power scenarios submitted or used in the previous two exams (2016 and 2018).
Region 1 attempts to implement a policy of having at least one low power scenario used in every other exam to help ensure the unpredictability of the scenarios and to try to meet the intent of NUREG-1021.
In Scenario #1, the examiner challenged the validity of CT#3 as it did not seem to be safety significant to control an RCS cooldown given that a large break LOCA had occurred. The licensee will check/review the site CT documentation.
Also in Scenario #1, the examiner directed the licensee to delay the start of the SGTR to occur several minutes after the trip (instead of during the trip) so that there would be at least one malfunction that occurs after EOP entry.
The examiner directed the licensee to include the Tech Spec calls in Scenario #2 as they were not designated on the D-1 Form nor on the 301-5 Form. There appear that a Tech Spec-related event needs to be added to the scenario.
The examiner directed the licensee to include the bean count for Scenario #4 on the 301-5 Form.
The examiner explained to the licensee that power changes which are the mitigative actions for a component malfunction should not be counted as stand-alone reactivity events. Therefore, Scenario #2 Event 7 should be combined with Event 6; Scenario #3 Event 4 should be combined with Event 3; and Scenario #4 Event 5 should be combined with Event 4.
Comments on Revise Millstone Op Test Outline Submitted on June 12, 2020 The licensee submitted an additional scenario (A fifth one) to support an exam schedule of four days of scenarios with two or three runs instead of three days of scenarios with four runs.
There were numerous errors on Form ES-301-5. There were instances of the math errors for the TOTAL column. Also, some events were credited to the wrong position. Some events were missing from the form. Some events were placed in the wrong category. The licensee was provided with several examples and was directed to revise the form (after incorporating the comments below) and ensure its accuracy.
No comments on Scenario 1.
Scenario 2 Corrected event 2 as only being a TS item for the SRO as there were no board actions.
Combined events 5 and 6 since the down power is the mitigative action for event 5.
Made event 7 a C for the BOP to trip the unit.
The third CT was not a CT as it specified actions that the crew was expected to do properly.
Scenario 3 Combine events 3 and 4 since the down power is the mitigative action for event 3.
The major events are similar to scenario 4 (ESD outside vs inside CNMT) therefore adjust crew rotations so no crew is in both scenarios 3 and 4.
Scenario 4 Events 4 and 5 should be combined since the down power is the mitigative action for event 4.
Break out event 6 item for a malfunction for the BOP.
Combine events 8 and 9 since together they create the intended effect.
Delete CT3 as it is redundant to CT2.
Scenario 5 Event 2 is an event for the BOP not ATC.
Events 3 and 4 should be combined since the down power is the mitigative action for event 3.
Event 5 is an event for the AT and the BOP.
Break up event 6 into separate events.
There is only one CT. There are supposed to be 2 or more. Create a second CT.