ML20042C045
| ML20042C045 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 10/12/2019 |
| From: | Greg Werner Operations Branch IV |
| To: | Arizona Public Service Co |
| References | |
| Download: ML20042C045 (15) | |
Text
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 Facility: Palo Verde Exam Date: October 7, 2019 1
2 3
Attributes 4
Job Content 5
6 Admin JPMs ADMIN Topic and K/A LOD (1-5)
U/E/S Explanation I/C Cues Critical Scope Overlap Perf.
Key Minutia Job Link Focus Steps (N/B)
Std.
General comments:
- 1. Admin JPMs that have a KEY should be included for each admin JPM and correctly marked as KEY for A4, as an example.
- 2. Initiating Cues should match between examiner record and handout page to applicant.
- 3. Try to avoid cueing in the questions for the applicant to answer in the Admin JPM. If you provide a form to fill out with blanks, just ask them to complete the form or table.
A1 Determine regulatory requirements for conditions of the license (2.1.4) 1 X
X E
This JPM has cueing in the initiating cue and does not have any JPM steps marked as critical.
The cue should read something like this:
- 1) List any reportable changes, the organization, and associated time requirements based on this medical evaluation
- 2) List additional changes to the conditions of the license (if any) for the individual A2 Determine total RCS Identified leakrate (2,2,12) 3 X
X E
Need to have Key for A2 in RED at top of Key for this JPM to ensure it is not handed out and for examiners during grading. Box the answer in Red for each JPM step that is critical for grading purposes. Example, 1741.97 is the first number listed as
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 critical in JPM and on the KEY it should be Boxed in Red.
You cant have a range for these steps since all values are exact (pulled from data, table). The actual value I got for step 1 is 1741.97 using your given steam table values in the key. This would be the answer with no error bars. I got 1741.56 for final (step 2) and for change in RDT volume step 3 of JPM,
-0.42 gallons. This should be the answers and we can discuss it when we see your steam tables there at the site during prep week.
A3 Minimize contaminated release from secondary (2.3.14) 3 X
E In the question after the initiating cue, you should just ask the applicant to circle the appropriate answers for each pump suction below since you list all three below the water level elevation blank.
A4 Change RCS Level while in a drained condition (2.1.25) 3 X
E This JPM needs to have a KEY to show what a value was pulled from the graph in App W and the extraction of data from the core data book to arrive at the answers.
A5 (SRO)
Assess risk for unplanned Maintenance (2.1.39) 3 X
X E
This JPM should have a KEY to show what values were pulled from the specific table of each Appendix or at least put the appendix and page number in the JPM steps (page 1 of App D, page 3 of App E, etc). JPM step 4 is missing the details for Item 2 in the list of mandatory items. Could have a footnote for examiner that item 22 is not applicable since this is not a reactivity event.
A6 (SRO)
Fatigue Rule Applicability (2.1.5) 3 X
E The cue should just be to complete the table below by circling the appropriate responses.
A7 (SRO)
Ensure compliance with TS
( 2.2.40) 3 X
E The cue should just be to determine all applicable TS LCOs and required actions. You should also
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 put the three inop valves (577, 560, and 694) in the JPM standard A8 (SRO)
Ensure Compliance with ODCM
( 2.3.11) 3 X
E The cue should just be to complete the table below. Dont cue to ODCM.
A9 (SRO)
Complete Emergency Form (2.4.40) 2 X
E No Marked up EP-0541 form as Key provided for this JPM. K/A is wrong.
It has to be from section 4 of the generic KAs so 2.4.40 would work but you guys need to figure out which one applies for your LOs for this activity. It is not really a PAR nor is it an EAL call since you give them that. The only real SRO item is determining that this SGTR release is within federal limits.
1 2
LOD (1-5) 3a I/C 3b Cues 3c Critical 3d Scope 3e Overlap 3f Perf.
3h Key 4a Minutia 4b Job Link 5
U/E/S 6
Explanation Simulator/In-Plant Safety Function and K/A JPMs General comments for all JPMS
- 1.
JPM standards should have a clear line item for where/when in the JPM the alternate path starts.
- 2.
Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the standard is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators,
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 etc, should be in the notes or in separate lines between the steps of the JPM.
- 3.
Should incorporate procedure steps in the JPM standard just below the JPM step 2 box.
S1 1 (029 EA1.08) 2 E
Need to mark where the alternate path begins in the JPM (before step 5 of this JPM)
S2 2 (013 A3.02) 3 X
E Need a value range for adequate flow in the JPM Task standard. JPM step 6 is the RNO for the procedure and therefore is the alt path start of the JPM. Need to mark where the alternate path begins in the JPM on all alt path JPMs.
S3 4P (005 A4.01) 3 E
Should JPM step 1 state in the standard Throttled.without tripping pumps?
S4 4S (62 AA1.02) 3 X
E Need system number on KA (62) JPM standard step 2 the bottom group of valves (the 500 series valves) have a specific combination to ensure blowdown is stopped and this is not clear in this critical step. It needs to be repaired to state the exact combination of valves necessary to ensure blowdown is stopped. Step 3 of JPM standard needs to have a value for feed flow instead of as needed to be legally defendable. Step 4 of JPM standard is ensure both MFPs are tripped but the standard should state that Tripped both MFPs if they are required to be tripped. This is always the difference in the JPM step and the standard (ie what is actually expected for switch manipulation).
This is especially important on this step if it is critical as marked. The history section should state that this is the normal path version of JPM S5-2016 NRC Exam.
S5 5 (026 A2.08) 4 S
S6 6 (062 A4.01) 3 X
E This JPM needs to have clarity in its explanation on why it does not overlap with CT1 on scenario2 for this exam. It is also not marked on the cover sheet as time critical when it is a TC JPM. One suggestion to fix this JPM is to have power come from the alt feeder breaker (ie the alt feeder is closed so the EDG breaker cant close so either trip the alt feeder breaker so the EDG can close on the bus or close the alt feeder breakers supply breaker directly above the alt feeder breaker and power from offsite power or a transformer. Simple repair but a new JPM.
S7 7 (012 A4.06) 3 S
S8 9 (060 AA2.05) 2 X
U This JPM is not alt path as written (setup is not correct). You cant start them in the appendix A and have that be alt path. This
ES-301 Operating Test Review Worksheet Form ES-301-7 Rev. 11 has been discussed heavily at NEI national workshops. You need to start them somewhere else first. I suggest that you use CREFAS system with a high alarm on RU29/30, which then leads to alarm windows 5A04A/B, which they then verify it is a valid hi-hi alarm, then see that CREFAS is lit but all components did not realign, and they have to realign them per a different procedure than the ARI. CRVIAS is always manually started only so it really isnt as good at setting up as RNO from an alarm where you want an alt path JPM. You could generate the hi-hi alarm (2 e 5) by sluicing or something like that but you dont want to complicate the JPM with other ESFAS system initiations such as CPIS, etc.
P1 8 (065 AA2.03) 3 E
Coversheet is not marked as Alt path. Need to mark where the alternate path begins in the JPM (before step 10 of this JPM).
Probably need a picture of the relief valve (specifically on components need to operate) in each JPM packet if we are not going to use a ladder.
P2 6 (064 A4.01) 3 X
E All JPMs need to have the examiner use a pen to indicate the switch or button the applicant is intending to manipulate, not actually read the quoted cue for most of these items because this is not allowed as a general rule. Cues are incorrect on steps 4 and 5 for in-plant JPM cues. You should use something like the switch is to the left or right.
P3 2 (004 A4.08)
X E
All JPMs need to have the examiner use a pen to indicate the switch or button the applicant is intending to manipulate, not actually read the quoted cue for most of these items because this is not allowed as a general rule. Instead of providing the valve name the examiner must use as an example the valve (that the applicant indicated was being manipulated) will not rotate in the CCW direction and the valve stem is extended.
ES-301 6
Form ES-301-7 Rev. 11 Instructions for Completing This Table:
Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.
- 1.
Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.
(ES-301, D.3 and D.4)
- 2.
Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)
- 3.
In column 3, Attributes, check the appropriate box when an attribute is not met:
The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)
The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)
All critical steps (elements) are properly identified.
The scope of the task is not too narrow (N) or too broad (B).
Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)
The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.
A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).
- 4.
For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:
Topics are linked to the job content (e.g., not a disguised task, task required in real job).
The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)
- 5.
Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.
- 6.
In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.
Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.
ES-301 7
Form ES-301-7 Rev. 11 Facility: Palo Verde General comments on scenarios Exam Date: October 7, 2019 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation General Comments for all scenarios
- 1.
EVERY SWITCH that is touched must be in the D-2 guide. For example, during a downpower, if they use turbine or boration, the specific steps to accomplish either of these must be included for the event.
- 2.
When several components are broken to create an event, it counts as one overall malfunction (such as a pump trip with its standby pump failing to auto start - this is counted as 1 malfunction or 1 event).
- 3.
Measurable Performance indicators for CT table and in the D-2 body of the guide should have the actual switches manipulated for success of the CT. As an example, lets say that a NSSS Group isolation fails to isolate AFW as expected in scenario x, so the applicant must manually close the inbd and outbd isolation valves for AFW. Both of these valves should be listed in the CT table.
As an example for scenario 1, for the CT table on page 3 of D-1, the measurable performance indicator for CT-2 must include BOTH a) the expected actions and b) the boundary conditions for completing the expected actions.
For CT-2 the expected actions are not in the Measurable PI description in the table. Also, the Performance Feedback is incomplete in that power can be verified by observing voltage on the S04 bus (which you have in the wrong item, the Meas PI section).
- 4.
In the D-2 form for each CT, you need to include the expected actions (ie the board manipulations) necessary to complete the CT in the body of the D-2 (page 19 and 21).
- 5.
For parameters to record for grading purposes, we will need to work thru the required parameters to capture for scenarios for grading during validation week.
ES-301 8
Form ES-301-7 Rev. 11 Facility: Palo Verde Scenario: 1 Exam Date: October 7, 2019 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation Place abbreviated CT-1 and CT-2 just under turnover on D-1 first page.
1 3
X E
D-1 form is wrong with failure to 100% versus D-2 of 60%. We need to document that this is a different kind of failure than the final elements in scenario1. It appeared as if we had two of these malfunctions of the same type (ie SG water level transmitter failures) and this is not true.
The licensee is changing the guide for this this event so it can be clear that one is a control configuration channel failure while the other event on scenario2 is a safety configuration channel.
Anyway to a TS event because we need two TS items and the second TS item starts after the major event is in progress (event 6) and this is not allowed per NUREG-1021.
2 3
S 3
3 S
4 3
S 5
3 S
6 3
E This TS item starts way before the major event. We should give them the small RCS leak first in order to address TS calls for this event,. Then ramp the size up so that the crew can diagnose the leak. The NUREG-1021 does not allow a TS call once a Major is already in progress. If we bring in the leak early Nothing else needs to be changed on the major.
ES-301 9
Form ES-301-7 Rev. 11 7
2 E
For the CT table on page 3 of D-1, the measurable performance indicator for CT-2 must include BOTH a) the expected actions and b) the boundary conditions for completing the expected actions.
For CT-2 the expected actions are not in the Measurable PI description in the table. Also, the Performance Feedback is incomplete in that power can be verified by observing voltage on the S04 bus (which you have in the wrong item, the Meas PI section).
In the D-2 form for each CT, you need to include the expected actions (ie the board manipulations) necessary to complete the CT in the body of the D-2 (page 19 and 21).
8 3
S 9
3 E
This event is really event 7 and the associated CT-1 goes with it for restoring feed. Move this to event 7 and move event 7 down and CT-2 will go with it on the order it appears in the D-2. Ensure CT-1 has the boundary conditions in the Meas PI in the table.
ES-301 10 Form ES-301-7 Rev. 11 Facility: Palo Verde Scenario: 2 Exam Date: October 7, 2019 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation Place abbreviated CT-1 and CT-2 just under turnover on D-1 first page.
There are 9 malfunctions not 10 in this scenario (Event 1 counts as 1).
Update target attribute table for 9 malfunctions.
1 3
S 2
3 S
3 X
3 E
EVERY SWITCH that is touched must be in the D-2 guide. For the downpower, if they use turbine or boration, the specific steps to accomplish either of these must be included at this event (put both in with conditional as listed page 10) 4 X
3 E
Need switches for turbine load reduction on page 12. You have the boration switches in this event (put them in event 3 above too).
5 3
E List the second CEA that drops (D-1 and D-2) 6 3
S 7
3 E
CT-1 is here so on CT table (page 3) need to place boundary conditions in Meas PI for both CTs.
8 3
S 9
4 E
CT-2 is here and need to place boundary conditions in Meas PI (page 3). Also, need to clarify in the title of CT-2 that this is cross-connecting EDG A to the B train switchgear.
ES-301 11 Form ES-301-7 Rev. 11 Facility: Palo Verde Scenario: 3 Exam Date: October 7, 2019 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation Place abbreviated CT-1 and CT-2 just under turnover on D-1 first page.
There are 7 malfunctions not 8 in this scenario (Event 4 counts as 1).
Update target attribute table for 7 malfunctions.
1 3
S 2
3 S
3 3
S 4
4 E
Counts as one malfunction for purposes of event 4.
5 3
S 6
3 E
CT-1 is here on D-1 form. On CT table (page 3) need to place boundary conditions in Meas PI for both CTs (copy from CT title). Also, are H2 analyzers needed for all LOCAs (ie a CT) per USAR or just LB LOCA? Need valve numbers for two valves in CT (both in table and in the body of D-2).
7 3
E CT-2 is here on D-1 form. On CT table (page 3) need to place boundary conditions in Meas PI for both CTs (copy from CT title). Also, the closure of the RWT valves (530 and 531) are critical to ensuring proper NPSH for pumps drawing suction from the sump so these should be part of the CT, correct?
ES-301 12 Form ES-301-7 Rev. 11 Facility: Palo Verde Scenario: 4 Exam Date: October 7, 2019 1
2 3
4 5
6 7
8 9
10 Event Realism/Cred.
Required Actions Verifiable actions LOD TS CTs Scen.
Overlap U/E/S Explanation Place abbreviated CT-1 and CT-2 just under turnover on D-1 first page.
There are 10 malfunctions not 11 in this scenario (Event 1 counts as 1).
Update target attribute table for 10 malfunctions.
1 3
S 2
2 S
3 4
S 4
3 S
5 3
S 6
3 S
7 3
S 8
3 S
9 3
S 10 3
E CT-1 and CT-2 are here on D-1 form. On CT table (page 3) need to place boundary conditions in Meas PI for both CTs (copy from CT title). Also need to put actual valve numbers for all necessary valves in CT-2 (both in table and in the body of D-2) to meet the critical task. It also helps if you put a blank for the time that the CIAS setpoint was exceeded and its value (helps the examiner).
ES-301 13 Form ES-301-7 Rev. 11 Instructions for Completing This Table:
1 Use this table for each scenario for evaluation.
2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.
3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)
5 Check this box if the level of difficulty is not appropriate.
6 Check this box if the event has a TS.
7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.
8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.
10 Record any explanations of the events here.
In the shaded boxes, sum the number of check marks in each column.
In column 1, sum the number of events.
In columns 2-4, record the total number of check marks for each column.
In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.
In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)
In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)
In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)
In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.
ES-301 14 Form ES-301-7 Rev. 11 Facility: Palo Verde Exam Date: October 7, 2019 Scenario 1
2 3
4 5
6 7
8 11 Event Totals Events Unsat.
TS Total TS Unsat.
% Unsat.
Scenario Elements U/E/S Explanation 1
9 0
2 0
2 0
0 E
See above comments for edits 2
9 0
2 0
2 0
0 E
See above comments for edits 3
7 0
2 0
2 0
0 E
See above comments for edits 4
10 0
2 0
2 0
0 E
See above comments for edits Instructions for Completing This Table:
Check or mark any item(s) requiring comment and explain the issue in the space provided.
1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).
This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).
2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:
- a.
Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.
- b.
TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)
- c.
CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.
7 In column 7, calculate the percentage of unsatisfactory scenario elements:
8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.
9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.
Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.
2 + 4 + 6 1 + 3 + 5100%
ES-301 15 Form ES-301-7 Rev. 11 Site name: Palo Verde Exam Date: Exam Date: October 7, 2019 OPERATING TEST TOTALS Total Total Unsat.
Total Total Unsat.
Explanation Edits Sat.
Admin.
JPMs 9
1 8
0 Sim./In-Plant JPMs 10 0
8 2
Scenarios 4
0 4
0 Op. Test Totals:
23 1
20 2
4.3%
Satisfactory submittal.
Instructions for Completing This Table:
Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.
- 1.
Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.
For scenarios, enter the total number of simulator scenarios.
- 2.
Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.
- 3.
Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.
- 4.
Total each column and enter the amounts in the Op. Test Totals row.
- 5.
Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.
Refer to ES-501, E.3.a, to rate the overall operating test as follows:
- satisfactory, if the Op. Test Total % Unsat. is 20%
- unsatisfactory, if Op. Test Total % Unsat. is > 20%
- 6.
Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:
- The JPM performance standards were incorrect.
- The administrative JPM tasks/keys were incorrect.
- CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).
- The EOP strategy was incorrect in a scenario(s).
- TS entries/actions were determined to be incorrect in a scenario(s).