ML18304A014

2018-09 Draft Operating Test Comments

ML18304A014
Person / Time
Site:	Palo Verde
Issue date:	09/26/2018
From:	Vincent Gaddy Operations Branch IV
To:	Arizona Public Service Co
References
50-528/18-09, 50-529/18-09, 50-530/18-09 50-528/OL-18, 50-529/OL-18, 50-530/OL-18
Download: ML18304A014 (8)

Text

ES-301 Operating Test Review Worksheet Form ES-301-7 Facility: Palo Verde (2018-302 JPM Retake Exam) Exam Date: September 26, 2018 1

2 3

Attributes 4

Job Content 5

6 Admin JPMs ADMIN Topic and K/A LOD (1-5)

U/E/S Explanation I/C Cues Critical Scope Overlap Perf.

Key Minutia Job Link Focus Steps (N/B)

Std.

Evaluate crew staffing in each unit and determine if the unit is in compliance with Conduct of Operations and Technical Specifications Conduct of Operations G 2.1.5 SRO IR: 3.9 2

X X

X X

E Conduct of Operations, 40DP-9OP02, section 4.8.1.1 states that the additional reactor operator (who happens to be on Unit 1) may be filling an Auxiliary Operator position. Could he fill the fire team advisor position and the outside area watch? It seems that there are enough of the correct type of operators to cover all of the required positions on Unit

1. This brings into question whether or not the performance standard is actually correct.

Do you plan on giving the handouts as submitted? If so, this seems to be a very low level of difficulty and nearly a direct lookup. This and the references given on the cue sheet would, at least, cue applicants to all information needed to correctly complete the task.

Should an ENS Communicator be designated?

As written, this task provides very little discriminatory value.

8/30/2018: Licensee modified the initiating cue and JPM to address all of the comments above. The licensee will not be giving out the reference handouts and is requiring the applicant to determine and look up their own references. The edited JPM is appropriately discriminating and provides a reasonable level of difficulty. The JPM is now Satisfactory.

Manually calculate RCS leakage and evaluate Technical Specifications Conduct of Operations G 2.1.20 SRO IR: 4.6 2

S

ES-301 Operating Test Review Worksheet Form ES-301-7 Determine impacts of failed Startup Channel NI to refueling operations and Technical Specifications Equipment Control G 2.2.40 SRO IR: 4.7 2

S Evaluate the Release Flowchart and PAR flowchart following a SGTR/LOOP resulting in an Alert classification Radiation Control G 2.3.14 SRO IR: 3.8 1

U This does not appear to be a Radiation Control JPM. It is definitely an Emergency Plan JPM. See the descriptions listed in ES-301 of NUREG-1021 (pages 2 and 3 of 33). This JPM was rated as Unsatisfactory due to not meeting the intended Administrative Topic area.

How could this not be a time critical JPM?

If an applicant took twice the validation time it would exceed the 15 minute reporting requirement.

This level of difficulty is also very low based on the indications given and the information provided.

8/30/2018: Licensee replaced the JPM.

The new K/A is 2.3.15, SRO IR is 3.1.

The JPM requires the use and interpretation of radiation monitor readings to determine plant shutdown requirements following a S/G tube leak.

The level of difficulty of the new JPM is appropriate and rated as a 2.

The new JPM is Satisfactory.

Classify a loss of power event concurrent with an EDG with a loss of support system Emergency Plan G 2.4.40 SRO IR: 4.5 3

X E

The task standard states that the event must be classified within 14 minutes, but the coversheet states that it is not time critical. The initiating cue also states that it is time critical.

8/30/2018: Licensee corrected the coversheet.

The JPM is now Satisfactory.

ES-301 Operating Test Review Worksheet Form ES-301-7 Simulator/In Plant JPMs Safety Function and K/A LOD (1-5)

Attributes 4

Job Content U/E/S Explanation I/C Focus Cues Critical Steps Scope (N/B)

Overlap Perf.

Std.

Key Minutia Job Link Simulator:

Restore CEA overlap following a RPCB and trip the reactor following two dropped CEAs Safety Function 1 K/A: 001 A2.11 IR: 4.4/4.7 3

S Simulator:

Verify CIAS actuation Safety Function 5 K/A: 103 A3.01 IR: 3.9/4.2 3

S Simulator:

Place Boron Dilution Alarm System (BDAS) in service Safety Function 7 K/A: 015 A4.02 IR: 3.9/3.9 3

S In-Plant:

Local Operation of Turbine Driven AFW Pump AFA-P01 Safety Function 4S K/A: 061 A2.04 IR: 3.4/3.8 3

S In-Plant:

Align Train B EW to SFP Cooling Safety Function 8 K/A: 033 A2.02 IR: 2.7/3.0 3

S

ES-301 4

Form ES-301-7 Instructions for Completing This Table:

Check or mark any item(s) requiring a comment and explain the issue in the space provided using the guide below.

1.

Check each JPM for appropriate administrative topic requirements (COO, EC, Rad, and EP) or safety function requirements and corresponding K/A. Mark in column 1.

(ES-301, D.3 and D.4)

2.

Determine the level of difficulty (LOD) using an established 1-5 rating scale. Levels 1 and 5 represent an inappropriate (low or high) discriminatory level for the license that is being tested. Mark in column 2 (Appendix D, C.1.f)

3.

In column 3, Attributes, check the appropriate box when an attribute is not met:

The initial conditions and/or initiating cue is clear to ensure the operator understands the task and how to begin. (Appendix C, B.4)

The JPM contains appropriate cues that clearly indicate when they should be provided to the examinee. Cues are objective and not leading. (Appendix C, D.1)

All critical steps (elements) are properly identified.

The scope of the task is not too narrow (N) or too broad (B).

Excessive overlap does not occur with other parts of the operating test or written examination. (ES-301, D.1.a, and ES-301, D.2.a)

The task performance standard clearly describes the expected outcome (i.e., end state). Each performance step identifies a standard for successful completion of the step.

A valid marked up key was provided (e.g., graph interpretation, initialed steps for handouts).

4.

For column 4, Job Content, check the appropriate box if the job content flaw does not meet the following elements:

Topics are linked to the job content (e.g., not a disguised task, task required in real job).

The JPM has meaningful performance requirements that will provide a legitimate basis for evaluating the applicant's understanding and ability to safely operate the plant. (ES-301, D.2.c)

5.

Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 5.

6.

In column 6, provide a brief description of any (U)nacceptable or (E)nhancement rating from column 5.

Save initial review comments and detail subsequent comment resolution so that each exam-bound JPM is marked by a (S)atisfactory resolution on this form.

ES-301 5

Form ES-301-7 Facility: Palo Verde (2018-302 JPM Retake Exam) Scenario: N/A Exam Date: September 26, 2018 Event Realism/Cred.

Required Actions Verifiable actions LOD TS CTs Scen.

Overlap U/E/S Explanation N/A N/A - JPM Retake exam only.

ES-301 6

Form ES-301-7 Instructions for Completing This Table:

Use this table for each scenario for evaluation.

2 Check this box if the events are not related (e.g., seismic event followed by a pipe rupture) OR if the events do not obey the laws of physics and thermodynamics.

3, 4 In columns 3 and 4, check the box if there is no verifiable or required action, as applicable. Examples of required actions are as follows: (ES-301, D.5f) opening, closing, and throttling valves starting and stopping equipment raising and lowering level, flow, and pressure making decisions and giving directions acknowledging or verifying key alarms and automatic actions (Uncomplicated events that require no operator action beyond this should not be included on the operating test unless they are necessary to set the stage for subsequent events. (Appendix D, B.3).)

5 Check this box if the level of difficulty is not appropriate.

6 Check this box if the event has a TS.

7 Check this box if the event has a critical task (CT). If the same CT covers more than one event, check the event where the CT started only.

8 Check this box if the event overlaps with another event on any of the last two NRC examinations. (Appendix D, C.1.f) 9 Based on the reviewers judgment, is the event as written (U)nacceptable (requiring repair or replacement), in need of (E)nhancement, or (S)atisfactory? Mark the answer in column 9.

10 Record any explanations of the events here.

In the shaded boxes, sum the number of check marks in each column.

In column 1, sum the number of events.

In columns 2-4, record the total number of check marks for each column.

In column 5, based on the reviewer's judgement, place a checkmark only if the scenario's LOD is not appropriate.

In column 6, TS are required to be 2 for each scenario. (ES-301, D.5.d)

In column 7, preidentified CTs should be 2 for each scenario. (Appendix D; ES-301, D.5.d; ES-301-4)

In column 8, record the number of events not used on the two previous NRC initial licensing exams. A scenario is considered unsatisfactory if there is < 2 new events. (ES-301, D.5.b; Appendix D, C.1.f)

In column 9, record whether the scenario as written (U)nacceptable, in need of (E)nhancement, or (S)atisfactory from column 11 of the simulator scenario table.

ES-301 7

Form ES-301-7 Facility: Palo Verde (2018-302 JPM Retake Exam) Exam Date: September 26, 2018 Scenario 1

2 3

4 5

6 7

8 11 Event Totals Events Unsat.

TS Total TS Unsat.

CT Total CT Unsat.

% Unsat.

Scenario Elements U/E/S Explanation N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A - JPM Retake exam only.

Instructions for Completing This Table:

Check or mark any item(s) requiring comment and explain the issue in the space provided.

1, 3, 5 For each simulator scenario, enter the total number of events (column 1), TS entries/actions (column 3), and CTs (column 5).

This number should match the respective scenario from the event-based scenario tables (the sum from columns 1, 6, and 7, respectively).

2, 4, 6 For each simulator scenario, evaluate each event, TS, and CT as (S)atisfactory, (E)nhance, or (U)nsatisfactory based on the following criteria:

a.

Events. Each event is described on a Form ES-D-2, including all switch manipulations, pertinent alarms, and verifiable actions. Event actions are balanced between at-the-controls and balance-of-plant applicants during the scenario. All event-related attributes on Form ES-301-4 are met. Enter the total number of unsatisfactory events in column 2.

b.

TS. A scenario includes at least two TS entries/actions across at least two different events. TS entries and actions are detailed on Form ES-D-2. Enter the total number of unsatisfactory TS entries/actions in column 4. (ES-301, D.5d)

c.

CT. Check that a scenario includes at least two preidentified CTs. This criterion is a target quantitative attribute, not an absolute minimum requirement. Check that each CT is explicitly bounded on Form ES-D-2 with measurable performance standards (see Appendix D). Enter the total number of unsatisfactory CTs in column 6.

7 In column 7, calculate the percentage of unsatisfactory scenario elements:

8 If the value in column 7 is > 20%, mark the scenario as (U)nsatisfactory in column 8. If column 7 is 20%, annotate with (E)nhancement or (S)atisfactory.

9 In column 9, explain each unsatisfactory event, TS, and CT. Editorial comments can also be added here.

Save initial review comments and detail subsequent comment resolution so that each exam-bound scenario is marked by a (S)atisfactory resolution on this form.

2 + 4 + 6 1 + 3 + 5100%

ES-301 8

Form ES-301-7 Site name: Palo Verde (2018-302 JPM Retake Exam) Exam Date: September 26, 2018 OPERATING TEST TOTALS Total Total Unsat.

Total Total Unsat.

Explanation Edits Sat.

Admin.

JPMs 5

1 2

2 20 One Administrative JPM rated as Unsatisfactory because JPM topic did not appear to meet the intended Administrative topic.

Sim./In-Plant JPMs 5

0 0

5 0

Scenarios N/A N/A N/A N/A N/A N/A - JPM Retake exam only.

Op. Test Totals:

10 1

2 7

10 Instructions for Completing This Table:

Update data for this table from quality reviews and totals in the previous tables and then calculate the percentage of total items that are unsatisfactory and give an explanation in the space provided.

1.

Enter the total number of items submitted for the operating test in the Total column. For example, if nine administrative JPMs were submitted, enter 9 in the Total items column for administrative JPMs.

For scenarios, enter the total number of simulator scenarios.

2.

Enter the total number of (U)nsatisfactory JPMs and scenarios from the two JPMs column 5 and simulator scenarios column 8 in the previous tables. Provide an explanation in the space provided.

3.

Enter totals for (E)nhancements needed and (S)atisfactory JPMs and scenarios from the previous tables. This task is for tracking only.

4.

Total each column and enter the amounts in the Op. Test Totals row.

5.

Calculate the percentage of the operating test that is (U)nsatisfactory (Op. Test Total Unsat.)/(Op. Test Total) and place this value in the bolded % Unsat. cell.

Refer to ES-501, E.3.a, to rate the overall operating test as follows:

• satisfactory, if the Op. Test Total % Unsat. is 20%

• unsatisfactory, if Op. Test Total % Unsat. is > 20%

6.

Update this table and the tables above with post-exam changes if the as-administered operating test required content changes, including the following:

• The JPM performance standards were incorrect.

• The administrative JPM tasks/keys were incorrect.

• CTs were incorrect in the scenarios (not including postscenario critical tasks defined in Appendix D).

• The EOP strategy was incorrect in a scenario(s).

• TS entries/actions were determined to be incorrect in a scenario(s).