ML17066A184

Response to NRC Requests for Additional Information (Rais) Regarding Emergency License Amendment Request (LAR) to Extend Diesel Generator 3B Completion Time

ML17066A184
Person / Time
Site:	Palo Verde
Issue date:	01/02/2017
From:	Andrews G Arizona Public Service Co
To:	Document Control Desk, Office of Nuclear Reactor Regulation
References
102-07412-MLL/TNW
Download: ML17066A184 (29)

Text

10 CFR 50.90 MARIA L LACAL Senior Vice President, Nudear Oaps. Regulatory & Oversight

• Palo Verde Nudear Generating Station 102-07412-MLL/TNW P.O. Box 52034 Phoenix, AZ 85072 January 2, 2017 Mall Station 7605 Tel 623.393.6491 U. S. Nuclear Regulatory Commission ATIN:. Document.Control Desk Washington, DC 20555-0001

Reference:

Arizona Public Service Comp~ny Letter 102-07411, Emergency Ucense Amendment Request tq Extend Diesel Generator 38 Completion Time, dated December 30, 2016 [Agency Documents Access and Management System (ADAMS) Accession No. ML16365A240]

Subject:

Palo Verde Nuclear Generating Station (PVNGS)

Unit 3 . *, , ,

Docket No. STN 50~530 Renewed Operating License No. NPF-74

• Response to NRC Requests for Additional information (RAis) Regarding Emergency License Amendment Request (LAR) to Extend Diesel Generator 3B Completion Time By the referenced letter Arl.zon_a Public S~rvlce Company (APS) submitted a risk-Informed license amendment request (LAR) to extend the Palo Verde Nudear Generating Station (PVNGS) Unit 3, Technical Specification (TS) required action 3.8.1.B.4 completion time from 21-days to 62-days. The LAR Is for the duration of the repair and testing to re-establish operablllty of the Unit 3 train 'B' emergency diesel generator (3B DG) .that experienced a failure on December 15, 2016. Bye-malls dated .December 31, 2016, and January 1, 2017, ..

the NRC staff provided requests for addltlonal Information (RAls) to support their review of the LAR. The enclosure to this letter provides the APS response to the. RAls.

No new commitments are being made by this letter. The compensatory measures and commitments provided by the referenced letter wlll remain In effect for the duration of the

• extended completion time.

By copy of this letter, this RAI response Is being forwarded to the Arizona Radiation Regulatory Agency In accordance with 10 CFR 50.91(b)(1).

APS requests approval of the LAR on an emergency basis prior to the expiration of.the current 21-day completion time, which expires at 3:56 am on January-5, 2017. APS wlll Implement the TS amendment Immediately following NRC approval. Absent approval, PVNGS Unit 3 would be required to begin shutdown, pursuant to TS 3.8.1, Condition H.

A member of the STARS Alliance LLC Callaway ;* Diablo-C:anyon

• Palo Verde

• wolf creek

102-07412-MLL/TNW ATIN: Document Control Desk U. S. Nuclear Regulatory Commission Response to RAis Regarding LAR to Extend Diesel Generator 3B Completlo.n Time Pc:tge 2 Should you have any questions concerning the content of this letter, please contact Thomas

• Weber, Department Leader, Nuclear Regulatory Affairs, at (623) 393-5764.

I declare under penalty of perjury that the foregoing Is true and correct.

Executed on : Januarv 2. 2017 (Date)

Sincerely, Andrews, -' Digitally signed by Andrews, George W(Z99748)

ON: cn='Andrews, George W(Z99748)

George R~~n~ am approving _this aocumept for Maria Lacal W(Z9.9748) Date: 2017.01.02 12:37:07 -07'00' MLL/TNW/CJS

Enclosure:

Response to NRC Requests for Additional Information Regarding Emergency License

~mend~ent Request to Extend Diesel Generator 3B Completion Time*

cc: K. M. Kennedy. NRC Region IV Regional Administrator S. P. Llngam . NRC NRR Project Manager for PVNGS*

M. M. Watford NRC NRR Projec:t Manager* * *

• C.A. Peabody NRC Senior Resident Inspector for PVNGS T. Morales Arizona Radiation R~gulatory Agency (ARRA) .

f.

Response to NRC Requests for Additional Information Regarding Emergency *License 'Amendment Request to

. _Extend Di~sel Generator_ 38_ Corripletion*Time

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time

Background

By Reference 1, Arizona Public Service Company (APS) submitted a risk-informed license amendment request (LAR) to extend the Palo Verde Nuclear Generating Station (PVNGS)

Technical Specification (TS) required action 3.8.1.B.4 completion time from 21-days to 62-days. The LAR is for the duration of the repair and testing to re-establish operability of the Unit 3 train 'B' emergency diesel generator (3B DG) that experienced a failure on December 15, 2016. By e-mails dated December 31, 2016, and January 1, 2017, the NRC staff provided requests for additional information (RAls) to support their review of the LAR.

The context for the emergency LAR and this RAI response is provided by the following.

background information. By letter number 102-07406, dated December 21, 2016 [Agency Documents Access and Management System (ADAMS) Accession Number ML16356A689],

and supplemented by letter number 102-07410 (ADAMS Accession Number ML16356A715),

dated December 23, 2016, APS submitted a deterministic LAR to extend the TS required

• action 3.8.1.B.4 completion time from 10-days to 21-days for the purpose of collecting and

• analyzing data associated with the 3B DG engine failure and continue repair of the 3B DG.

The NRC staff issued license amendment number 199 for Unit 3 by letter dated .December 23, 2016 (ADAMS Accession Number ML16358A676). As part of the LAR, APS indicated. that after analysis of causal information and if there was a determination that there* is no common mode failure potential for the Unit 3 train 'A' emergency diesel generator (3A DG),

a risk-informed LAR would be submitted for the duration of the repair and testing of the 3B DG.

Disassembly and irispedion of the damaged 3B DG has been aggressively a*nd continuously pursued since initial failure on December 15, 2016. APS established an outage control center (OCC) to schedule, man.age.and oversee the work activities needed for the repairs.

Multi-discipl"ine teams were formed to assess the extent of damage, inspect and recover parts, and determine the cause of failure. APS has determined that the cause of failure of

'.the 3B DG is attributed to high *cycle fatigue and that the mode of failure is not common to the 3A DG or the emergency diesel generators in Units 1 and 2.

• Therefore, in accordance with the provisions of Section* 50. 90 of Title 10 of the Code of Federal Regulations (iO CFR), APS submitted an emergency risk-informed LAR for an extension of the completion time described in TS 3.8.LB.4 for the PVNGS 3B DG (Reference 1). Specifically, the emergency risk-informed LAR would extend, on a one-time basis, the TS required action 3.8.1.B.4 completion time from 21-days to 62-days for the purpose of completing repairs and testing to. re-establish operability of the 3B DG.

Each NRC staff request is presented in this enclosure, followed by the APS response. The RAls and related responses are organized by NRC staff branch.

1

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Probabilistic Risk Assessment Licensing Branch (APLA)

APLA RAl-1:

The license amendment request (LAR) for Palo Verde Nuclear Generating Station (PVNGS),

Unit 3, dated December 30, 2016, states that the plant-specific risk assessment of the proposed change to.the Technica*I Specification (TS) completion time (CT) follows the guidance in Regulatory Guide (RG) 1.177, Revision 1. Section 2.3.6, "Use of Compensatory Measures in Technical Specification Change Evaluations," of RG 1.177 states:

"When compensatory measures are part of the TS change evaluation, the risk impact of these measures should be considered and presented, either quantitatively or qualitatively. When a quantitative evaluation is used, the total impact of these measures should be evaluated by comparison to the "small" guideline (Principle 4, as describ_ed in Part B of this regulatory guide). This includes (1) evaluation of .the proposed TS changes without the compensatory measures, (2) evaluation of the ,

proposed TS changes with the compensatory measures, and (3) specific discussion of how each of the compensatory measures is credited in the PRA_model or during the evaluation process."

In addition, according to Frequently Asked Question (FAQ) 12-0064, "Hot Work/Transient Fire Frequency Influence Factors," dated January 17, 2013 (ADAMS Accession No~

ML12346A488), transient influence factor may not be assigned a ranking value of O, unless associated activities and/or entrance during power operation are precluded by design and/or

• operation. Admin.istrative controls significantly impact the characteristics and likelihood of transient fires, but they do not preclu_de their'.occurrence, sir:ice there is 'industry evidence of failure to follow administrative control procedures. Design factors that would preclude transient fires would be features that physically prevent access to. a location even. if those features might be overcome* by extraordina*ry actions (e.g., by breaching .a physical barrier or through use of a cutting torch). Operational factors that would preclude transients would be features that make a location essentially.uninhabitable by personnel. '

Section 4.3.2, "Tier 2: Avoidance of Risk Significant Plant.Configurations," of the LAR dated Decemb~r 30, 2016 states the following:. * -. *

~ .,

"In *addition, the PRA analys.is cred'its the following actions to further reduce Ji re PRA risk as documented in Engineering Evaluation 16-15545-023 (Reference 14). These aqditional actions have been added as commitments in the list of compensatory measures being taken during this extended LAR period (See Attachment 3 to this enclosure): * *

• Additional dedicated auxiliary operator added to each shift to implement the modification that cross-ties fire water to the train 'N' auxiliary feedwater sy?tem.* .

..- - * * * ..~. , >

• . Posting a continuous fire watch with a fire extinguisher and training. to utilize the

• .. extinguisher in fire zone FCCOR2 '(120' Corridor Building) .. This action* improves

• detection and response timing assumed in the fire PRA for t_his area sihce it contains no detectors or suppression systems. This area contains numerous ..

power and control cables including offsite power supply, reactor coolant pump

.control, and nuclear cooling w,ater control. .

• 2

\,

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 38 Completion Time

• Establish transient combustible and hot work exclusion zones by procedure and using barriers/signage in the following compartments, and conducting shiftly walkdowns of these zones by the Fire Marshal or his designee. These areas were selected based on their high contribution to core damage frequency from transient combustible fires.

- Fire zones FCCOR2 (120' Corridor Building) and FCCOR2A (120' Corridor Riser Shaft)

- Fire zones FCTB04 (upper level only, non-class DC Equipment, [FCTB04-TRAN1]) . * *

- Fire zone FC86A (train 'A' Seismic Gap, make part of train 'A' Electrical Protected Equipment)

- Fire zone FCTB100 zone ZTlG (SW corner, south half of 100' Turbine between columns TA and TC)"

It is-not clear how these compensatory measures were credited in the fire PRA -to further reduce fire risk. Explain and justify how the fire PRA model (e.g., fire ignition frequencies, fault tree and eventtree logic,-failure data, human error probabilities,*depend~ncies) was adjusted to credit these compensatory measures. In addition, provide the following:

• Explain the role of the additional dedicated auxiliary operator such that the human error probability for aligning firewater to the steam generator reduces from 2.93E-02 (on page 34 of Attachment 16 to the LAR; with a time delay of 41 m,inutes) to 3.32E-3 _(on page 38 of Attachment 16 to the LAR, with a tir:ne delay of 18 minutes).

• *Discuss how the human failure event dependency( es) between depressurizing the steam generator and aligning firewater to the steam generator were addressed in the PRA. - -

• • Provide the quantitative impact of these compensatory measures on fire ICCDPs and ICLERPs for this *one-time TS *CT extension assuming no credit for the portable diesel*

generators and assuming credit for the portable diesel generators.

APS Response to APLA RAI-*1*:

The following describes how the fire PRA model (e.g., fire ignition frequencies, fault tree arid event tree logic, failure data, human error probabilities, dependencies) was adjusted to

• credit these compensatory measures. * * *

• Compensatory Measure:* Additional Dedicated Auxiliary Operator, Operations Night Order, and Shiftly Briefing to Implement Firewater to Auxiliary FE;!edwater Cross-connect:

Each Unit 3 shift is briefed on an operations night order that describes actions for a dedicated auxiliary operator to align firewater to the train 'N' auxiliary feedwater pump discharge per emergency.operating procedure 40EP-9E010, Standard Appendix 118, when auxiliary feedwater initially fails, instead of waiting for efforts to recover the normal sources of feedwater. This allows the dedicated auxiliary operator to work in parallel with the actions being taken to attempt local recovery-of the steam-driven au_xiliary feedwater pump, the train 'N' auxiliary feedwater pump and other normal sources of feedwater. The dedicated operator provides assurance of the timeliness to align firewater to a steam generator and ensures that actions to restore normal feedwater sources proceed in parallel. A simulator scenario time line for a fire induced loss of offsite power and loss of all feedwater sources showed the dedicated auxiliary operator was dispatched at 15 minutes to implement 3

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Standard Appendix 118. when crediting the dedicated auxiliary operator, night order and shiftly briefing. The human reliability analysis (HRA) for this action assumed a Tdelay of 18 minutes. The base case human reliability analysis for this action (without the dedicated auxiliary operator, night order and shiftly briefing) assumed a Tdelay of 41 minutes to

• account for attempts to restore auxiliary feedwater pump*s, main feedwater, and low pressure feed from the condensate pumps before entering Standard Appendix 118 to utilize the firewater to auxiliary feedwater cross-con.nect. The increased margin afforded by the dedicated auxiliary operator, night order, and shiftly briefing reduces the dependency assigned to recovery actions, since there is more time to. identify an error and take action within the required. time before core damage.

The steps necessary to align firewater to the.trainlN' auxiliary feedwater pump discharge, depressurize a steam generator, and align flow to a steam generator are all in Standard Appendix 118.- All the subject steps were included in the human reliability analysis. Success can be validated by :feedwater flow indication and by .!?team generator level indication in the control *room. Steps for monitoring these parameters are at the end of Appendix 118 as well as in emergency operating procedure 40EP-9E006~ Loss of Feedwater. These steps allow for recovery of potential failures of the ,auxiliary operator actions by the control room operator,

• and of the control room operator actions by the control room supervisor. Based on the time margin with the dedicated auxiliary operator, night order; aQd shiftly bri_efing compensatory.

measures, these recoveries were assigned a low dependency. For the base cas~ without these comp_ensatory measures, th~se recoverie_s were assigned a high dependency.

In the HRA for the* operator actions to implement use of the firewater cross-.conQect to .

auxiliary feedwater, *the delay time from the cue to the .execution (Tdeiav) was reduced.~from 41 minutes to 18 minutes based.. on the additional dedicated auxiliary operator, night order,.

and shiftly br:iefing. Based on review ofthe timing associated with other steam generator *

• heat removal (SGHR) recovery human failure events (HFEs), no changes wer.e made to ~tie existing dependency analysis for combination events (i.e., use of the firewater cross-connect is still credited as a .contingency after failure to recover auxiliary feedwater) .

. The fire-adjusted HFE probabilities were recalculated to account for fire induced impacts (e.g., stress, location accessibility, instrumentation cues, etc.). The human error probabilities (HEPs) were updated if") the fault tree database, and the HEP combination event values were recalculated from the dependency analysis. No .revisions were made to *

• application of the HFEs in the-fault trees or recovery rules.

Compensatory Measure: Posting-a Continuous Fire Watch with a Fire Extinguisher and Training to Utilize the Extinguisher in Fire Zone FCCOR2 (120' Corridor Building)'**

Posting a continuous fire watch with a fire extinguisher and training to utilize the extinguisher in fire tone FCCOR2 (i20' Corridor Building) provided for an .adjustment to the .

detection and suppression capabilities credited in this fire compartment. For all fire scenarios modeled in the Corridor Building 120' elevation (FCCOR2), with the exception*of any high energy arcing fault (HEAF) scenarios, prompt detection (PD) and prompt suppression (PS) were considered* available in the calculation of the zone-of-influence and hot gas layer non-suppression probabilities.

The split-fraction probabilities for failure of prompt detection and failure of prompt suppression are applied per NUREG/CR-6850 Appendix P. The prompt suppression curve value for welding fires, as. provided in NUREG/CR-6850, Supplement 1, Chapter. 14, Table 4

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time 14-2 Original and Updated Mean Suppression Rates, is applied to the PS component of the calculation of non-suppression probabilities for scenarios crediting continuous fire watch.

Compensatory Measure: Establishing Transient Combustible and Hot Work Exclusion Zones in Designated Areas by Procedure and Using Barriers/Signage, and Conducting Shiftly Walkdowns of these Zones by the Fire Marshal or His Designee Establishing transient combustible and hot work exclusion zones in designated areas by procedure and using barriers/signage, and con.ducting shiftly walkdowns of these zones by the Fire Marshal or his designee provided for excluding the potential fo~ transient fires, transient fires due to welding and cutting (hot work), and cable fires due to welding and

. cutting in the designated areas. The* fire department reviewed all open transient combustible and hot work permits to identify. any impact to .these designated areas. .

Walkdown of the area*s by the fire department identified no transient combustible materials currently stored !n these locations. No hot work permits had been approved for these areas.

The transient fires (TRAN), transient fires due to welding and cutting (TFWC), and cable fires due to welding and cutting (CFWC) scenarios in the designated areas were excluded.

from the fire PRA model quantification. Scenario* exclusion is triggered by a setting in the*

quantification spreadsheet or the ignition source data table, but is equivalent to setting the scenario ignition frequencies to 0.0. Al.I other fire scenarios in these designated areas were unadjusted.

Requiring physical barriers or inaccessibility during plant operations as a basis for assigning ari ignition frequency influence factor of*O.O is reasonable for the baseline (ire PRA model given operating experience. However, temporarily assigning. a very small weighting factor (or negligible ignition frequency) is reasonable to* model the'realism *of a temporary plant configuration. Given the added oversight by plant management inthe outage control center (OCC) of all test and maintenance work activiti.es, increased scrutiny by the fire .ca,Ptairi reviewing transient combustible permits and hot work permits, and the additional shiftly walkdowns by the. Fire Marshal cif these areas, the inadvertent entry into the restricted areas, staging of transient combustibles, or establishing wor~ conditions* for hot work in these areas is considered negligible during the Unit 3 'B' diesel gene.rator*repair.

As a result of the area walkdowns by the fire department, no transient combustibles were identified or relocated to other areas. Thus the transient fire scenarios in other areas were not revised.

Quantative Impact of Not Crediting Compensatory Measures with and without Crediting Portable Diesel Generators

• The quantitative impact of the compensatory measures listed above on the fire ICCDP and ICLERP for this one-time TS completion time extension, assuming no credit for the portable diesel generators and assuming credit for the portable diesel generators, is found in the table below as documented in Reference 2.

5

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 38 Completion Time Case Fire Fire.

ICCDP IC LE RP No credit for portable diesel generators, no credit for 1.7E-5 4.3E-7 compensatory measures 1 No credit for portable diesel generators, credit for 8.9E-6 2.2E-7*

comoensatorv measures 1 Credit for portable diesel generators, no credit for 5.2E-6* 1.3E-7.

compensatory measures 1 .. ..

Credit for portable die*sel generators, *credit for ~ompensatory 1.8E-6 6.4E-8 measures 1 .,

Note:. .

1.* Compeilsatory*measures considered: (1) dedicated auxiliary operator to implement' * * **

• firewater to**auxiliary feedwater cross-connect, Night Order regarding priority of*

implementing firewater to a_uxiliary feedwater cross-connect, shiftly briefings on nig.ht order, (2) fire watch in 120' Corridor Building* and (3) four transient-combustible *and hot-work'e_xclusion zones. . . . .

~

'.,~

-6

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Prot>abilistic Risk Assessment Operations & Human Factors Branch (APHB)

APHB RAI-1:

Section 4.3.1 of the submittal states, in part: "A simulator exercise was performed to evaluate changes to emergency operating procedures involving this cross-connect and

• Confirm timing assumptions for the PRA human reliability analysis (see Attachment 16)."

Further, Attachment 16 to the submittal, Section titled "APS Response to TechniC:al Concern 10" states, in part: "From the Operator interviews, it would take 15 *minutes for the

'operators to complete the required actions per 40MT-9ZZ01 and an additional 5 minutes to start feeding the steam generators using AFN-POl Appendix 41 of 40EP-,9E010. Therefore execution time = 20 minutes (15 minutes + 5 minutes). Subsequently, 40MT-9ZZ01 has be.en revised to improve operator response. Based upon operator walk-th.roughs.with the revised procedur.es the maximum time recorded for.these*actions was 23 minutes.".

-. ~ . . - . .

Please provide apditional information clarifying the above statem~nts, as. follows: .

a. Identify which Emergency Operating Procedures (EOPs), besides 40MT-9ZZ01, if any, were revised in connection with the proposed amendment~ Summarize/

describe the changes made. *

b. Clarify whether changes to the EOPs were evaluated based on operator walk-throughs, simulator exercises, or a combination of the two. If possible, identify how many operator walk-throughs / simulator exercises were performed, utilizing different crews,.

c .. Clarify. why the*execution time (Texe) is determined to be *20 minutes, if the operator walk-throughs with the revised procedures identified 23 minutes as the amount of time *needed to perform the required actions. *

• APS Response to APHB RAI-la:

Reviews conduct~d by Operations and Training personnel of compensator'! measures established for the extended completion time for the 3B DG identified opportunities to improve the efficiency and clarity of procedure guidance. The compensatory measures reviewed included use of the portable DGs and use of the fire protection system (FP) to auxiliary _feedwater system (AF) cross-connect. As noted above* in the NRC question, procedure 40MT-9ZZ01 was revised and other EOPs were revised for this purpose. A

• description of. the changes to the EOPs is provided below:

40EP-9E006, i.oss of All Feedwater - Step 6 of the Loss of All Feedwater procedure provides instructions for restoring feedwater to at least one steam .

generator using various means. Operators are initially directed to recover auxiliary or main feedwater as the preferred actions. In the event these preferred actions are

. unsuccessful, the operations crew will transition to contingency step 6.1 (see revised step below) which provides direction for use of a low pressure feedwater source. If off-site power is available, the contingency step directs use of Standard Appendix 44 to directly feed the steam generators using the condensate pumps.

If off-site power is unavailable, the PVNGS emergency procedures contain guidance in the Functional Recovery Procedure, 40EP-E009, for use of Standard Appendix 118 to restore feedwater using the FP to AF cross-connection. To facilitate more timely restoration of feedwater when the FP to AF cross-connection is needed, step 6.1 of 40EP-9E006 was revised to direct use of Appendix 118 (also revised, see discussion 7

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time below) while implementing 40EP-9E006 without the need to implement the Functional Recovery Procedure to feed a steam generator with a Fire Protecti.on Pump.

6.1 Perform the following to establish a low pressure feedwater source:

a. IF ALL of the following:.

• Auxiliary or Main Feedwa,ter can NOT be restored Offsite power is available'

• Feeding a Steam Generator. *1.

with a condensate. pump is desired THEN PERFORM Appendix 44, Feeding with the Condensate Pumps.

b. IF feeding a Steam. Generator with a fire pump is desired,.

THEN PERFORM Appendix 118, Cross-connect FP to AF.

• 40EP-9E010, Standard Appendices - Standard Appendix 1is,-cross-c~nnect

• FP to AF, was extensively revised from the previous*guidan~e to._align the cross-connection flow path only (four manual valves) to provide comp.rehensiv'e guidance .

to align the. flow path. and provide the guidance needed to establish fe~dwater floVI{

.from the fire protection header:, including, steps fo align the teedw~ter systeni, select and deP.:ressurize a steam generator, establish makeup' to the RCS, start fire protection sY:stem pumps, and control ban.ds for steam. generator water l_evel and pressure. This change streamlin.es the actions*by bringing all ne<;:essary guidanc~ to establish feedwater from the FP header into ohe appendix and provides multiple

• attachments Jo the appendix allqwing different sectioQs of the appendix to be performed concurrently, where applicable. All steps required to lower steam generator pressure and manipulate the required syst~m~ components have been incorporated into the appendix *as Att'achrnents 118B - Feeding SGt With Fire Water and 118C - Feeding SG2 With Fire Water. This*revJsed guidan*ce employs a strategy similar to that of the existing Appendix 44, Feeding with the Condensate Pumps, which can be used to provide a source of low pressure feedwater. *

• 40EP-9E009, Functional Recovery Procedure, could also.be implemented during this event. Revision to this procedure was not required.as it already contains direction for use of Standard Appendix 118,*Cross-connect FP to AF. The previously discussed strategy for recovering feedwater flow during a loss of all f~edwater even,t would be identical if implemented via t~e Functional Recovery Procedure. * .,

APS Response to APHB RAl-lb:

Two simulator sessions were used to evaluate and validate draft changes to the EOPs. The first simulator session was 'staffed with shift operations personnel, the second simulator session was staffed .with shift operations personnel. and operations training personnel. Based 8 --

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time on the results of the simulator sessions, changes were made to the EOPs to provide additional enhancements to improve r~covery time. Then, using the revisions to the EOPs and the revised guidance of 40MT-9ZZ01, on-shift operations personnel have been briefed at the start of each shift. Operator feedback has been used to further enhance the strategies being used to implement the compensatory measures specified for the extended completion time. Examples of incorporated actions include changing the layout of Standard Appendix 118 for ease of use by the operators, designation of the station fire pumps as "Protected Equipment," and the use of Operations Department locking devices on locked Fire Protection valves instead of Fire Protection Department locks which could delay response.

Currently, 117 qualified operators have been briefed by Operations leadership and/or walked down the revised procedural guidance .based. .on their .. specific assignments.

APS Response to APHB RAI-lc:

The human reliability analysis (HRA) for use of the portable.diesel generators to power a Class lE 4160V AC bus was prepared in September 2016 to support crediting of this success path in the online 10 CFR 50.65 (a)(4) configuration risk management tool for configurations where the portable generators are deployed and connected to the FLEX connection box. The HRA timing analysis was based on 40MT-9ZZ01, Revision 2, in effect at the time of the analysis and input from Operations. It was assumed in the HRA that actions in the procedure to open load breakers on the bus prior *to energizing the bus were*

performed in the *38 minute Tdelay period when attempting to use the Station Blackout Gene.rators to supply the bus. *

• Operations.subsequently performed* additional walkthroughs of this procedure in December 2016 and made enhancements to 40MT-9ZZ01. The 23 *minute maximum observed implementation time for the_ enhanced 40MT-9ZZ01 was from operator.walkthroughs conducted in December 2oi6, induding use of two auxiliary operators to implement the procedure. The walkthroughs included one auxiliary operator opening load breakers to prepare the bus forJoading and this action took loriger than the other action of starting the generators by the seeond au~iliary operator. . .

Therefore, the HRA was conservative since( it accounted for opening the bus load breakers during the _38 minute Tdelay and did not account fpr use 9f two auxiliary operato'rs assigned to implement the enhanced version of this procedure. Nevertheless, substitution of the new 23 minute Texe for the existing assumed 20 minute Texe in the HJ:lA does not change any

• dependencies or the total human error probability (HEP) calculated for this HRA in the EPRI HRA Calculator.

• APHB RAI-2:

Section 4.3.l of the submittal states, in part: "Use of this cross-connect is p'roceduralized in the emergency operating procedures and is further described

. in a Unit 3 Operations. Night Order to emphasize the importance of timely use of this success path if necessary to prevent core damage."

• Further, Attachment 16 to the submittal, Section titled "APS Response to Technical Concern 10" states, in part: "However, to credit the 4160 VAC portable generators during LOOP ora Station Blackout, a night order or shiftly Control Room briefing is required with guidance to*

9

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time start the temporary 4160 VAC portable generators using 40MT-9ZZ01 if the emergency DG is unavailable and the SBOGs fail, due to simultaneously being in the EOPs."

Clarify if a Night Order or shift Control Room briefing is required and is being relied upon by

• APS, in order to successfully execute procedure 40MT-9ZZ01, within the required time frame (Texe). Further, explain why APS considers reliance on Night Orders or shift Control Room briefings to be Sl1fficient for the proposed 62-day duration of the TS required action 3.8.1.B.4 completion time.

APS Response to APHB RAI-2:

The Operations Night Order process provides a platform to distribute important information to the operating staff on a shiftly basis. These Night Orders are part of the formal shift .

turnover process administered per the guidance of procedure 40DP-90P33, Shift Turnover,

• and are required to be reviewed by the control room staff each shift. In this case, a Night Order was selected as one of the methods to communicate important background information to on-shift operations personnel related to use of compensatory measures during the extended TS completion time for the 38 DG being out of service.

Based on the nature of the operator actions needed in the event of a loss of feedwater or loss of all power, the strategy of discussing, reviewing and walking down these important actions every shift was chosen. This implementation of 'continual focus' was selected over a one-time review as a means to provide additional rigor and maintain the operator's heightened state of readiness to promptly initiate his or her compensatory actions throughout the extended out-of-service time.

• Prior to an operator assuming the dedicated FLEX diesel generator watch, he or she must complete a physical walkdown with the shift manager (or shift manager designee). This walkdown is performed in the field with the operating instructions in hand to ensure the operator is familiar with the location and operation of all required equipment. Other points of discussion include self-verification techniques, guidance on pre-,start checks that are .

performed throughout the shift and the use of electrical personal protection equipment (PPE). As an on-going measure to ensure the operators are prepared to initiate assigned compensatory actions without delay, the following actions are performed at the beginning of each shift: *

• Dedicated operators are assigned to perform the required compensatory actions. The names of the dedicated operators are annotated in the control room logs to ensure clarity of these assignments. * * *

• The shift manager discusses the content of the Night Order and required compensatory actions with the dedicated operators and the other members of the operations crew.

• Each dedicated operator reviews the specific guidance document for the assigned compensatory actions to ensure no changes have been made and to ensure adequate understanding of actions .

.

• Each dedicated operator walks down his or her assigned equipment to ensure they are prepared to implement the compensatory actions and to verify the equipment configuration will support the compensatory measure through the u*se of pre-,start checks for these generators.

• This combination of actions to establish and maintain operator knowledge and skill on each shift provides assurance that the operations crew is adequately staffed with proficient operators to support timely implementation of the compensatory measures. This approach 10

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time was chosen as the most effective method to sustain a high level of operator awareness and proficiency and are considered to be sufficient for the duration of the 62-day period that the extended completion time is expected to be in effect.

APHB RAI-3:

Section 4.5 of the submittal states, in part: "Training, briefings, and walkdowns are provided to the Operators responsible for operating the portable DGs as part of the preparation for use of the generators. Operating crews are briefed on the implementing procedure. Designated operators are familiar with instructions for starting and operating ttie portable DGs. Operations staff has received classroom training for FLEX strategies, whi!=h included the use of the portable DGs. Similar training is provided for the fire water to auxiliary feed water cross-tie."

Please provide additional information clarifying the above* statements, as follows:

a. Clarify if sufficient number of operators have been trained on the use of portable DGs.

b. If, so, please verify that the sufficient number of trained operators will be available during all shifts.

c. Clarify if the training received by the Operators was limited to the use of Flex 1 and Flex 2 DGs only, or if also included the use of the Rental DG, as shown in Figure 4, Section 4.1 of the submittal.

• APS Response to*APHB RAI-3:

a ..

• As of Janu~ry 1, 2017,-47 auxiliary operators have received the initial briefing and focused walkdown with a shift manager (or the shift manager designee).and are proficient to perform the responsibilities of the dedicated portable DG operator position. Records of the auxiliary operators who are proficient are maintained in the*

• Unit 3 Shift Manager's office. Further, as other operations crews rotate on-shift per the five week operations crew rotation schedule, additional operators will become proficient when they complete the briefings and focused walkdowns. The 47 operators who are currently proficient, are considered sufficient to fully staff the dedicated operator position throughout the extended completion time.

b. Dedicated operators are scheduled to cover each shift to fill this fundion. This additional staffing is over and above the minimum required operations shift staffing.

The process to administer operations on-shift staffing is an on-going process with reviews of shift staffing conducted each day. In addition, each shift, the Outage Control Center *oversees and monitors the implementation of the compensatory measures for this LAR including verification of assi'gnment of the dedicated operator positions in Unit 3.

c. The original training performed for *operation of the FLEX 1 *and 2 DGs did not consider the use of a third rental DG. However, the briefings and focused walkdowns performed to ensure auxiliary operator proficiency with the configuration in use for this LAR include the FLEX 1 and 2 DGs and the rental DG. The rental DG is similar to the FLEX 1 and 2 DGs with similar controls and hard card instructions. All three machines use an automatic synchronization feature. The actions required to start the diesel engine and close the output breaker on each of the three machines are the same. Operations with the three DGs in parallel is covered as part of the initial and 11

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time continuing briefings and procedure/equipment walkdowns performed by auxiliary operators with the shift manager (or shift manager designee).

APHB RAI-4: 6 of the submittal, Section titled "APS Response to Technical Concern 10" indicates that the Job Performance Measures (JPMs) were not selected for any of the sequences described in that section (e.g., 1FLEX-4160V--FAIL, Operators Fail to Align the 4160 VAC Portable Generators and lAF-FLEX-SGHR-HL, Operator Fails to Align FP to AF cross- connect to feed SG).

Provide additional information explaining why APS did not establish JPMs for those activities, and how does APS intend to ensure adequate operator training.

APS Response to APHB RAl-4:

Based on the nature of the operator actions needed in the event of a loss of feedwater or loss of all power, the strategy of discussing, reviewing and walking down these important actions every shift was chosen. This implementation of 'continual focus' was selected over a one-time training activity such as a JPM to provide additional rigor and maintain a heightened state of readiness to promptly initiate assigned compensatory actions throughout the extended out-of-service time.

Prior to an operator being assigned to one of the dedicated operator positions, he or she must complete a focused walkdown of the assigned equipment with the shift manager (or the shift manager designee). This walkdown is performed in the field with the operating instructions in hand to ensure the operator is familiar with the location and operation of all required equipment. Other points of discussion include self-verification techniques, industrial safety considerations, and guidance on DG pre-start checks that are performed each shift as appropriate. As an on-going measure to ensure the operators are prepared to initiate assigned compensatory actions without delay, the following actions are performed at the beginning of each shift:

• Dedicated operators are assigned to perform the required compensatory actions. The names of the dedicated operators are annotated in the control room logs to ensure clarity of these assignments.

• The shift manager discusses the content of the Night Order and required compensatory actions with the dedicated operators and the other members of the operations crew.

• Each dedicated operator reviews the specific guidance document for the assigned compensatory actions to ensure awareness of any revisions that have been made and to ensure adequate understanding of actions.

• Each dedicated operator walks down his or her assigned equipment to ensure they are prepared to implement the compensatory actions and to verify the equipment configuration will support the compensatory measure through the use of pre-start checks for these generators.

The combination of actions to establish and maintain operator knowledge and skill on each shift provides assurance that the operations crew is adequately staffed with proficient operators to support timely implementation of the compensatory measures. This approach was chosen as the most effective method to sustain a high level of operator awareness and proficiency over the 62-day period that the extended completion time is expected to be in effect.

12

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Component Performance NOE and Testing Branch (EPNB)

EPNB RAI-1:

Section 2.2, paragraph 1 - It is stated that the number 3 bearing seating surface was cracked. How does this relate to the failure of 9R?

APS Response to EPNB RAI-1:

The cracks identified on the number 3 crankshaft main bearing seating surface (seating surface), which is located on the engine center frame and functions with a main bearing cap to hold the number 3 main bearing halves in place, did not cause the number 9 master connecting rod to fail. As shown in Figure 1, two indications were noted during non-destructive examination (NDE) of the number 3 seating surface. The primary indication was an oval indication (approximately 3" x 5") with a secondary linear indication (highlighted with a red circle in Figure 1). The linear ind ication emanated from the oval indication.

13

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Oval Indication The oval indication was traced to an original manufacturing repair of a casting flaw, which was documented in Cooper-Bessemer Material Review Request 13021 and resolved through Salvage Procedure number 27, Bearing Bore Patch - Sand Inclusions. This repair was implemented prior to finish machining. Given the Original Equipment Manufacturer approval, this repair did not create any surface discontinuities on the number 3 crankshaft seating surface to adversely affect the alignment of the crankshaft bore. The oval crack indication was not a causal factor or a consequence of the number 9 master rod failure.

Linear Indication The linear indication is normal (perpendicular) to the oval indication repair boundary described above, and was not dispositioned in the manufacturer's Material Review Request. The morphology of this indication is visually consistent with over-load cracking in a cast material. The linear crack in the number 3 bearing seating surface initiated at the geometric stress riser of the manufacturing repair and resulted from the violent loading experienced by the engine from the failure on the number 9 connecting rod during the 2016 event. This linear crack is consistent with other post-failure consequential cracking in the cast block.

EPNB-1 Conclusion In summary, neither the oval nor the linear indications identified on the number 3 main bearing seating surface contributed to the number 9 master rod failure because these indications did not affect crankshaft alignment. The oval indication was from an original manufacturing repair of a casting flaw and the linear crack is consistent with other post-failure consequential cracking in the cast block.

EPNB RAI-2:

Section 3.2, last paragraph - It is stated that there is no common mode failure between diesels 3A and 3B. Is the licensee proposing that the connecting rods in diesel 3A are not subject to high cycle fat igue, or is the licensee proposing that the potential for high cycle fatigue exists in all diesel connecting rods but that the cause of the high cycle fatigue in diesel 3B is sufficiently well known (and absent in diesel 3A) such that the occurrence of this event does not increase the probability that a similar event will occur in diesel 3A?

APS Response to EPNB RAI-2:

The 3A DG Cooper-Bessemer KSV was designed and manufactured (10CFRSO, Appendix B, Quality Assurance Program) such that stresses seen in the rotating and reciprocating components are less than the material's endurance limit. As an element of the design, the engineering specification for the KSV engine includes a main-bearing horizontal and vertical bore alignment requirement to ensure the center frame-to-crankshaft stresses (impacting stresses experienced at the running gear) are acceptable. Given the above, the rods are not susceptible to high cycle fatigue provided design assumptions are maintained .

The cause of the high cycle fatigue in 3B DG is main bearing bore misalignment due to the previous failure in December 1986 that altered the engine's stress profile and that neither the misalignment nor the altered stress profile are present in 3A DG. As such, the 14

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time connecting rods in the 3A DG engine are not subject to the same stresses that both initiated the flaw and propagated the crack in the 3B DG engine. This analysis is further described in Attachment 4 of APS letter number 102-07411, dated December 30, 2016 (Reference 1).

The 3A DG has not experienced a catastrophic event and, therefore, does not have crankshaft misalignment. Without the misalignment the stress profile of the 3A DG is consistent with the original design.

EPNB-2 Conclusion The connecting rods in 3A DG are not subject to the stresses or the flaws which are precursors for high cycle fatigue. The PVNGS 3A DG has not experienced any master connecting rod related mechanical failures. Therefore, 3A DG alignment remains compliant to the original alignment design specification. With the proper crankshaft alignment the engine stresses remain below the endurance limit which precludes fatigue failure.

EPNB RAI-3:

Appendix 4 page 6 of 14, last paragraph (Palo Verde 2015 EDG 3B Event Findings) states:

"Nearly all of the rods had some minor fretting on the outboard edge of the backside of the bearing and transfer of material between surfaces." It is not clear whether this is referring to the interface between the main rod and the main journal bearing or the interface between the main rod and the articulated rod bushing. Additionally "inboard" and "outboard" are not clear in this context APS Response to EPNB RAI-3:

APS performed NDE (fluorescent fiquid penetrant testing, magnetic particle testing, and visual inspections) on all the 3B DG connecting rods. Fretting was identified via visual inspection on the master rod crank pin bore saddles and associated bearing caps (i.e.

backside of the bearings).

15

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Figure 2: Connecting Rod Diagram ARTICULATED-~

ROD

~~======:::=;~=---LOIL PASSAGES TO BUSHING MASTER ROD CRANK PIN BORE SADDLE BACKSIDE OF CRANK PIN BEARING -TOP SURFACE OF HALF TOP BEARING Cra n k Pin Journal ROD CAP As shown in Figure 2, APS is referring to the interface between the backside of the crank pin bearing - top half (bearing) and the master rod crank pin bore saddle (saddle). The visual inspections identified minor fretting on the outboard edge of both the saddle and bearing for nearly all connecting rod assemblies. These visual inspections also identified minor fretting on the inboard surface of the saddle and bearing on six of the nine 3B DG connecting rods. See Figure 3 for clarification of

" inboard " and "outboard " fretting locations.

16

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Visual inspection with a 200x magnification video camera identified micro cracks within the inboard fretting marks on number 2 master rod, number 4 master rod, and number 9 master rod near the oil groove in the ligament area. Visual inspection with the video camera did not identify micro cracks in any of the inspected outboard fretting marks.

Fretting identified on the outboard portions of the saddle and bearing are considered normal wear.

EPNB-3 Conclusion Figure 2 illustrates that the minor fretting interface is between the backside of the crank pin bearing - top half and the master rod crank pin bore saddle.

Figure 3 illustrates that the terms "inboard" and "outboard" refer to the relative position within the master rod saddle. Areas near the saddle oil groove are considered "inboard" and areas near the outer edges of the saddle are "outboard."

EPNB RAl-4:

Did the fatigue crack initiate on the surface mating with the journal bearing or the articulated rod bushing?

17

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time APS Response to EPNB RAI-4:

The high cycle fatigue crack on the PVNGS EDG 3B #9 master connecting rod was initiated on the crank pin bore surface (saddle) in the ligament region and propagated towards the articulated rod pin bore (Figure 4).

Figure 4: Fracture Initiation Location Fracture Origin on the Crank Pin Bore Although the initiating crack was damaged during the subsequent component failures, beach marks and other identifiers (found at the metallurgical lab) were used to pinpoint the origin (as observed in Figure 5).

18

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 38 Completion Time Figure 5: Failure Fracture Fracture Origin EPNB-4 Conclusion The crack initiated in the master rod crank pin bore saddle and propagated towards the articulated rod pin bore.

Figure 4 illustrates that the fatigue crack initiated in the master rod saddle ligament area .

Figure 5 illustrates the origin of the fatigue crack on the ligament.

EPNB RAI-5:

Text at various locations appears to imply that there were two points of failure associated with the ligament between the journal bearing and the articulated rod bore (so as to allow the articulated rod to be released from the main rod). Is this correct? Were both failures fatigue failures?

APS Response to EPNB RAI-5:

The high cycle fatigue crack originated on the surface of the crank pin bore (saddle) in line with the ligament and led to the PVNGS 38 DG number 9 master connecting rod failure.

As the crack propagated through the ligament into the articulated rod pin bore, the 19

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time structural integrity of the ligament was compromised. Once the ligament fractured, the bales failed in overload (See Figure 6). Following the failure of the bales, the articulated rod assembly (articulated rod, rod pins, and piston) was liberated to move outside of its designed path and led to consequential engine damage.

Figure 6: Master Rod Failure Ba l e ~

Arti cu l ated Rod Pi n Bore Ligament Area Fatigue Fractu re EPNB-5 Conclusion Given the above, the two failures were not both fatigue failures. The failure in the ligament was high cycle fatigue and the failure of the bales was due to overload following the ligament fatigue failure.

EPNB RAI-6:

Appendix 4 page 7 of 14 - The table shows cracks identified in master rods numbered 2 and

4. Can these cracks be expected to propagate and cause similar failures to that observed in 9? Why would the existence of such cracks not be expected in the other diesels? The significance of these questions is reinforced by bullet 4 on page 10 of 14 which states: "A singular microscopic crack was identified on master rods #2 and # 4 at the edge of the fretted zone. Note: These singular microscopic cracks are different than the patches of parallel micro-cracks seen in the 2003 STP and 2016 PV failures." The NRC staff notes that fretting fatigue is generally considered to be most likely to occur at the edges of fretting surfaces where stresses are highest. Due to the localized nature of these stresses, the enhancement of fatigue by fretting is difficult to quantify.

20

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 38 Completion Time APS Response to EPNB RAI-6:

Absence of Micro Cracks in 3A DG Given the main bearing bore misalignment present in the 3B DG engine since the prior repair, it is reasonable to expect a range of fretting near the oil groove of the master rod ligament to occur especially in the highest stress areas concurrent with misalignment.

Physical examination of the 3B DG master rods concluded that micro cracks were dependent on fretting. This conclusion is based on the fact that the micro cracks identified are solely located in fretting regions (inboard portion of the master rod saddle).

The 3A DG has not experienced a catastrophic event and, therefore, does not have main bearing bore misalignment. Without the misalignment, the stress profile of the 3A DG is consistent with the original design and therefore is not subject to fretting near the oil groove of the master rod ligament.

Micro Crack Propagation in 38 DG It is noted that two types of micro cracks have been observed in number 9 master rod; fretting fatigue cracks that continued to propagate into the base metal by high cycle fatigue and multiple parallel micro cracks in a hardened surface layer that arrest at the interface with the base substrate.

Propagation of a single micro crack (as documented for number 2 and number 4 master rods) cannot yet be predicted with confidence.

EPNB-6 Conclusion Micro cracks would not be expected in engines (3A DG or others) with design features (e.g. fits, tolerances, and alignments) within specifications.

EPNB RAI-7:

Appendix 4, page 11 of 14, the bullet items at the top of the page attribute the fretting to the asymmetrical machining of the crankshaft in 1987 . It is assumed that this refers to position number 9. Was there also similar misalignment/mismachining at position numbers 2 and 4? If not, to what is the presence of cracking at the edge of fretted locations attributed?

APS Response to EPNB RAl-7:

Appendix 4, page 11of14, Summary of Fretting Flaw was intended to assess stresses and potential causes that may result in fretting and evaluate these causes for common mode failure to 3A DG.

Fretting was found near the initiation point for the fatigue cracks that caused the 2003 STP and 2016 PVNGS master rod failures. At PVNGS, fretting was noted to be on one side of the oil groove (asymmetric), but not the other. Inspection of the other nine master rods noted a similar pattern of fretting on some of the rods as illustrated in Table 2 of as referenced in APS letter 102-07411 (Reference 1).

21

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Therefore, potential causes to asymmetrical fretting of the master rod ligament were considered. Crankshaft misalignment or crank pin misalignment were identified as possible causes. Both of these possible causes were considered credible as a direct consequence* of the 1986 3B DG number 9 master rod failure and subsequent repair.

• Crank Pin Misalignment Crank pin misalignment was considered as a possible cause given the number 9 crankpin had been machined (i.e., the outside diameter was reduced and a larger bearing was

• installed) during the 1987 engine repairs. However, neither number 2 or number 4, nor any other crank pin, were damaged during the 1986 failure and thus were not machined.

Pote-ntial crank pin misalignment from machining was unique to the 3B DG number 9

. crank pin.

Crankshaft Main Bearing _Bore Misalignment

• The number 9 master; rod failed as a result of the main bearing bore misalignment, as illustrated in Figures 8 and 9 of Attachment 4 as referenced in APS letter 102-07411, dated December J0, .2016 (Reference 1). Fretting occurred on multiple -master connecting rods because of the misaligned crankshaft bore, which includes the number 2 and number 4 master rods. , .

Cracking at Inboard Fretting Locations The singular microscopic cracks .found on the number 2 and number *4 master rods were unique; these rods did not have the same region of parallel micro cr(lcks as observed_ in" the* fracture surfaces of the STP 2003.and .PVNG*s 2016 master rod failures. The single mftro cracks i_dentified on master rods number 2 and nwmber 4 cannot be positively identifi~d as either a service-induced propagating crack, a consequential crack, or a crack iri a hardened surface layer; EPNB- 7 Conclusion Neither number 2 or number 4 crank pins were damaged during the 1986 failure and thus were not machined. The singular microscopic cracks on-the number 2 and number4

• master rods .are only found in areas of fretting. Fretting in these areas is attributed to-cranksha*ft main bearing bore alignment, whiCh was a direct conseq1,1e-nce of the 1986 3B DG master rod failure. Master rods installed in engines with cin aligned crankshaft are not susceptible to the* type of inboard fre~ting observed in 3B DG. *

• EPNB RAI-8:

Are simila*r engines in service in non-nuclear* applications? What is the reliability of these engines? Is there*_a difference in operating practice and testing (warmup, fast start versus .

slow start, length of operation per start) among the engines being considered (Palo Verde, engines at Braidwood, Byron, South Texas, and Susquehanna, non-nuclear engines)? Would these differences affect this failure?

• 22

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 38 Completion Time APS Response to EPNB RAI-8:

Non-Nuclear Applications Nineteen (19) Cooper-Bessemer KSV engines were installed into non-nuclear service.

These 19 engines included standby power for municipalities, main or propulsion power for marine (e.g., ferry and dredge) application, as well as standby power for radar installations at US Air Force Bases.

Reliability.

Of the nineteen (19) engines placed into non-nuclear service, fifteen (15) units have nbw been retired. Two (2) of the retired engines installed by the US Army Corps of Engineers as propulsion for a large class hopper dredge experienced connecting rod failures in 1992 and 1993. A report (MPR-2287) stated that a fatigue crack initiated at a pit in the connecting rod caused by relatively high lube oil acidity. It was reported that the lube oil had near zero Total Base Number (TBN) shortly before failure. Given that lube oil is transported throughout the diesel engines, the extreme acidity could have affected connecting rods at multiple locations. This failure mode has. been reviewed and deemed*

non-applicable to Palo Verde based on no deviations from specified TBN ..

• The other thirteen (13) engines were retired without any documented master rod failures.

There are four (4) engines that remain in service. They are:

Location Engine Model S12eed Rated Power Notes.

Shelby, OH 16 Cyl - Dual Fuel 514 RPM 3000 KWe First engine ordered in 1969 Colby, KS 12 Cyl - Dual Fuel 514 RPM* - 2500 KWe Second engine ordered in 1969.

Sumner, IA* 12 Cyl - Dual Fuel 514 RPM 2670 KWe Engine ordered in 1971 Arcadia, WI 12 Cyl - Dual Fuel 600 RPM 3080 KWe . Engine ordered in 1971 These four engines have operated upwards of 25,000 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> without master connecting rod failure.

• Operational Differences There are no remaining KSV diesel fuel (only) units; although, these dual fuel units.are typically operated on diesel rather than. in dual fuel mode. These engine-generator sets were originally operated in continuous duty but are now operated in staridpy. Many operate .for only about an hour each month. None of the engines are fast started or fast loaded. The diesel engines at PVNGS are operated in accordance with regulatory guidance and requirements which is consistent with other nuclear power engines.

EPNB-8 Conclusion A review of operating. practices of engines in nuclear and non-nuclear applications did not yield significant insights regarding this failure.

23

• Response to NRC RAis Regarding Em.ergency LAR to Extend Diesel Generator 3B Completion Time EPNB RAl-9:

On Page 3 of Appendix A to Attachment 3, it is stated that at the South Texas Projec:t (STP),

"The root cause was determined to be micro cracks in the master rod big end bore that served as a crack initiation site. Following the second failure, the 20-cylinder engine was rE;?moved from the STP site, totally disassembled, machined and line bored to correct main bearing misalignment, and completely rebuilt with a replacement crankshaft in a large machine facility in Houston, Texas. Since then, the engine as operated successfully for more than 1,000 hours0 days <br />0 hours <br />0 weeks <br />0 months <br />." Please explain why Palo Verde is not using the same repajr method that STP used to successfully repair their engine, given that the engine 38 crankshaft bore has .

been misaligned since the 1986 failure and repair.

APS Response to-EPNB RAI-9:

The repair method for STP and _PVNGS are detail!=!d in the table below:

Diesel Generator

• 2003 STP EOG 22 Repair .2016 PVNGS EOG 38 Repair Reoair Reoair Site Location Offsite On site Diesel Reoair Exoert Yes Yes Crankshaft Reolacement Yes Yes Yes Yes Cran~shaft line bore --

Center frame repairs* Yes Yes '

.NDE of connecting rods Visual, MT, PT- and SOx - Visual, MT,- PT and 200x maqnificatiori magnification -

As noted, the major. difference in the repair approach betWeeri STP and PVNGS is the physical location of the engine repair. STP chose to use a large machine*shop in Houston, Texas whereas Palo Verde is performing all repairs on site. APS'has chosen to use Goltens,

• an independent diesel engine repair specialist, to accomplish the rep.air.

EPNB-9 ConC/usion _

Neither the 2003 STP:nor the current PVNGS repairs are _simila_r to the previous in-situ repairs of these same engines. The 2003 STP repair methodology compared to the APS repair strategy a_t the PVNGS site,* as illustrated above, is'fundainentally the same: '

24

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time Electrical Engineering Branch CEEEB)

EEEB RAl-1:

Please describe the protective relays, which have been provided with each portable DG -

such as type,* model, location/connection of these protective relays. Also, confirm that the protection coordination of these protective relays with the downstream protection relays/devices has been evaluated.

APS Response to EEEB RAI-1:

Each 4160V AC FLEX Plus Cummins diesel generator (DG) is provided with a Cummins PowerCommand 3.3 (PC3.3) controller, a microprocessor based monitoring and control system, which includes generator protectio.n and overcurrent protection, in addition to voltage regulation and engine control functions: The FLEX DG protection system, referred to as the Cummins PowerCommand Control Amp Sentry, is internal to the PC3.3 controller and provides overcurrent protection functions* for the DG set. The Amp Sentry system performs .

tripping functions for the output circuit breaker to provide primary protection for short circuits for both phase and ground faults for the connections between the 4i60V FLEX Plu_s DGs, junction boxes, and 4160V Class 1E load center which also provides backup protection for the existing 4160V equipment between the junction box and 4160V Class 1E load center.*-

<' The rental portable DG is a Cummii:is 480V diesel generator which is provided with the

. Cummins PowerCommand .Control 3201 (PCC), an earlier version of the. PC3.3 controller ..

Overcurrent protection for the rerital DG is accomplished via a Micrologic 3.0 solid state trip system built into the_ Square D Masterpact model NW30H circuit breaker.

• The cqntrollers and .circuit breakers fo~ each of the three portable DGs are mounted. o_n the integrated; trailer-mounted diesel generator 'set.-

• In support of the FLEX Plus initiative, PVNGS purchased two portable Cummins two megawatt 41GOV generators that could be paralleled together to supply the Class 1E 4160V AC buses and enable the use of additional plant loads to *address broader operational conditions, such as shutdown cooling and Mode 5 entry. Calculations were completed in January*2016 (SDOC.NM1000:.A00180), with the purpos_e to: (1) verify that short drcuit levels during FLEX Plus generator operation do not exceed the equipment ratings, (2) calculate the arc flash levels during FLEX Plus generator operation, and (3) verify that settings for circuit breakers and relays added in the FLEX modification meet the design criteria, induding protective device coordination. That calculation concluded that fault conditions downstream of the 4kV load breakers would trip the associated upstream load breaker and not impact the portable diesel g~nerator feeder breakers.

For this RAI question, a new engineering evaluation was completed, using the Electrical Transient Analysis Program (ETAP) software tool which determined that fault and locked rotor conditions on the limiting load would trip the assoeiated load breaker closest to the fault, allowing continued function of the three portable DGs and other ESF components. This evaluation identified that the rental DG initial overcurrent setting was not optimized and the specified setting has been changed and the breaker trip setting adjusted to provide increased margin and greater confidence that the rental DG is coordinated. APS has, therefore, confirmed that the protection coordination of the portable DG protective relays with the downstream protection relays/devices has been appropriately evaluated .

.. 25 --

Response to NRC RAis Regarding Emergency LAR to Extend Diesel Generator 3B Completion Time References

l. Arizona Public Service Company Letter 102-07411, Emergency License Amendment

. Request to Extend Diesel Generator 38 Completion Time, dated December 30, 2016

[Agency Documents Access and Management System (ADAMS) Accession No. ML16365A240]

2. Engineering Evaluation 16-15545-023, Revision 2, PRA Input to second One Time

• Unit 3 Emergency Diesel Generator Train B License Amendment Request, dated January 1, -2017 26