ML24117A035

From kanterella
Jump to navigation Jump to search
TVA Sequoyah Nuclear Plant - 1B Emergency Diesel Generator Regulatory Conference May 02, 2024 -Presentation Slides
ML24117A035
Person / Time
Site: Sequoyah  Tennessee Valley Authority icon.png
Issue date: 05/02/2024
From:
Tennessee Valley Authority
To:
NRC/RGN-II
References
Download: ML24117A035 (1)


Text

Sequoyah Nuclear Plant May 02, 2024 1B Emergency Diesel Generator Regulatory Conference

TVA Introductions and Meeting Agenda - Tom Marshall 2

  • Purpose Tom Marshall, Site Vice President
  • Background Information James Hodge, Director of Engineering
  • Event and Immediate Actions Timeline John Tuite, Director of Maintenance
  • Timeline Continued Beth Jenkins, Plant Manager
  • Causal Product Results Brad Basham, Root Cause Lead
  • Root Cause Review and Considerations Gary Thompson, Diesel Technical Expert, MPR
  • Compliance Summary for Performance Deficiency Rick Medina, Site Licensing Manager
  • Probabilistic Risk Evaluation Frank Hope, J-H Risk-Informed Engineering Mgr
  • Closing Matt Rasmussen, TVA Senior Vice President

Purpose - Tom Marshall 3

Provide an overview of the event, timeline associated with the event, corrective actions, the root cause analysis and the safety significance of the event.

Provide Sequoyahs conclusion based on material analysis findings along with third party technical input.

Provide a detailed summary of the risk factors and inputs for the significance determination of the identified performance deficiency.

Background Information - James Hodge 4

  • Valve lash is controlled through a combination of:
  • Rocker Arm Adjusting Screw
  • The hydraulic lash adjusters are incorporated into the tip of the valve bridge assembly that spans across two exhaust valves that interacts with a single rocker arm.

Background Information 5

  • Fig. 2 Local lash - Is concentrated on either the inboard or outboard exhaust valves and the respective hydraulic lash adjuster.
  • Fig. 1 Global lash - Involves both inboard and outboard exhaust valves, their respective hydraulic lash adjuster assemblies, and is ultimately impacting the whole valve bridge assembly.

Background Information 6

Exhaust valve bridge translates the action of a single rocker arm to a pair of exhaust valves.

Valve bridge spring seat rest in a corresponding semi-spherical seat in the cylinder head.

Each valve bridge contains two hydraulic lash adjuster assemblies.

During engine operation the hydraulic lash adjusters receive oil through oil gallies in the bridge assembly.

Event and Immediate Actions Timeline - John Tuite 7

1B diesel generator 24-hour run starts and after 4 hr. and 24 min run, diesel generator was emergency stopped 09/19/23 09/20/23 1B diesel generator investigation finds failed piston in cylinder #14 1B diesel generator restored with 32 new cylinder power pack assemblies 09/27/23 10/19/23 Successfully completed 12-year PM engine overhauls including power pack replacement on 1A, 2A, and 2B diesels SEP OCT NOV Equipment Apparent Cause Evaluation was completed to conclude an exhaust valve spring retainer failed resulting in damage to the engine 1B2 #14 powerpack.

No failure mode nor firm evidence have been attributed to maintenance activities related to 4-year PM. The vendor analysis of the damaged component was requested to identify causal factors.

11/02/23 11/06/23 Sequoyah confirmed receipt of failed material sent to ESI for a failure analysis report

Timeline Continued - Beth Jenkins 8

Sequoyah Initiated a Level 1 Root Cause Evaluation 01/02/24 12/22/23 Engineering Systems Inc.

(ESI) completed their failure analysis which concluded a loose rocker arm adjusting screw locknut.

Sequoyah Received the failed material back from ESI and sent to TVA Central Labs and Services for metallurgic analysis 12/21/23 01/08/24 Jensen Hughes begins Probabilistic Risk Analysis DEC JAN FEB MPR Engineers visit Central Labs and Services to inspect components 01/31/24 02/15/24 Sequoyah completed and provided an evaluation of risk impact for the DG failure NRC assessed the significance of the finding using the best information available at the time of the SERP 02/21/24 02/28/24 Sequoyah completes and provides Level 1 Root Cause Assessment

Root Cause Analysis Team Charter - Brad Basham 9

Root Cause Team Composition

  • Team Leader - Root Cause SME
  • Analyst - Corporate Root Cause analyst / SME
  • Two Experienced Diesel Generator Engineers
  • Experienced Licensing Engineer
  • Mechanical Maintenance Supervisor
  • Consultant from MPR, Gary Thompson Approach
  • Validate the as found conditions
  • Conduct interviews
  • Reviewed the previous level 2 evaluation
  • Examine the ESI failure analysis
  • Perform review of maintenance work practices, maintenance work history and training
  • Document all potential equipment failure modes and build failure sequencing
  • Conduct extensive OE search
  • Then cross examine using metallurgical examination

Root Cause Investigation - Brad Basham 10 Maintenance Video Walk-through SQN Diesel Generator Maintenance Demo

Root Cause Investigation - Brad Basham 11 Failure Scenarios examined as correlated to similar industry Operating Experience:

Exhaust valve spring retaining ring failure ANO cracked retaining rings Turkey Point cracked retaining rings Rocker arm set screw loose locknut Davis Besse loose locknut Hydraulic lash adjuster failure Turkey Point lash adjuster failure We specifically mapped the loose locknut failure scenario using Industry precedent root cause findings based on the similarity of the ESI conclusion.

12 Davis Besse OE #219503 vs Sequoyah Fig. 3 Davis Besse Damaged Fig. 4 Sequoyah Event Bridge Stem Spherical Brass Seat (No Damage to Bridge Stem Spherical Brass Seat)

  • Based on comparison to the Davis Besse OE, the Sequoyah initiating event did not cause excessive global lash.

Event Comparison Davis Besse Event:

Tapping noise was heard Damage to bridge stem brass seat Dislodging of the bridge stem Damaged cylinder head socket Sequoyah Event:

No noise heard in multiple opportunities No Bridge Stem brass seat damage Bridge stem not dislodged No damage to cylinder head socket

Davis Besse OE #219503 vs Sequoyah Figure 5 Davis Besse Damaged Bridge Stem Cylinder Head Socket with Brass Shavings from Bridge Stem Seat Figure 6 Sequoyah Bridge Stem Cylinder Head Socket (No Elongation Damage)

14 Central Labs Metallurgic Analysis

  • Metallurgic analysis found a pre-existing fatigue crack on the plunger spring.
  • The exhaust valve spring seats had a highly undesirable microstructure that would have made it more susceptible to fracture.

Figure 7 Exhaust Valve Bridge Internals

15 Material Flaw Figure 8 The rear outboard lash adjuster spring w as examined in a scanning electron microscope for fractographic analysis.

The fracture has a helicoidal shape that is indicative of fatigue in springs.

A fatigue crack initiation site w as found at the indicated location on the opposite, inside surface of the spring.

Figure 9 A small thumbnail region oriented 45° from the main spring axis w as found on the inside surface of the rear outboard lash adjuster spring.

Fig 9 is a backscattered electron (BSE) image of the fracture.

The contrast mechanism in BSE images is that of atomic number contrast w here lighter areas contain elements w ith higher atomic numbers and vice versa. In effect, the light areas on fracture are the spring material and the dark areas are typically carbonaceous deposits and/or oxides.

This fracture surface was sonicated in alcohol to remove any loose deposits on the surface.

How ever, oily deposits persisted around the fatigue initiation site.

This indicates that this fracture has been present for a long period of time, i.e., there is a good likelihood that this fatigue crack predated the fracture event.

Figure 8 Rear Outboard Lash Adjuster Spring Figure 9 Inside Surface of Adjuster Spring

Root Cause Investigation - Brad Basham 16 LASH ADJUSTER FAILURE SCENARIO

1. Lash Adjuster spring cracks
2. Lash Adjuster spring fails
3. Lash Adjuster Plunger collapses
4. Local Lash (gap) forms between exhaust valve stem and lash adjuster
5. Hammering effect on valve stem and valve bridge assembly
6. The magnitude of the vibrations and impact forces increases as the local lash increases. Excess vibrations and impact forces on the following:

i.

Exhaust valve ii.

Rocker arm adjusting screw iii.

Lash adjuster iv.

Rocking force on valve bridge stem

7. Global Lash (gap) forms between bridge assembly and rocker arm adjusting screw (after the locknut loosens sufficiently)
8. Retaining Ring failure, dropped exhaust valve, exhaust valve bridge stem fracture, inboard lash adjuster fails, etc

17 Diesel Failure Expert Perspective - Gary Thompson (MPR)

TVA engaged MPR Associates to provide technical support for the SQN RCA effort. Support was provided by Dr. Gary Thompson and other MPR engineers.

Scope of MPR Support

  • Provide direct support to SQN RCA team (e.g., perform visual inspections, discuss key observations)
  • Perform a separate, comprehensive review of relevant information - identify root and contributing cause based on support-refute matrix (11 potential causes considered)

Conclusions Most probable cause is lash adjuster failure due to fatigue fracture of internal spring Lash adjustment screw locknut loosening was a consequence Based on Central Labs evidence of spring fatigue fracture Dissimilarity between SQN and Davis-Besse failures Asymmetry of SQN failure damage Same TVA team used for all lash adjustments; all other locknuts > 80 ft-lbf after failure Number of starts and operating hours between lash adjustment and failure Lack of audible noise prior to failure event (was prominent during Davis-Besse event)

Potential for locknut loosening following valvetrain component failure

18 Valvetrain Operating Loads Assessment Conclusions Lash adjuster failure results in significant vibrational loading of valvetrain components Lash adjustment screw and locknut subjected to increased axial loading and additional lateral loading Loads are significant in magnitude and occur at relatively high frequency (900 rpm or 15 Hz)

Threaded fasteners are susceptible to self-loosening due to vibration, in particular from lateral loading Locknut @ 80 ft-lbf - adequate adjustment screw preload (normal conditions); marginal preload (w/ failed lash adjuster)

Locknut loosening as a consequence of a lash adjuster failure is plausible and should be expected Preload = 6,000 lbf (10,000 to 14,000 lbf is typical)

Elongation = 0.7 mil

< 5° counter-rotation of locknut results in full loss of preload Normal Condition Spring compression 300 to 700 lbf (axial)

Loads increase and decrease smoothly Failed Lash Adjuster Impact loading at high velocities Bending moment on valve bridge assembly Up to 1,900 lbf (axial)

Up to 600 lbf (lateral)

In addition to normal loads

19 Self-Loosening from Vibration Vibrational self-loosening of threaded fasteners is well-documented in technical literature G. Junker - performed pioneering research in late 1960s*; subject of numerous subsequent studies Junker vibration test - developed machine for studying phenomenon and testing effectiveness of solutions Junker, G.H. Video Click Here

  • Junker, G.H., New Criteria for Self-Loosening of Fasteners Under Vibration, SAE Trans 78, 314-355 (1969).

20 Compliance Summary - Rick Medina No LER Submitted

  • When the event occurred, Sequoyah entered our Past Operability Evaluation process.
  • In accordance with NRC NUREG-1022 For the purpose of evaluating the reportability of a discrepancy found during surveillance testing that is required by the TS, licensees should do the following:

For testing that is conducted within the required time, it should be assumed that the discrepancy occurred at the time of its discovery unless there is firm evidence, based on a review of relevant information such as the equipment history and the cause of failure, to indicate that the discrepancy existed previously.

  • Time of Discovery was September 19, 2023. It is not conclusive, and no firm evidence exists that the failure was attributed to Preventive Maintenance activities completed in January 2023 which performed checks and inspections on diesel generator 1B.
  • The station remained in process and had no condition prohibited by Tech Spec or met any other conditions to report under §50.73, therefore, was not required to submit a Licensee Event Report (LER).

21 Compliance Summary Performance Deficiency Summary

  • It is Sequoyahs conclusion that the 1B diesel generator failure occurred on September 19th and the engine was operable until that date.
  • Based on the contrast in thoroughness and technical justification, it is more reasonable to conclude the cause of the 1B diesel cylinder failure was due to an equipment failure and therefore not a performance deficiency.
  • A Self-Revealed AV of TS 5.4.1, Procedures, was identified. The licensees procedures for maintenance on the 1B diesel generator were not adequately prescribed and/or accomplished in accordance with documented instructions and procedures of a type appropriate to the circumstances.
  • Sequoyah Procedure MMTP-104 requires that in part If no torque value can be determined for the specific application, Engineering is to provide specific written guidance or design output for tightening requirements.

Example 4.m states: The PD did not adversely affect the mitigating systems cornerstone objective because the inadequate proce dure would not have resulted in equipment damage. Specifically, although not required by the procedure, maintenance worker trainin g would have the worker set the torque switch to the prior setting.

  • Sequoyah's RCE was completed and delivered one week after the NRC's SERP. We respectfully request that you consider our conclusion and results of the causal analysis, restoring compliance, and ensuring our continued safe operations.

PRA Analysis SQN uses RG 1.200 PRA Model for FPIE, Flood, Fire & Seismic Hazards Includes Level 2 Model Based on NUREG/CR-6595 Approved for 10 CFR 50.69 (ML22334A073)

Approved for RMTS (ML22210A118 & ML15236A351)

No Open Peer Review Findings RASP Handbook Methodology Used to Evaluate Risk Associated with DG 1B Failure

PRA Analysis SQN PRA Model Uses Static Event Trees to Evaluate Event Progression which Assume Run Failures During Mission Time Occur at t = 0 hour0 days <br />0 hours <br />0 weeks <br />0 months <br /> NRC RASP Methodology Assumes Variable 1B-B Diesel Run Capability During Exposure Time that Does Not Easily Fit into t = 0 hour0 days <br />0 hours <br />0 weeks <br />0 months <br /> Run Failure Assumption 0

4 8

12 16 20 24 0

60 120 180 240 Available EDG 1B-B Run TIme (hr)

Exposure Time (days)

PRA - Model Adjustments Model Updates were Performed Consistent with ASME/ANS RA-Sa-2009 to Accommodate Variable Diesel Run Failure Timing and Increase Realism:

High Pressure Fire Protection Credit for Steam Generator Feed Full Circuit Analysis for Permanently Installed 3MW FLEX Diesel Generators to Allow Credit in Additional Fire Areas RCP Seal LOCA Adjustment for Time Dependent Late EDG Failures Main Control Room Abandonment Adjustment for Late EDG Failures LERF Adjustment to Classify Late SBO Sequences As Non-Early Release Turbine Building Fire Modeling Refinement to Split Turbine-Generator Fire Scenarios into Catastrophic vs. Non-Catastrophic Model Updates Supported by Operations Interviews and MAAP Analysis

PRA - Model Conservatisms No Credit for FLEX High, Intermediate and Low Pressure Pumps FLEX 6.9kV and High Pressure Fire Protection Not Credited for Sequences with Assumed Immediate SBO Conditions These Conservatisms Significantly Affect the Results if RASP Handbook CCF Adjustment for Failure to Run During 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> Mission Time is Conservatively Applied at Time Zero (i.e., increased probability of all diesels start but failing to run applied at t = 0 hr due to same cause)

Qualitative/Quantitative Consideration for FLEX/Fire Water Strategies Can Be Applied to Show Significance Determination Not Affected by CCF Adjustments Even if Conservative Diesel Failure Timing Assumptions are Applied

BDB Strategy Considerations SQN Beyond Design Basis Strategies More Robust than Industry Norms Two Installed 3MW 6.9kV FLEX DGs Either DG Can Support PRA Loads NEI 12-06 Timeline ~3 hr 30 min to Energize 6.9kV Shutdown Board Accelerated Non-ELAP Deployment if TDAFW fails; ~2 hr 20 min in Tabletop Exercise from SBO initiation SQN Uses INL DG Failure Data for FLEX Generators (FTS/FTL/FTR)

Data Treatment Accepted by NRC for RICT (ML22118A496, ML22210A118)

BDB Strategy Considerations High Pressure Fire Protection Has Permanent Connection to AFW Strategy Applicable to ECA-0.0 and FR-H.1 Quick Deployment - ~1 hr 50 min Following LOOP Initiator

BDB Strategy Considerations

  • SQN Calculation MDN-000-999-2010-0221 3MW FLEX DG 3.5 hr Deployment Supports RCP Seal LOCA Mitigation Accelerated FLEX DG Deployment for Non-ELAP Procedure Path Results in Power Available Prior to Core Damage for Early TDAFW Failure Sequences Fire Water Credit for SG Feed Also Feasible to Mitigate AFW/FLEX Failures Insights Also Apply to Hydrogen Igniters for Level 2 Application MAAP Run*

AFW Available RCP Seal Leakage (per RCP)

RCS Cooldown Core Damage Time 3MW FLEX Deployment Time Fire Water Deployment Time SBO-02 Yes 21 gpm No

> Mission Time N/A N/A SBO-07 Yes 182 gpm No 6.43 hr 3.50 hr N/A SBO-12 Yes 480 gpm No 2.52 hr 3.50 hr N/A SBO-11 Yes 480 gpm Yes 5.05 hr 3.50 hr N/A SBO-05 Fails at 4 Hrs 21 gpm No 6.51 hr 3.50 hr 5.83 hr SBO-09 Fails at 4 Hrs 182 gpm No 5.65 hr 3.50 hr N/A SBO-05 No 21 gpm No 2.66 hr 2.33 hr 1.83 hr SBO-10 No 182 gpm No 2.43 hr 2.33 hr N/A

Conditional CCF Adjustment TVA Risk Results Based on No CCF Adjustment Because Section 5.0 of RASP Handbook is Not Appropriate for Fatigue Related Material Failures Sensitivity Performed Using Alpha Factor Method from RASP Handbook Timing of Maintenance Related Run Failures Cant Be Predicted Maintenance Related Run Failures Occurring at Same Time is Not Realistic Run Failures with Maintenance Related Proximate Cause Can Be Assumed to Occur with Uniform Distribution Throughout Surveillance Frequency Run Failures with Maintenance Related Proximate Cause Can Be Assumed to Occur with Uniform Distribution Throughout PRA Mission Time INL SPAR Data Framework - DG Run CCF Failure (EPS-EDG-FR) Requires At Minimum Successful Start, Load and One Hour Run of the DGs

Sensitivity Analysis for Conditional CCF Adjustment Sensitivity #1 NUREG/CR-6268 Timing Factors for Multiple Failures High (1.0) Failures Separated by No More Than the PRA Mission Time Medium (0.5) Failures Separated by 1-2 PRA Mission Times Low (0.1) Failures Separated by 2-3 PRA Mission Times Not CCF (0.0) Failures Separated by >3 times the PRA Mission Time Sensitivity #2 PM Frequency Weighting Factor for Multiple Failures CCF Multiplier Using Ratio of Exposure Time to PM Interval (4 year assumed)

Applied to All Common Cause Failure Groups Sensitivities Do Not Include Quantitative Credit for FLEX/Fire Water to Mitigate Time Zero Common Cause Failure of All Diesels to Run 24 Hours.

Early/Late Model Refinements Can Also Be Applied if CCF Events are Broken Up into Different Timing Bins Over the Mission Time.

Sensitivity Analysis for Conditional CCF Adjustment

PRA Results and Conclusion Combined internal events, internal flood, seismic PRA risk, and fire risk analysis results including the supplemental information for the identified deficiency and calculated exposure time:

o CDF calculated to be 5.14E-06/yr (Unit 1) and 9.31E-07/yr (Unit 2) o LERF calculated to be 4.08E-07/yr (Unit 1) and 9.53E-08/yr (Unit 2)

Results are well below the Yellow risk threshold of 1E-5/yr (CDF) and 1E-6/yr (LERF) for Unit 1.

Results are below the White risk threshold of 1E-6/yr (CDF) and 1E-7/yr (LERF) for Unit 2.

Closing - Matt Rasmussen 33 Impactful Event for the Sequoyah Nuclear Station and the TVA Fleet Lessons Learned Corrective Actions Summary of TVAs position An independent failure analysis provided technical justification for a most likely cause of the Diesel Generator failure.

It is Sequoyahs conclusion that the 1B diesel generator failure was due to a material flaw, and we did not have reasonable ability to foresee and prevent this event.

Based on our PRA insights and review of conservatisms, the significance of the event would be White on Unit 1 and Green on Unit 2.