ML24145A085
ML24145A085 | |
Person / Time | |
---|---|
Site: | Sequoyah ![]() |
Issue date: | 05/30/2024 |
From: | Louis Mckown, Mark Franke NRC/RGN-II/DRP/RPB5 |
To: | Jim Barstow Tennessee Valley Authority |
References | |
EA-24-008 IR 2024091 | |
Download: ML24145A085 (10) | |
Text
EA-24-008 Jim Barstow Vice President, Nuclear Regulatory Affairs & Support Services Tennessee Valley Authority 1101 Market Street, LP 4A-C Chattanooga, TN 37402-2801
SUBJECT:
SEQUOYAH, UNITS 1 AND 2 - FINAL SIGNIFICANCE DETERMINATION -
NRC INSPECTION REPORT 05000327/2024091 AND 05000328/2024091
Dear Jim Barstow:
This letter and enclosed inspection report provide you the final significance determination of the preliminary greater-than-green finding discussed in the U.S. Nuclear Regulatory Commissions (NRC) inspection report 05000327, -328/2024090 dated March 19, 2024 (Agency Documents Access and Management System (ADAMS) Accession Number ML24066A197). The preliminary finding concerned self-revealed apparent violations (AV) of Technical Specifications (TS) 3.8.1, AC Sources - Operating, and 5.4.1, Procedures, associated with Sequoyah Nuclear Plants failure to establish, implement, and maintain adequate procedures for maintenance activities on the 1B-B Diesel Generator (DG) exhaust valves, which was understood to be the cause of the 1B-B DG failure on September 19, 2023.
At Tennessee Valley Authoritys (TVA) request, an open regulatory conference was conducted with members of your staff on May 2, 2024, to discuss TVAs position on this issue (ADAMS Accession Number ML24117A035). During the conference, we determined that you and the NRC shared a common understanding of the facts associated with the event. Your organization provided additional information in the form of operating experience, technical data, and descriptions of applicable DG maintenance practices to support your position.
TVA representatives, including diesel engine experts, provided relevant information resulting from a root cause investigation and detailed analyses. These attributed the DG failure to a material flaw developed at a fatigued location in the rear outboard valve bridge lash adjuster spring that was not reasonably within TVAs ability to foresee and prevent. The information TVA provided also included a technical explanation for the loosening of the valve bridge adjuster screw and locknut caused by the lash adjuster spring failure.
Additionally, TVA used a detailed material evaluation performed by TVA Central Lab & Services to compare Sequoyahs failure against other well understood operating experience. TVAs diesel experts shared that in cases of maintenance-induced failure such as due to improper tightening of the rocker arm valve spring adjuster screw and locknut, the DG failed within minutes of restoration. This does not align with Sequoyahs DG failure. Additionally, the experts provided examples of operating experience that showed that while the lash adjuster spring may have failed prior to surveillance testing on September 19, 2023, the engine remained functional May 30, 2024
J. Barstow 2
until the complete failure of the valve spring seat. TVA concluded that the combination of the material characteristics of the exhaust valve spring seat with the unusual loads developed by the fatigue failed lash adjuster spring resulted in the DG failure.
The NRC performed an independent review of the information that you presented at the May 2, 2024, regulatory conference meeting including equipment history, operating experience, and completed equipment and causal analyses. Based upon the newly provided data, the NRC determined that there was no performance deficiency directly associated with the failure of the 1B-B DG. However, the NRC has concluded that a minor violation of TS 5.4.1, Procedures, occurred associated with the licensee's failure to adequately establish and implement maintenance instructions and practices. This minor violation could not be directly attributed to the 1B-B DG failure observed on September 19, 2023, and is documented in the enclosed inspection report.
Moreover, the agency found through its independent review of the best available information that a loss of design basis functionality prior to the failure of the 1B-B DG did not occur.
Therefore, the agency has determined that the AV associated with TS 3.8.1, AC Sources -
Operating, communicated in NRC inspection report 05000327, -328/2024090, dated March 19, 2024 (ADAMS Accession Number ML24066A197) no longer applies.
In accordance with 10 CFR 2.390 of the NRCs Rules of Practice and Procedure, a copy of this letter, its enclosure, and your response, if you choose to provide one, will be made available electronically for public inspection in the NRC Public Document Room or from ADAMS, accessible from the NRC Web site. To the extent possible, your response should not include any personal privacy, proprietary, or safeguards information so that it can be made available to the Public without redaction.
Sincerely, Mark E. Franke, Director Division of Reactor Projects Docket Nos. 05000327 and 05000328 License Nos. DPR-77 and DPR-79
Enclosure:
- 1. Inspection Report cc w/encl: Distribution via LISTSERV Signed by Franke, Mark on 05/30/24
SUNSI Review X
Non-Sensitive
Sensitive X
Publicly Available
Non-Publicly Available OFFICE RII:DRP RII:DRP RII:DRP RII:DRP RII:EICS NAME D. Hardage S. Ninh S. Sandal L. McKown J.R. Ortiz for M. Kowal DATE 5/24/2024 5/24/2024 5/24/2024 5/28/2024 5/28/2024 OFFICE RII:ORA HQ:NRR HQ:NRR HQ:OE RII:DRP NAME D. Cylkowski for S. Price M. Franovich R. Felts D. Bradley for J. Peralta M. Franke DATE 5/29/2024 5/29/2024 5/29/2024 5/29/2024 5/30/2024
Enclosure U.S. NUCLEAR REGULATORY COMMISSION Inspection Report Docket Numbers:
05000327 and 05000328 License Numbers:
DPR-77 and DPR-79 Report Numbers:
05000327/2024091 and 05000328/2024091 Enterprise Identifier:
I-2024-091-0000 Licensee:
Tennessee Valley Authority Facility:
Sequoyah, Units 1 and 2 Location:
Soddy Daisy, TN 37379 Inspection Dates:
March 17, 2024 to May 29, 2024 Inspectors:
D. Hardage, Senior Resident Inspector S. Ninh, Senior Project Engineer A. Price, Resident Inspector A. Rosebrook, Senior Reactor Analyst S. Sandal, Senior Reactor Analyst Approved By:
Mark E. Franke, Director Division of Reactor Projects
2
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a NRC inspection at Sequoyah, Units 1 and 2, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations No findings or violations of more than minor significance were identified.
Additional Tracking Items Type Issue Number Title Report Section Status AV 05000327,05000328/20 23004-01 Failure to establish and implement adequate maintenance procedures on the 1B Diesel Generator EA-24-008 71111.15 Closed
3 INSPECTION SCOPES Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.
INSPECTION RESULTS Assessment 71111.15 Post Regulatory Conference Assessment of September 19, 2023 Sequoyah Diesel Generator 1B-B Failure On September 19, 2023, during a scheduled 24-hour surveillance test of the Sequoyah Diesel Generator (DG) 1B-B which started at 11:21am EST, operators performed an emergency stop of the DG upon receipt of multiple main control room and local annunciators at 3:44pm EST. The DG came to rest at 3:46pm EST. The total runtime for the surveillance test was 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> and 24 minutes. The DG is part of the Class 1E safety related system that provides emergency power to shutdown boards on both units during analyzed events.
At DG 1B-B, Engine 1B2, Cylinder #14, the rear outboard exhaust valve was discovered free of the valve spring seat and stem lock. This exhaust valve had sheared at the mid-stem with the upper portion seized within the upper cylinder head stem guide. Below, within the combustion chamber, the lower stem had broken free from the valve head. The lower stem portion and valve head caused catastrophic damage to the cylinder face, fuel injector, and walls puncturing the jacket cooling water surrounding the cylinder. This resulted in the engine coolant leaking through the damaged areas rendering the engine failed and unrecoverable.
Each cylinder head has three rocker assemblies that operate the fuel injection and combustion exhaust processes based upon their interaction with an overhead cam which times their actions with the combustion cycle of the entire engine. The two exhaust rockers labeled front and rear based upon their relation to the front and rear of the engine assembly border the fuel injection rocker assembly on either side. Each exhaust valve rocker translates motion to an exhaust valve bridge which drives a set of two exhaust valves. The exhaust valve rocker transfers this motion via a hollow rocker arm adjusting screw whose head points downward interfacing with the hemispherical rocker arm adjusting screw bridge seat. The adjusting screw is threaded up through the end of the rocker arm and affixed in place with a locking nut tightened against the upper face of the rocker arm. The exhaust valve bridge is T-shaped with the rocker interface at the center top of the T, a bridge stem which extends centrally downward, and two lash adjusters located above the parallel exhaust valve stems which extend downward through the cylinder head to the exhaust valve heads. The rear or forward exhaust valve sets are individually labeled inboard if located close to the engine centerline or outboard if located away from the engine centerline when referenced to the bridge stem centrally located at T leg of the bridge.
When the cam drives the exhaust valve rocker adjusting screw head downward, this moves
4 the entire bridge assembly downward. This, in turn, translates the downward motion through the two lash adjusters onto the tops of the exhaust valve stems. The exhaust valves open, extending off the cylinder head and allowing exhaust gases to be removed from the combustion chamber. Valve lash is the amount of distance between the ends of the T shaped lash adjusters and the ends of the valve stem. Incorrect valve lash can cause damage to the cylinder, the valves, and/or the valve assembly. Insufficient or negative lash can prevent valves from seating properly leading to high cylinder temperatures and valve damage as the valve cannot properly dissipate heat into the cylinder head when they make contact. Excessive or loose lash can create elevated loads on valves and valve train components leading to their early fatigue failure. To ensure proper valve lash, technicians can set valve lash either locally at an individual valve lash adjuster impacting just the inboard or just the outboard exhaust valve or globally at the rocker arm adjusting screw impacting both the inboard and outboard valves at the same time.
Upon disassembly of the 1B-B DG cylinder #14, the rear outboard valve spring seat was found free of the spring, split into four equal pieces with the valve stem lock. The spring seat was heavily worn and apparently failed due to fatigue. The tip of the valve stem experienced contact damage impacting the lash adjuster plunger. The outboard lash adjuster plunger spring was found broken into two pieces which allowed greater travel of the valve stem into the outboard lash assembly.
The rear inboard valve spring seat remained intact. However, the tip of the inboard valve stem and lash adjuster and plunger did show indication of abnormal wear and contact damage. The lash adjuster plunger spring was broken into three pieces and the lash ball check retainer was lodged within the plunger spring. This prevented the plunger from receding into the lash assembly and allowed less travel of the valve stem into the inboard lash assembly.
The rear valve bridge adjustment screw was discovered fully retracted within the rear exhaust rocker arm with thread damage on the lower 13 threads as well as damage to the rocker arm adjacent to the adjustment screw. The rear valve adjustment screw locknut was discovered removed from the screw but nearby. The rear exhaust bridge had contact damage at the rocker arm interface, a fractured stem, and contact damage on both sides the spring seat.
Following the September 2023 event at cylinder #14, the licensee contracted Engine Systems Incorporated (ESI), for a failure analysis to incorporate their findings into a Level 2 Evaluation Report (CR1881328). ESI is the exclusive worldwide nuclear distributor for EMD Power Products, such as the EMD 645 model DG used at Sequoyah. Within the analysis completed in December 2023, ESI concluded that the initiating condition for the failure was separation of the rear bridge adjuster screw locknut and screw resulting in excessive global lash. The loosening of the locknut allowed the adjuster screw to withdraw into the rocker arm as discovered. This reduced the compression on the bridge causing the elevated loads experienced by both inboard and outboard lash assemblies and the damage observed.
Eventually, the elevated loads caused fractures in the outboard exhaust valve spring seat allowing the valve stem lock to free and the outboard exhaust valve to drop into the combustion chamber. ESI, additionally, identified that while not captured within the EMD 645 manual, standard industry practice ensures tightening of the rocker arm adjustment locknut to 80 ft-lbf minimum. The inspectors found that the Sequoyah maintenance instruction PM 063604820 last performed in January 2023 on the 1B-B DG did not provide explicit torque values for the bridge adjuster locknut. ESI did, further, note that EMD does have a new lash adjustment setting procedure associated with a new model of valve bridge and lash adjusters.
5 The inspectors observed that TVA had implemented the change to the new model of valve bridge and lash adjusters for the EMD 645 DGs at Browns Ferry Nuclear Plant in 2012 and 2013 as well as at Watts Bar Nuclear Plant in 2021 and 2022. TVA planned to implement this change at Sequoyah in mid-November 2023 and pulled up these plans implementing them immediately following the failure of the 1B-B DG.
Based upon the potential significance of the results from ESI, the licensee chose to perform a root cause analysis (CR 1900849) with technical support from diesel generator experts at MPR Associates, Inc. (MPR), and materials analytics experts at TVA Central Labs & Services (CLS). As a result of their detailed investigation, the licensee found excessive local lash to be the most probable cause of the failure of the #14 cylinder. Sequoyah determined that the rear outboard lash adjuster spring experienced fatigue failure initiating from a fatigue related surface flaw which occurred at the spring at an undetermined time prior to failure. The excess lash developed when the failed spring collapsed, providing less compression on the rear outboard exhaust valve. This, in turn, resulted in additional loads on the rear outboard exhaust valve spring seat and lash assembly. These additional loads eventually led to the untimely fatigue failure of the rear outboard valve spring seat causing it to split into four approximately equal parts. CLS observed that the material composition of the spring seat was particularly susceptible to fatigue failure with changes in impact loading. Upon the valve spring seat failure, the valve stem lock became dislodged shortly after causing the valve to drop into the combustion chamber. This resulted in a large local lash that imparted repetitive lateral forces onto the rocker arm valve spring adjuster screw and locknut. The licensee observed that well known mechanical engineering operating experience has identified that certain systems designed to hold under axial loading with minimal pre-load can be highly susceptible to loosening or loss of pre-load under repetitive lateral forces. With the loosening of the bridge adjuster screw, the system began experiencing excessive global lash. This led to the as found conditions at both the inboard and outboard exhaust valve assemblies.
The inspectors received support by combustion engine, materials, and mechanical engineering subject matter experts (SME) in the Mechanical Engineering and Inservice Testing Branch, PRA Licensing Branch C, and the Piping and Head Penetrations Branch.
The NRC staff preliminarily found excessive global lash to be the most probable cause of the event. Consistent with the position of ESI, the inspectors and SMEs observed that the event impacted both the inboard and outboard exhaust valves. While the outboard lash adjuster spring failed once, the inboard lash adjuster spring failed twice with the ball retainer becoming dislodged and entwinned in the spring. The NRC team noted that the wear pattern on the cylinder head side of the rear spherical valve bridge spring seat was worn (polished) evenly high on the spherical dome indicating excessive wear. The rear spring side seat had heavy even material loss from spring contact across the seating surface and neck. While the outboard valve spring seat failed from the excess loads, the broken inboard lash assembly compressed the spring such that the plunger locked in position. The NRC technical staff determined that failure could have been eminent.
However, while the NRC team found excessive global lash as a probable cause, the agency experts recognized the credibility and likelihood in the licensee position that excessive local lash was an alternative probable cause. Similarly, in a letter to the licensee dated April 29, 2024, ESI agree[d] with the technical merit of the reports prepared by TVA and MPR and concede[d] the [outer] lash adjuster failure scenario presented is plausible. During the Regulatory Conference, the NRC asked questions of the licensee diesel engine experts concerning available operating experience. The diesel engine experts did observe that in cases of maintenance induced failure such as due to improper tightening of the rocker arm
6 valve spring adjuster screw and locknut, the DG failed within minutes of restoration not hours.
Nuclear industry operating experience did identify fatigue cracking at valve spring seats.
Cracking was limited in these events to a single location on the valve spring seat. In these situations, ESI and EMD determined that the causes were related to failures within the lash adjuster mechanisms and that full engine functionality remained. The valve spring seat cracking was only discovered due to scheduled performance of comprehensive engine diagnostics performed at very long preventive maintenance frequencies. Lash adjuster subcomponent analysis was not performed as the material was not retained. The diesel engine experts also shared that anecdotal operating experience beyond nuclear applications did not retain the lash adjuster subcomponents upon similar failures. At the Regulatory Conference, the licensee diesel engine experts observed that while the lash adjuster spring may have failed before surveillance testing on September 19, 2023, they were aligned with the prior assessment of ESI and EMD. Hence, the diesel experts concluded that the engine remained functional until the approximate time of the event as there is no firm evidence to indicate when the failure of the lash adjuster spring would result in the failure of the valve spring seat.
As a result of the information provided during the Regulatory Conference, the agency has found through its independent review of the best available information, including but not limited to the equipment history and causal analyses, that it cannot be determined if excessive local lash or excessive global lash caused the cylinder #14 rear outer exhaust valve to drop into the combustion chamber of the 1B-B Diesel Generator at Sequoyah Nuclear Plant on September 19, 2023. Therefore, the NRC has determined that there is no firm evidence to clearly establish any loss of functionality prior to the failure of the 1B-B DG.
Hence, the agency has concluded that the apparent violation of technical specification 3.8.1, AC Sources - Operating, communicated in NRC inspection report 05000327/2024090 and 05000328/2024090, dated March 19, 2024 (ADAMS Accession Number ML24066A197) no longer applies.
Minor Violation 71111.15 Failure to Adequately Establish and Implement Diesel Generator Preventive Maintenance Instruction at Sequoyah Nuclear Plant Minor Violation: During performance of exhaust valve lash adjustment in accordance with maintenance instruction PM 063604820, craft did not stop and seek clarification in accordance with MMTP-104, Guidelines and Methodology for Assembling and Tensioning Threaded Connections, Revision 13. MMTP-104 requires that craft to stop and seek clarification when they encounter tighten in a safety significant restoration in lieu of a specified torque value. PM 063604820 included directions to tighten the bridge adjustment screw but PM 063604820 did not include specified torque values for the activity. Improper performance of PM 063604820 during tightening of the bridge adjustment screw and locknut can result in excessive global lash causing one or more exhaust valves on a rocker arm bridge assembly to drop into the combustion chamber. The inspectors determined that the licensee's failure to adequately establish and implement maintenance instructions and practices was a performance deficiency reasonably within their ability to foresee and prevent.
Screening: The inspectors determined the performance deficiency was minor. Based upon the failure mode uncertainty discussed in the above Assessment, the agency has determined the performance deficiency was not more than minor. While the performance deficiency is associated with the Procedure Quality attribute of the Mitigating Systems cornerstone, it
7 cannot be determined that the performance deficiency adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. That is, in this case, the agency cannot establish with certainty that the failure to adequately establish and implement maintenance instructions and practices in accordance with regulatory requirements and self-imposed standards contributed to the failure of the 1B-B DG. This position is consistent with IMC 0612 Appendix E, Examples of Minor Issues, example 4.m wherein, although not required by the procedure, maintenance worker training would have the worker set the critical parameter.
Enforcement: This failure to comply with Sequoyah Units 1 and 2 Technical Specification 5.4.1, Procedures constitutes a minor violation that is not subject to enforcement action in accordance with the NRCs Enforcement Policy. The licensee restored compliance by incorporating required information into station preventive maintenance procedures.
EXIT MEETINGS AND DEBRIEFS The inspectors verified no proprietary information was retained or documented in this report.
On May 29, 2024, the inspectors presented the NRC inspection results to Tom Marshall and other members of the licensee staff.
8 DOCUMENTS REVIEWED Inspection Procedure Type Designation Description or Title Revision or Date 71111.15 Corrective Action Documents 1881328 71111.15 Corrective Action Documents 1882753 71111.15 Corrective Action Documents 1900849 71111.15 Miscellaneous ESI response - TVA lash adjuster potential Part 21 April 19, 2024 71111.15 Miscellaneous TVA Response to ESI on SQN EDG 1B2 Failure April 25, 2024 71111.15 Miscellaneous ESI Response - Plausibility of Lash Adjuster Failure Scenario April 29, 2024 71111.15 Miscellaneous ML24117A035 TVA Sequoyah Nuclear Plant - 1B Emergency Diesel Generator Regulatory Conference May 02, 2024 -
Presentation Slides April 25, 2024