IR 05000348/2023002
| ML23220A208 | |
| Person / Time | |
|---|---|
| Site: | Farley  |
| Issue date: | 08/10/2023 |
| From: | Alan Blamey Division Reactor Projects II |
| To: | Brown R Southern Nuclear Operating Co |
| References | |
| EA?23?080 IR 2023002 | |
| Download: ML23220A208 (1) | |
Text
SUBJECT:
JOSEPH M. FARLEY NUCLEAR PLANT-INTEGRATED INSPECTION REPORT 05000348/2023002 AND 05000364/2023002 AND APPARENT VIOLATION
Dear R. Keith Brown:
On June 30, 2023, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at Joseph M. Farley Nuclear Plant. On August 2, 2023, the NRC inspectors discussed the results of this inspection with Mr. Dan Williams, Site Regulatory Affairs Manager, and other members of your staff. The results of this inspection are documented in the enclosed report.
Section 71152A of the enclosed report discusses a finding with an associated apparent violation for which the NRC has not yet reached a preliminary significance determination. This involved a self-revealed apparent violation of Title 10 of the Code of Federal Regulations (10 CFR) Part 50, Appendix B, Criterion XVI, associated with the failure to identify and correct a condition adverse to quality associated with the installation instructions of a lube oil coupling assembly for the unit 1 B emergency diesel generator and resulting inoperability of the diesel.
We intend to issue our final safety significance determination and enforcement decision, in writing, within 90 days from the date of this letter. The NRCs significance determination process (SDP) is designed to encourage an open dialogue between your staff and the NRC; however, neither the dialogue nor the written information you provide should affect the timeliness of our final determination. We ask that you promptly provide any relevant information that you would like us to consider in making our determination. We are currently evaluating the significance of this finding and will notify you in a separate correspondence once we have completed our preliminary significance review. You will be given an additional opportunity to provide additional information prior to our final significance determination unless our review concludes that the finding has very low safety significance (i.e., Green).
If you disagree with a cross-cutting aspect assignment in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; and the NRC Resident Inspector at Joseph M. Farley Nuclear Plant.
August 10, 2023 This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with 10 CFR 2.390, Public Inspections, Exemptions, Requests for Withholding.
Sincerely, Alan J. Blamey, Chief Reactor Projects Branch #2 Division of Reactor Projects Docket Nos. 05000348 and 05000364 License Nos. NPF2 and NPF8
Enclosure:
As stated
Inspection Report
Docket Numbers:
05000348 and 05000364
License Numbers:
Report Numbers:
05000348/2023002 and 05000364/2023002
Enterprise Identifier:
I2023002-0015
Licensee:
Southern Nuclear Operating Company, Inc.
Facility:
Joseph M. Farley Nuclear Plant
Location:
Columbia, AL
Inspection Dates:
April 01, 2023, to July 01, 2023
Inspectors:
A. Alen, Senior Project Engineer
P. Gresh, Emergency Preparedness Inspector
K. Kirchbaum, Senior Operations Engineer
P. Meier, Senior Resident Inspector
C. Scott, Senior Project Engineer
S. Temple, Resident Inspector
J. Walker, Sr Emergency Preparedness Inspector
Approved By:
Alan J. Blamey, Chief
Reactor Projects Branch 2
Division of Reactor Projects
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees
performance by conducting an integrated inspection at Joseph M. Farley Nuclear Plant, in
accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs
program for overseeing the safe operation of commercial nuclear power reactors. Refer to
https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
Emergency Diesel Generator Lube Oil Coupling Leak
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Pending
AV 05000348/202300201
Open
EA23080
[H.13] -
Consistent
Process
A self-revealed finding with its safety significance as yet to be determined (TBD) and an
associated apparent violation (AV) of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective
Action, was identified for the licensees failure to identify nonconforming work instructions for
installation of an emergency diesel generator (EDG) lube oil coupling following a unit 1 B EDG
coupling assembly failure in November 2022. Specifically, the licensee failed to adhere to the
troubleshooting standards when it did not evaluate available evidence surrounding the
coupling assembly failure. This resulted in another coupling assembly failure and lube oil leak
during a surveillance run on February 26, 2023, rendering the 1B EDG inoperable.
Additional Tracking Items
Type
Issue Number
Title
Report Section
Status
LER 2023001-00 for Joseph
M. Farley Nuclear Plant, Unit
due to DC Ground on
Turbine Trip Solenoid
Closed
PLANT STATUS
Unit 1 began the report period at approximately 100 percent rated thermal power (RTP) and
remained at or near 100 percent RTP through the end of the report period.
Unit 2 began the inspection period at approximately 100 percent RTP. On April 10, 2023, the
unit was powered down to approximately 88 percent RTP for turbine valve testing. On April 11,
2023, the unit was returned to approximately 100 percent RTP following the turbine valve
testing. On June 2, 2023, an unplanned power reduction to approximately 68 percent RTP was
performed due to bus duct cooling issues. Following restoration of bus duct cooling, the unit was
restored to approximately 100 percent RTP on June 3, 2023. On June 13, 2023, the licensee
performed a planned shutdown of the unit to replace a leaking pressurizer safety valve. On June
14, 2023, the unit entered Mode 5 and on June 19, 2023, the unit was restarted and reached
approximately 100 percent RTP on June 23, 2023. Unit 2 remained at approximately 100
percent RTP through the end of the report period.
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in
effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with
their attached revision histories are located on the public website at http://www.nrc.gov/reading-
rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared
complete when the IP requirements most appropriate to the inspection activity were met
consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection
Program - Operations Phase. The inspectors performed activities described in IMC 2515,
Appendix DProperty "Inspection Manual Chapter" (as page type) with input value "NRC Inspection Manual 2515,</br></br>Appendix D" contains invalid characters or is incomplete and therefore can cause unexpected results during a query or annotation process., Plant Status, observed risk significant activities, and completed onsite portions of
IPs. The inspectors reviewed selected procedures and records, observed activities, and
interviewed personnel to assess licensee performance and compliance with Commission rules
and regulations, license conditions, site procedures, and standards.
REACTOR SAFETY
71111.01 - Adverse Weather Protection
Seasonal Extreme Weather Sample (IP Section 03.01) (1 Sample)
(1)
The inspectors evaluated readiness for seasonal extreme weather conditions prior to
the onset of seasonal hot temperature for the following systems during the month of
May 2023:
Service water system
Isophase bus cooling
River water intake
Impending Severe Weather Sample (IP Section 03.02) (1 Sample)
(1)
The inspectors evaluated the adequacy of the overall preparations to protect risk
significant systems from impending severe thunderstorms and ensure the ability of
personnel to respond to an emergency on June 14, 2023 (procedure
FNP0AOP21.0).
71111.04 - Equipment Alignment
Partial Walkdown Sample (IP Section 03.01) (3 Samples)
The inspectors evaluated system configurations during partial walkdowns of the following
systems/trains:
(1)
Unit 2 auxiliary building vital direct-current (DC) battery charger alignment with the
'2B' battery charger unavailable on April 11, 2023, (dwgs. D207083, D207082)
(2)
Unit 2 B train high head safety injection system with the swing pump aligned to the
B train during the C pump maintenance outage on April 25, 2023, (D205038,
D205039)
(3)
Emergency diesel generator 2C while the 12A emergency diesel generator is out
of service for a planned maintenance outage during the week of May 7, 2023,
(procedure FNP0SOP38.0 and dwg. A181005)
71111.05 - Fire Protection
Fire Area Walkdown and Inspection Sample (IP Section 03.01) (6 Samples)
The inspectors evaluated the implementation of the fire protection program by conducting a
walkdown and performing a review to verify program compliance, equipment functionality,
material condition, and operational readiness of the following fire areas:
(1)
Service water intake structure pump room (FA 72) during the week of April 17, 2023
(procedure FNP0FPP3.0)
(2)
Service water intake structure 'B' train switchgear room (FA 75) during the week of
April 17, 2023 (FNP0FPP3.0)
(3)
Emergency diesel generator '2B' room (FA 59) on May 30, 2023 (FNP0FPP2.0)
(4)
Emergency diesel generator '12A' room (FA 61) on June 27, 2023 (FNP0FPP2.0)
(5)
Unit 1 'B' DC switchgear room (FA 1019) on June 27, 2023 (FNP1FPP1.0)
(6)
Unit 1 and 2 main control room (FA 044) on June 27, 2023 (FNP1FPP1.0)
71111.11Q - Licensed Operator Requalification Program and Licensed Operator Performance
Licensed Operator Performance in the Actual Plant/Main Control Room (IP Section 03.01) (1 Sample)
(1)
The inspectors observed and evaluated licensed operator performance in the control
room for the following activities:
Unit 2 main turbine valve testing on April 10, 2023
Unit 2 operator testing of the 'B' motor-driven auxiliary feedwater pump
utilizing the hot shutdown panel May 2, 2023 (procedure FNP2STP73.1)
Licensed Operator Requalification Training/Examinations (IP Section 03.02) (2 Samples)
(1)
The inspectors observed licensed operator continuing training 'as-left exam' (233
AsFound Exam Scenario #1000) on May 1, 2023.
(2)
The inspectors observed and evaluated operator training on the simulator for an
emergency exercise with offsite participation involving a reactor trip and small break
loss of coolant accident on May 16, 2023.
71111.12 - Maintenance Effectiveness
Maintenance Effectiveness (IP Section 03.01) (2 Samples)
The inspectors evaluated the effectiveness of maintenance to ensure the following
structures, systems, and components (SSCs) remain capable of performing their intended
function:
(1)
Emergency diesel generator 1-2A maintenance outage that included a turbo charger
replacement and circulating oil pump discharge line modifications during the week of
May 8, 2023 (work orders (WO) SNC1462848; SNC785001)
(2)
Extent of condition for all of the station's emergency diesel generators associated with
the Flexmaster coupling leaks as a result of a major oil leak on the unit 1 'B'
emergency diesel generator on February 26, 2023, and a 1C emergency diesel
generator oil leak identified on May 17, 2023, (condition report (CR) 10972464)
Quality Control (IP Section 03.02) (1 Sample)
The inspectors evaluated the effectiveness of maintenance and quality control activities to
ensure the following SSC remains capable of performing its intended function:
(1)
Safety-related components received, handled, stored, and issued for maintenance by
the warehouse on April 5, 2023, (procedures SCM005, NMP-MA009)
71111.13 - Maintenance Risk Assessments and Emergent Work Control
Risk Assessment and Management Sample (IP Section 03.01) (4 Samples)
The inspectors evaluated the accuracy and completeness of risk assessments for the
following planned and emergent work activities to ensure configuration changes and
appropriate work controls were addressed:
(1)
Unit 2 turbine-driven auxiliary feedwater system planned maintenance outage on
April 3 and April 4, 2023 (procedure NMP-OS010)
(2)
Unit 2 risk during planned maintenance on the 'B' charging pump and unit 1 S motor
control center on April 11 and April 12, 2023
(3)
Unit 1 planned maintenance on the 'B' residual heat removal pump with a planned
'yellow' risk condition on April 25, 2023 (NMP-OS010)
(4)
Emergency diesel generator '12A' planned maintenance outage and associated risk
for unit 1 and unit 2 during the week of May 7, 2023 (NMP-DP001)
71111.15 - Operability Determinations and Functionality Assessments
Operability Determination or Functionality Assessment (IP Section 03.01) (5 Samples)
The inspectors evaluated the licensee's justifications and actions associated with the
following operability determinations and functionality assessments:
(1)
Emergency diesel generator 1-2A with scaffolding erected near the turbo charger on
May 3, 2023 (procedure NMP-MA010)
(2)
Unit 2 'B' battery charger multiple fault alarms identified on April 10, 2023, (CR
10963469)
(3)
Unit 1 component cooling water leak identified on April 22, 2023, (CR 10966335)
(4)
Emergency diesel generator 1C lube oil leak identified on May 17, 2023, (CR
10972464)
(5)
Unit 1 'B' accumulator relief valve nitrogen leak identified on May 26, 2023, (CR
10974885)
71111.20 - Refueling and Other Outage Activities
Refueling/Other Outage Sample (IP Section 03.01) (1 Sample)
(1)
The inspectors evaluated the unit 2 force outage for replacement of the 'A' pressurizer
safety valve from June 13 to June 20, 2023.
71111.24 - Testing and Maintenance of Equipment Important to Risk
The inspectors evaluated the following testing and maintenance activities to verify system
operability and/or functionality:
Post-Maintenance Testing (PMT) (IP Section 03.01) (6 Samples)
(1)
Unit 2 turbine-driven auxiliary feedwater valve testing from the hot shutdown panel
following a planned pump maintenance outage on April 4, 2023 (procedure
FNP2STP73.1)
(2)
Unit 1 'S' motor control center (supplies various support loads for the 1-2A
emergency diesel) planned maintenance outage for breaker testing and inspection
during the week of April 10, 2023 (FNP0EMP1323.01)
(3)
Unit 2 'B' charging pump testing following planned maintenance during the week of
Aril 10, 2023 (WOs SNC788624; SNC1390807)
(4)
Unit 2 'C' charging pump testing following balancing line repairs on April 27, 2023,
(WO SNC1162964and FNP2STP4.3)
(5)
Unit 1 'A' charging pump testing following inboard seal replacement on May 23, 2023,
(WO SNC1448127, and FNP1STP4.1)
(6)
Unit 2 'A' pressurizer safety valve testing following valve replacement on
June 19, 2023, (WO SNC1348879 and FNP0MP-3.3)
Surveillance Testing (IP Section 03.01) (5 Samples)
(1)
Emergency diesel generator '12A' 24-hour surveillance run on May 15, 2023,
(FNP0STP80.6)
(2)
Emergency diesel generator '1C' 1-hour surveillance run on May 17, 2023,
(FNP0STP80.2)
(3)
Emergency diesel generator '2B' fast-start test on May 26, 2023, (FNP2STP80.1)
(4)
Unit 2 'B' component cooling water pump quarterly testing on June 12, 2023 (FNP-
STP223.2)
(5)
Unit 1 turbine-driven auxiliary feedwater pump steam supply valve testing on June 26,
2023, (FNP1STP21.3)
Inservice Testing (IST) (IP Section 03.01) (1 Sample)
(1)
Unit 2 'B' motor-driven auxiliary feedwater pump quarterly inservice testing on May 2,
2023 (FNP2STP22.2)
71114.02 - Alert and Notification System Testing
Inspection Review (IP Section 02.0102.04) (1 Sample)
(1)
The inspectors evaluated the maintenance and testing of the alert and notification
system during the week of June 5, 2023.
71114.03 - Emergency Response Organization Staffing and Augmentation System
Inspection Review (IP Section 02.0102.02) (1 Sample)
(1)
The inspectors evaluated the readiness of the Emergency Response Organization
during the week of June 5, 2023.
71114.04 - Emergency Action Level and Emergency Plan Changes
Inspection Review (IP Section 02.0102.03) (1 Sample)
(1)
The inspectors evaluated submitted emergency action level, emergency plan, and
emergency plan implementing procedure changes during the week of June 5, 2023.
This evaluation does not constitute NRC approval.
71114.05 - Maintenance of Emergency Preparedness
Inspection Review (IP Section 02.01 - 02.11) (1 Sample)
(1)
The inspectors evaluated the maintenance of the emergency preparedness program
during the week of June 5, 2023.
71114.06 - Drill Evaluation
Drill/Training Evolution Observation (IP Section 03.02) (1 Sample)
The inspectors evaluated:
(1)
Licensed operator response during a simulator exam involving manual actuation of
safety injection and a failed emergency diesel generator. This simulator exam
represented a drill and exercise performance opportunity (233 As-Found Exam
Scenario #1000) on May 1, 2023.
OTHER ACTIVITIES-BASELINE
71151 - Performance Indicator Verification
The inspectors verified licensee performance indicators submittals listed below:
MS09: Residual Heat Removal Systems (IP Section 02.08) (2 Samples)
(1)
Unit 1 (April 1, 2022 - March 31, 2023)
(2)
Unit 2 (April 1, 2022 - March 31, 2023)
MS10: Cooling Water Support Systems (IP Section 02.09) (2 Samples)
(1)
Unit 1 (April 1, 2022 - March 31, 2023)
(2)
Unit 2 (April 1, 2022 - March 31, 2023)
EP01: Drill/Exercise Performance (DEP) Sample (IP Section 02.12) (1 Sample)
(1)
January 1, 2022, through December 31, 2022.
EP02: Emergency Response Organization (ERO) Drill Participation (IP Section 02.13) (1 Sample)
(1)
January 1, 2022, through December 31, 2022.
EP03: Alert And Notification System (ANS) Reliability Sample (IP Section 02.14) (1 Sample)
(1)
January 1, 2022, through December 31, 2022.
71152A - Annual Follow-up Problem Identification and Resolution
Annual Follow-up of Selected Issues (Section 03.03) (1 Sample)
The inspectors reviewed the licensees implementation of its corrective action program
related to the following issues:
(1)
Unit 1 'B' emergency diesel generator major oil leak identified on February 27, 2023,
(CR 10951589)
71152S - Semiannual Trend Problem Identification and Resolution
Semiannual Trend Review (Section 03.02) (1 Sample)
(1)
Emergency diesel generators oil leaks identified over the last five years (CRs
10920545, 10951589, 10557856)
71153 - Follow Up of Events and Notices of Enforcement Discretion
Event Report (IP section 03.02) (1 Sample)
The inspectors evaluated the following licensee event reports (LERs):
(1)
LER 05000348/2023001-00, "Automatic Reactor Trip due to DC Ground on Turbine
Trip Solenoid" (ADAMS Accession No. ML23089A356). The inspectors determined
that it was not reasonable to foresee or correct the cause discussed in the LER;
therefore, no performance deficiency was identified. The inspectors did not identify
a violation of NRC requirements. The inspectors reviewed the circumstances
regarding the trip in the first quarter of 2023 under the Farley baseline inspection
report 2023001 (ADAMS Accession No. ML23122A168) as maintenance
effectiveness (71111.12) sample 1 and plant modification (71111.18) sample 1.
INSPECTION RESULTS
Emergency Diesel Generator Lube Oil Coupling Leak
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Pending
AV 05000348/202300201
Open
EA23080
[H.13] -
Consistent
Process
A self-revealed finding with its safety significance as yet to be determined (TBD) and an
associated apparent violation (AV) of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective
Action, was identified for the licensees failure to identify nonconforming work instructions for
installation of an emergency diesel generator (EDG) lube oil coupling following a unit 1 B
EDG coupling assembly failure in November 2022. Specifically, the licensee failed to adhere
to the troubleshooting standards when it did not evaluate available evidence surrounding the
coupling assembly failure. This resulted in another coupling assembly failure and lube oil leak
during a surveillance run on February 26, 2023, rendering the 1B EDG inoperable.
Description: While unit 1 was operating (Mode 1) on February 26, 2023, an oil leak of
approximately 22 to 50 gallons per minute (gpm) occurred on the 1B EDG oil circulating
pump discharge pipe coupling during a technical specification (TS) one hour surveillance run.
This rendered the 1B EDG inoperable based on the rate of the oil leak. The 1B EDG was
restored to an operable status on March 3, 2023, following repairs and a modification to
mitigate future failures.
A similar failure at the same location on the 1B EDG occurred while unit 1 was shutdown
(Mode 5) on November 4, 2022, following a planned coupling replacement. The event was
less significant because the 1B EDG was not required to be operable in Mode 5 and the oil
leak was identified during a maintenance run before crediting the EDG for operability.
However, the approximate leak rate and failure mode were the same as the February 26,
2023, event.
The oil circulating pump discharge coupling is designed to absorb a certain amount of
vibration and accommodate some misalignment between the two adjoining pipes. However,
the vendor instructions provide limits on torquing, allowable misalignment, and minimum pipe
insertion depth. When the coupling is installed in accordance with the vendor instructions, the
coupling is rated for 200 psig with sufficient external restraints to account for end loads
developed by internal pressure. The coupling sees head pressure developed by the
circulating oil pump because it is located at the discharge side. The pump is normally running,
whether the EDG is on or off, to provide constant oil filtration and keep the engine internals
warm while the EDG is in the standby condition. When the EDG is off, the circulating oil is at
approximately 25 psig. When the EDG is running, the oil pressure increases to approximately
114 psig due to the engine driven main oil pump. When the EDG was at normal operating
speed and oil pressures, the vertical pipe run at the discharge of the coupling was pulled out
of the top in an upward angular direction, with the pipe assembly center of rotation located
downstream at a 90-degree elbow and threaded connection to a three-way valve. The
inspectors determined the failure was most likely caused by inadequate installation,
inadequate external restraints, or a combination of both combined with the increased oil
pressure during the 1B EDG run.
Due to the similar failure modes of both coupling assembly events, Southern Nuclear
Company (SNC) had an opportunity to prevent the occurrence in February 2023. The original
work order (WO SNC1091597) that replaced the coupling as part of the planned maintenance
in November 2022 appeared to be sufficient. However, during the November post
maintenance test the coupling failed which provided evidence of the couplings new failure
mode. SNC had an opportunity to evaluate the available evidence via their corrective action
program (CAP) to identify the nonconforming work order instructions that led to the coupling
assembly failure. The CAP includes the use of WOs, such as troubleshooting WOs, to
evaluate available evidence and disposition conditions adverse to quality. The following three
paragraphs discuss the link between Criterion XVI and the troubleshooting process.
The SNC Quality Assurance Topical Report (QATR) describes the methods and establishes
quality assurance program and administrative control requirements that meet Title 10 of the
Code of Federal Regulations10 CFR 50, Appendix B. Section 16, Corrective Action, of the
QATR describes the methods to meet Criterion XVI. It states in part, when complex issues
arise where it cannot be readily determined if a condition adverse to quality exist, SNC
documents establish the requirements for documentation and timely evaluation of the issue.
This process starts when a condition report (CR) is written. In accordance with the SNC CAP
procedure that fulfills the regulatory requirements of Criterion XVI (NMP-GM002, version
16), a CR is defined in part, as a document that is initiated to identify any condition
potentially adverse to quality.
The SNC Quality Assurance Program as described in the QATR is also applied to certain
equipment and activities that are not safety related but support safe plant operations. These
activities include those pertaining to maintenance and the assessment and evaluation of
failed items while restoring to their intended condition, such as troubleshooting. As described
in the QATR, SNC commits to compliance with ASME NQA11994. Subpart 2.18 of the
NQA11994 requires that an assessment of failure cause and required maintenance shall
be consistent with the type of item failure and the importance of the item. It further requires
that for failures identified that could have serious effect on safety or operability, an
engineering evaluation shall be performed and documented to substantiate or revise the
failure assessment and corrective action planning.
SNC initiated CR 10920885 to identify the November 4, 2022, 1B EDG lube oil leak and
coupling assembly failure. The licensee closed the CR to WO SNC1399361 to implement
corrective actions in accordance with procedure NMP-GM002001, version 43.0, Corrective
Action Program Instructions. Therefore, the WO was a part of the CAP as defined in NMP-
GM002001. Based on the WO description and inspector interviews with Farley maintenance
and engineering personnel, the purpose of WO SNC1399361 was to provide instructions for
identifying the cause and correct the failed coupling assembly. The specific repair activities
required to address the failure were unknown thus requiring more evidence about what
happened and how it happened in accordance with NMP-MA012003, Maintenance
Standards and Guidelines, for troubleshooting. NMP-MA012003 refers to NMP-AD002,
Conduct of Problem Solving and Troubleshooting, for more specific troubleshooting
performance standards. Due to the unknown repair activities to address the coupling
assembly issue related to a failure of a risk significant safety related EDG, at minimum, NMP-
AD002 requires simple troubleshooting. The amount of troubleshooting rigor increases if the
immediate cause of the failure is not identified.
SNC did not implement the troubleshooting standards when completing WO SNC1399361.
Based on the completed WO record and interviews, maintenance personnel did not identify
an immediate cause of the coupling assembly failure before restoring 1B EDG to operable
status. The corrective actions to address the failure consisted of disassembling and
inspecting the coupling and the circulating oil pump discharge check valve for foreign
material. Maintenance personnel did not identify foreign material or issues with the operation
of the check valve or coupling. Even without a specific cause, the coupling and check valve
were replaced with new like-for-like replacements. The WO lacked any additional information
or documentation to support potential causes or mitigative actions and no engineering
evaluations were performed. If an immediate cause cannot be identified, NMP-AD002
requires operational decision-making per procedure NMP-OS003, Operational Decision
Making Issue Evaluation Process, before restoring the equipment to operable status. One
purpose of the operational decision-making process is for evaluating decisions, such as
potential mitigative actions, affecting the reliability of safety related equipment like the 1B
EDG.
Following the February 26, 2023, 1B EDG coupling assembly failure, the licensee performed
a more rigorous evaluation in which they determined more specific instructions were required
to address the coupling assembly failure. The WO used to repair the failure (SNC1447993)
provided specific guidance for asfound data collection and documentation. Additional steps
required asleft data regarding the adequacy of the piping arrangement to ensure a correct
coupling piping insertion depth. Nothing conclusive was found regarding the immediate
cause. Therefore, the licensee implemented WO SNC1449078 to modify the external
restraints. This solution was developed to mitigate the lube oil piping from pulling out of the
top of the coupling. The modification consisted of adding a welded restraint to the existing
rigid structure and replacing the original conduit clamps with ubolts to increase the rigidity of
the lube oil piping.
If the mitigative actions discussed above were implemented following the November 2022
coupling assembly failure, it is reasonable to assume the February 2023 failure would have
been prevented or minimized to maintain the availability or operability of the 1B EDG. The
licensees causal analysis (CAR 3922914) completed on July 3, 2023, further supports this
conclusion. The analysis determined the 1B EDG coupling assembly failure was directly
caused by the loosening and retightening of the piping during the coupling replacement
which resulted in some loss of pipe thread engagement. This reduced the pipe assembly
rigidity such that the end loads created from the upward motion of the lube oil flow and
pressure created a moment arm that rotated the pipe assembly such that the upper vertical
run of pipe dislodged from the coupling. The analysis also determined that additional
restraints would have prevented the failure.
Corrective Actions: Following the February 2023 event, the licensee added additional external
restraints to the 1B EDG circulation pump lube oil pipe before restoring it back to an operable
status. An extent of condition was performed on all the other EDGs. The concern identified in
this report only applied to the 12A and 2B EDG as they have the same coupling in the same
configuration as the 1B EDG. The licensee evaluated the 12A and 2B EDGs coupling for
adequate installation and monitored for movement during runs. In May 2023, the vulnerability
of the coupling failure was eliminated on the 12A and 2B EDG following a modification that
replaced the coupling with hard pipe. The same modification for the 1B EDG is planned for
July 2025.
Corrective Action References:
CR 10951589: Identified the 1B EDG leak in February 2023
WO SNC1449078: Implementation of modification to the 1B EDG external supports
TE 1123337: Extent of condition evaluation
WO SNC1462848 & SNC1462849: Implementation of the modification to eliminate the
coupling on the 12A and 2B EDGs
CAR 392214: 1B EDG lube oil leak Equipment Reliability Checklist (causal evaluation)
Performance Assessment:
Performance Deficiency: The failure to adhere to the troubleshooting standards as required
by procedure NMP-MA012003, version 7.1, Maintenance Standards and Guidelines, and
NMP-AD002, version 13.8, Conduct of Problem Solving and Troubleshooting following a
substantial 1B EDG lube oil leak on November 4, 2022, was a performance deficiency. As a
result, the licensee failed to identify the nonconforming work order instructions used to
address the November 2022 coupling assembly failure which resulted in another failure that
rendered the 1B EDG inoperable on February 26, 2023.
Screening: The inspectors determined the performance deficiency was more than minor
because it was associated with the Equipment Performance attribute of the Mitigating
Systems cornerstone and adversely affected the cornerstone objective to ensure the
availability, reliability, and capability of systems that respond to initiating events to prevent
undesirable consequences. Specifically, the condition affected the reliability of the 1B EDG to
perform its design basis function.
Significance: The inspectors assessed the significance of the finding using IMC 0609
Appendix AProperty "Inspection Manual Chapter" (as page type) with input value "NRC Inspection Manual 0609</br></br>Appendix A" contains invalid characters or is incomplete and therefore can cause unexpected results during a query or annotation process., The Significance Determination Process (SDP) for Findings At-Power. The
finding could not be screened to be of very low safety significance (i.e., Green because the
condition represented a loss of the PRA function of one train of a multi-train TS system for
greater than its TS allowed outage time, therefore a detailed risk evaluation was required.
The significance determination for the finding is pending a detailed risk assessment that will
be conducted by a regional Senior Reactor Analyst in accordance with IMC 0609 Appendix A.
Cross-Cutting Aspect: H.13 - Consistent Process: Individuals use a consistent, systematic
approach to make decisions. Risk insights are incorporated as appropriate. The licensee
made assumptions about the acceptability of the coupling repair in November 2022 without
formally evaluating the available evidence. In addition, in making the decision to restore the
1B EDG to operable status following the November 2022 repairs, the licensee failed to
consider the risk significance of a potential similar failure while in Mode 1. (DM.1)
Enforcement:
Violation: 10 CFR 50 Appendix B Criterion XVI Corrective Action, states, in part, measures
shall be established to assure that conditions adverse to quality, such as nonconformances
are promptly identified and corrected.
Technical Specification (TS) Limiting condition for operations (LCO) 3.0.1 requires, in part,
that LCOs shall be met during the modes of applicability. TS LCO 3.8.1, AC Sources,
requires, in part, two operable diesel generator sets capable of supplying the onsite Class 1E
distribution systems while in Modes 1, 2, 3, or 4.
Contrary to the above, on November 4, 2022, the licensee failed to identify and correct a
condition adverse to quality associated with nonconforming work instructions for the
installation of a lube oil coupling assembly for the unit 1B EDG following a coupling assembly
failure and substantial lube oil leak. In addition, between December 7, 2022, to March 3,
2023, while the plant was in the modes of applicability, the 1B EDG was inoperable.
Specifically, the licensee did not adequately disposition the failure via troubleshooting WO
SNC1399361 used to implement corrective actions in accordance with procedure NMP-
GM002001, Corrective Action Program Instructions, version 43.0. The disposition was
inadequate because the licensee failed to adhere to its troubleshooting standards and did not
evaluate available evidence surrounding the coupling assembly failure after the immediate
cause of the failure could not be identified during implementation of WO SNC1399361. As a
result, following the failure on November 4, 2022, repairs to the EDG were limited to
replacement of the coupling assembly in accordance with the existing nonconforming work
instructions. This resulted in the inoperability of the EDG due to a similar failure on
February 26, 2023, during a surveillance run. With the 1B EDG inoperable, the licensee failed
to meet the LCO in accordance with TS 3.0.1 and 3.8.1 between December 7, 2022, and
March 3, 2023.
Enforcement Action: This violation is being treated as an apparent violation pending a final
significance (enforcement) determination.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
On August 2, 2023, the inspectors presented the integrated inspection results to Mr. Dan
Williams, Site Regulatory Affairs Manager, and other members of your staff.
On July 18, 2023, the inspectors presented the integrated inspection results to Mr.
Delson Erb, Site Vice President, and other members of the licensee staff.
On June 8, 2023, the inspectors presented the emergency preparedness program
inspection results to Mr. Delson Erb, Site Vice President, and other members of the
licensee staff.