ML23272A027
| ML23272A027 | |
| Person / Time | |
|---|---|
| Site: | Calvert Cliffs  |
| Issue date: | 09/29/2023 |
| From: | Raymond Mckinley Division of Operating Reactors |
| To: | Rhoades D Constellation Energy Generation, Constellation Nuclear |
| References | |
| EA-23-097 IR 2023050 | |
| Download: ML23272A027 (1) | |
See also: IR 05000317/2023050
Text
September 29, 2023
David P. Rhoades
Senior Vice President
Constellation Energy Generation, LLC
President & Chief Nuclear Officer (CNO)
Constellation Nuclear
4300 Winfield Road
Warrenville, IL 60555
SUBJECT:
CALVERT CLIFFS NUCLEAR POWER PLANT, UNIT 1 - SPECIAL
INSPECTION REPORT 05000317/2023050 AND PRELIMINARY WHITE
FINDING AND APPARENT VIOLATION
Dear David Rhoades:
This letter transmits a finding that has preliminarily been determined to be White; a finding with
low to moderate increased safety significance that may require additional inspections by the
U.S. Nuclear Regulatory Commission (NRC). Namely, on April 27, 2023, the NRC completed its
initial assessment of the 1A emergency diesel generator (EDG) failure, which occurred on
April 24, 2023, at Calvert Cliffs Nuclear Power Plant, Unit 1. Based on this initial assessment,
the NRC sent a special inspection team to your site on May 1, 2023.
On August 31, 2023, the NRC completed its special inspection and discussed the results of this
inspection with Patrick Navin, Site Vice President and other members of your staff. The results
of this inspection are documented in the enclosed report.
Section 93812 of the enclosed report documents the preliminary White finding. The finding
involved not properly developing and implementing adequate maintenance instructions and
practices for the 1A EDG since the engine was originally commissioned. We assessed the
significance of the finding using the Significance Determination Process (SDP) and readily
available information.
The finding is also an apparent violation of NRC requirements and is being considered for
escalated enforcement action in accordance with our Enforcement Policy, which can be found at
http://www.nrc.gov/about-nrc/regulatory/enforcement/enforce-pol.html. Because we have not
made a final determination, no notice of violation is being issued at this time. Please be aware
that further NRC review may prompt us to modify the number and characterization of the
apparent violations. We intend to issue our final significance determination and enforcement
decision, in writing, within 90 days from the date of this letter. The NRCs SDP is designed to
D. Rhoades
2
encourage an open dialogue between your staff and the NRC; however, neither the dialogue
nor the written information you provide should affect the timeliness of our final determination.
Before we make a final decision, you may choose to communicate your position on the facts
and assumptions used to arrive at the finding and assess its significance by either (1) attending
and presenting at a regulatory conference or (2) submitting your position in writing. Written
responses should reference the inspection report number and enforcement action number
associated with this letter in the subject line.
If you request a regulatory conference, it should be held within 40 days of your receipt of this
letter. Please provide information you would like us to consider or discuss with you at least
10 days prior to any scheduled conference. The focus of a regulatory conference is to discuss
the significance of the finding. If you choose to attend a regulatory conference, it will be open for
public observation.
If you decide to submit only a written response, it should be sent to the NRC within 40 days of
your receipt of this letter. The response should be clearly marked as a "Response to An
Apparent Violation; (EA-23-097)" and should include for the apparent violation: (1) the reason
for the apparent violation or, if contested, the basis for disputing the apparent violation; (2) the
corrective steps that have been taken and the results achieved; (3) the corrective steps that will
be taken; and (4) the date when full compliance will be achieved. Your response may reference
or include previously docketed correspondence if the correspondence adequately addresses the
required response. Additionally, your response should be sent to the U.S. Nuclear Regulatory
Commission, ATTN: Document Control Center, Washington, DC 20555-0001 with a copy to
Brice Bickett, Chief, Projects Branch 3, U.S. Nuclear Regulatory Commission, Region I, 475
Allendale Road, Suite 102, King of Prussia, PA 19406-1415 within 40 days of the date of this
letter.
Please contact Brice Bickett at 610-337-5312, or in writing, within seven days from the issue
date of this letter to notify the NRC of your intentions. If we have not heard from you within
seven days, we will continue with our significance determination and enforcement decision. If
you choose not to request a regulatory conference or to submit a written response, you will not
be allowed to appeal the NRCs final significance determination.
Three findings of very low safety significance (Green) are also documented in this report. Two of
these findings involved violations of NRC requirements. We are treating these violations as non-
cited violations (NCVs) consistent with Section 2.3.2 of the Enforcement Policy.
If you contest the violations or the significance of the NCVs documented in this inspection
report, you should provide a response within 30 days of the date of this inspection report, with
the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control
Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region I; the
Director, Office of Enforcement; and the NRC Resident Inspector at Calvert Cliffs Nuclear
Power Plant.
D. Rhoades
3
If you disagree with a cross-cutting aspect assignment or a finding not associated with a
regulatory requirement in this report, you should provide a response within 30 days of the date
of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory
Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the
Regional Administrator, Region I; and the NRC Resident Inspector at Calvert Cliffs Nuclear
Power Plant.
This letter, its enclosure, and your response (if any) will be made available for public inspection
and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document
Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public
Inspections, Exemptions, Requests for Withholding.
Sincerely,
Raymond R. McKinley, Deputy Director
Division of Operating Reactor Safety
Docket No. 05000317
License No. DPR-53
Enclosure:
Inspection Report 05000317/2023050
w/Attachments 1 and 2, Detailed Risk
Evaluations, and Attachment 3, Special
Inspection Team Charter
cc w/encl: Distribution via LISTSERV
Raymond R.
McKinley
Digitally signed by Raymond R.
McKinley
Date: 2023.09.29 11:19:26
-04'00'
x
SUNSI Review
x
Non-Sensitive
Sensitive
x
Publicly Available
Non-Publicly Available
OFFICE RI/DORS
RI/DORS
RI/EAGL
OE/EB
RI/DORS
NAME
J DeBoer
D Werkheiser
M McLaughlin
D Bradley
B Bickett
DATE
9/21/23
9/21/23
9/27/23
9/27/23
9/28/23
OFFICE RI/DORS
NAME
R McKinley
DATE
9/28/23
Enclosure
U.S. NUCLEAR REGULATORY COMMISSION
Inspection Report
Docket Number:
05000317
License Number:
Report Number:
Enterprise Identifier: I-2023-050-0002
Licensee:
Constellation Energy Generation, LLC
Facility:
Calvert Cliffs Nuclear Power Plant, Unit 1
Location:
Lusby, MD
Inspection Dates:
May 1, 2023 to August 31, 2023
Inspectors:
J. DeBoer, Senior Project Engineer, Team Lead
R. Clagg, Senior Project Engineer
G. Dipaolo, Senior Resident Inspector
B. Dyke, Operations Engineer
A. Tran, Resident Inspector
D. Werkheiser, Senior Reactor Analyst
Approved By:
Raymond R. McKinley, Deputy Director
Division of Operating Reactor Safety
2
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees
performance by conducting a special inspection at Calvert Cliffs Nuclear Power Plant, Unit 1 in
accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs
program for overseeing the safe operation of commercial nuclear power reactors. Refer to
https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
Failure to ensure applicable guidance regarding allowable piston crack lengths was updated
in the stations vendor technical manual for SACM diesel generators
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Green
Open/Closed
[H.1] -
Resources
93812
The inspectors identified a finding (FIN) of very low safety significance (Green) when
Constellation failed to incorporate applicable guidance into the stations vendor technical
manual Diesel Generator SACM Manual, 18002-091 VTM, Revision 124 consistent with CC-
AA-204, Control of Vendor Equipment Manual, Rev 13. Specifically, Constellation failed to
evaluate and update allowable piston crack length guidance regarding the stations SACM
emergency diesel generator (EDG) into their station procedures and technical manual. As
such, the station was unaware of the appropriate criteria and did not act or evaluate adverse
conditions associated with piston crown cracking observed on the 0C station blackout (SBO)
in May 2023.
Failure to provide adequate work instructions to ensure debris, as a result of the damage
during April 24, 2023, 1A EDG failure, did not adversely affect 1A EDG following restoration to
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Green
Open/Closed
[H.12] - Avoid
Complacency
93812
A self-revealing Green finding and associated non-cited violation (NCV) of Calvert Cliffs
Nuclear Power Plant, Unit 1 Technical Specifications (TS) 5.4.1, Procedures, was identified
for Constellations failure to establish and implement adequate maintenance work instructions
to inspect and assess the condition of the shaft driven lube oil pump gears and bearings
following the 1A EDG 1A2 1B piston failure on April 24, 2023. Specifically, Work Order (WO)
C93909573 failed to include sufficient guidance to inspect and verify the lube oil system, in
particular the shaft driven lube oil pumps on the 1A EDG, were free from debris and damage
because of the April 2023 failure. As a result, 1A EDG failed approximately 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into a
12-hour run on May 18, 2023.
3
Failure to sample stored fuel oil to ensure Emergency Diesel Generator Technical
Specification 3.8.3, Diesel Fuel Oil, requirements were met
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Green
Open/Closed
[P.1] -
Identification
93812
The inspectors identified a Green finding and associated non-cited violation (NCV) of Calvert
Cliffs Nuclear Power Plant, Unit 1 TS 3.8.3, Diesel Fuel Oil, Condition E, for Constellations
failure to perform the required action within the completion time. Specifically, the station failed
to perform the required action of TS 3.8.3, Condition E when upon receipt and subsequent
addition to the 1A EDG fuel oil storage tank (FOST), of new fuel oil not within limits, the
station failed to ensure the 1A FOST fuel oil properties were within limits within the 30-day
allowed completion time.
Failure to establish and implement adequate maintenance practices and work instructions
regarding maintenance of the 1A EDG contributing to the 1A EDG failure on April 24, 2023
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Preliminary White
Open
[H.12] - Avoid
Complacency
93812
A self-revealing finding and apparent violation of Calvert Cliffs Nuclear Power Plant, Unit 1,
TS 5.4.1, Procedures, was identified because activities affecting quality for the 1A EDG were
not adequately prescribed and/or accomplished by Constellation in accordance with
documented instructions and procedures of a type appropriate to the circumstances.
Specifically, Constellation failed to adequately establish and implement maintenance
instructions and practices that reasonably ensured the reliability, availability, and operability of
the 1A EDG. The 1A EDG failed on April 24, 2023, while performing a monthly surveillance
due to the 1A2 B1 piston failure requiring the engine to be placed in an emergency shutdown
condition. The most probable direct cause of the failure was due to loss of compression in the
1A2 1B cylinder due to an improper fueling condition.
Additional Tracking Items
None.
4
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in
effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with
their attached revision histories are located on the public website at http://www.nrc.gov/reading-
rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared
complete when the IP requirements most appropriate to the inspection activity were met
consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection
Program - Operations Phase. The inspectors reviewed selected procedures and records,
observed activities, and interviewed personnel to assess licensee performance and compliance
with Commission rules and regulations, license conditions, site procedures, and standards.
OTHER ACTIVITIES - TEMPORARY INSTRUCTIONS, INFREQUENT AND ABNORMAL
93812 - Special Inspection (1 Sample)
In accordance with the attached updated Special Inspection Team (SIT) Charter, the inspection
team conducted a detailed review of Calvert Cliffs 1A EDG failure on April 24, 2023, and additional
failure of the engine on May 18, 2023, at the Calvert Cliffs Nuclear Power Plant operated by
Constellation (hereafter referred to as "the licensee").
As detailed in the updated SIT Charter, the following items were reviewed:
Description of Event and Reactive Inspection Basis
The 1A EDG is a safety-related 4.16kV, three-phase, 60-cycle, tandem-diesel engine generator set
which has a continuous rating of 5400kW. The 1A EDG consists of two UD45 Societe Alsacienne
De Constructions Mecaniques De Mulhouse (SACM) engines (1A1 and 1A2) that are connected to
a common generator. The engines were installed at Calvert Cliffs Unit 1 in 1996.
On April 24, 2023, while conducting the monthly run of the 1A EDG, the 1A2 engine developed
significant lube oil leakage. Approximately 1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into a 2-hour run, the 1A2 Exhaust Gas Temp
HI alarm came in. While investigating, the operators noted a significant amount of white smoke
coming from the engine in addition to oil leaks from both ends of the engine and, shortly thereafter,
the local control room received the 1A2 lube oil high differential pressure alarms. The 1A EDG was
promptly shutdown. Following the emergency shutdown of the engine, the 1A2 engine was
assessed for damage. The 1A2 EDG was identified to have significant damage to the 1A2 B1
piston crown, the skirt of the piston was fully scratched and showed signs of melted aluminum, the
piston liner was severely scored, large amounts of aluminum debris were found inside the
crankcase sump, and both lube oil filters were found clogged with aluminum shavings.
As a result, the station performed corrective maintenance on the 1A2 engine which included
flushing the fuel oil and lube oil systems, filter replacements, crankcase cleaning, and replacing the
1A2 B1 cylinder liner, connecting rod, piston and rings, head assembly, fuel pump, fuel injector,
wrist pin bearing and pin, as well as corresponding replacements on the 1A2 A1 cylinder. In
addition, maintenance personnel borescoped all 16 cylinders, and all 16 fuel injectors were
pressure and spray tested satisfactorily. Following the corrective maintenance on the 1A2 engine,
the station performed a 12-hour break-in run for the new pistons, performed an additional oil
change, and borescoped the pistons for unexpected wear. Operators then performed a loaded 12-
hour run for post-maintenance operability and declared the engine operable on May 2, 2023, at
0413.
5
Subsequently, on May 18, 2023, while running the 1A EDG per OI-21A, 1A Diesel Generator, the
1A EDG tripped off on Lube-Oil Pressure LO-LO alarms on the 1A2 engine. This was
approximately 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into a 12-hour run. Station personnel identified the cause of the low lube
oil pressure was due to the failure of one of the two shaft-driven lube oil pumps on the 1A2 engine.
The station elected to replace all four lube oil pumps on the 1A EDG. Maintenance personnel also
identified, at this time, significant erosion on six piston crowns and elected to replace 24 of the 32
pistons in the 1A EDG.
The NRC used Management Directive 8.3, NRC Incident Investigation Program, to evaluate the
event (April 24 1A EDG failure) and determine the appropriate NRC response. Based on the
deterministic criteria and risk insights, the NRC determined the event warranted follow-up via a
special inspection. Specifically, the deterministic criteria were met due to repetitive nature and
damage observed on the 1A EDG failures occurring in February 2022 and April 2023. The initial
event risk increase assessment was approximately 4E-6/yr to 5E-6/yr, which was in the SIT range.
This SIT was chartered to identify the circumstances surrounding the April 2023 event and review
the licensees actions to address the causes. The SIT charter was updated to review the
circumstances of a subsequent 1A EDG failure on May 18, 2023.
1. Develop a sequence of events of the 1A EDG failure to include follow-up actions taken by
Constellation. The review should consider licensee-identified timelines and applicable computer
data and logs, as applicable.
2017
Maintenance of the 1A EDG, replaced several power packs
including the 1B cylinder on the 1A2 engine.
February 19, 2022 @ 1:06 am
1A EDG automatically trips on high crankcase pressure of the
1A1 engine.
February 19 - February 24,
2022
Inspection performed on 1A1 EDG. Damage was discovered
on the 3A cylinder of the 1A1 engine which included significant
melting of the piston head and scuffing of the cylinder liner.
Foreign material was discovered in the 3A fuel injector which
caused inadequate fuel delivery leading to piston damage.
February 24, 2022 @ 5:11 pm
Following repairs and post maintenance testing, the 1A EDG
was declared operable.
April 23, 2023 @ 10:55 pm
Slow speed started 1A EDG for scheduled surveillance test.
April 23, 2023 @ 11:18 pm
Paralleled the 1A EDG to the 114kV bus.
April 23, 2023 @ 11:34 pm
1A EDG achieved full load.
April 24, 2023 @
approximately 1:05 am
An Exhaust High Temperature alarm received on the 1A2
engine due to a larger than normal differential temperature
between one cylinder and the average cylinder temperature.
The equipment operator in the field reported smoke above the
1A2 engine. Shortly after, the equipment operator reported two
oil leaks spraying from the engine, one from the governor rack
operating rod boot and one from the generator shaft lab seal.
At the same time, the Lube Oil Filter Differential Pressure High
alarm was received for the A and B filters on the 1A2 engine.
6
April 24, 2023 @ 1:11 am
Control room operators rapidly unloaded and shutdown the 1A
EDG. The equipment operator noted an audible knocking as
the engine slowed down. The 1A EDG was declared inoperable
and the 0C EDG was pre-aligned to the 11 4kv bus. The 1A2
1B piston was found significantly damaged.
April 24 - April 29 2023
Repair and maintenance activities performed on the 1A EDG.
April 29, 2023
A 12-hour loaded maintenance break-in run was performed
which included vibration and acoustic monitoring of the engine.
Following the run, the cylinders were inspected via borescope,
lube oil and filters were replaced, and valve lash checks
performed.
May 1, 2023
NRC Special Inspection initiated.
May 1, 2023
A 12-hour loaded run of 1A EDG for post-maintenance and
surveillance testing was performed satisfactorily.
May 2, 2023 @ 4:13 am
May 17, 2023 @ 7:01 pm
12-hour loaded reliability run of the 1A EDG initiated.
May 18, 2023 @ 12:37 am
The 1A EDG automatically shut down approximately 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />
into the 12-hour run. Multiple alarms are received which
indicate a loss of lube oil pressure
May 18 - June 6, 2023
It is discovered that one of two engine driven lube oil pumps
associated with the 1A2 engine has sustained a sheared shaft
due to clogging of an oil port which normally cools the shaft. All
four lube oil pumps are replaced. 20 pistons and associated
assemblies are also replaced.
June 7, 2023 @ 4:25 am
4-hour loaded run completed satisfactorily on the 1A EDG.
July 14, 2023
1A EDG removed from service to replace all 32 fuel injectors.
July 16, 2023 @ 2:22 pm
Post maintenance test of the 1A EDG completed satisfactorily.
2. Evaluate the adequacy of operator response to the 1A EDG failure. This should include a review
of abnormal and alarm response procedure adherence and technical specification compliance.
The inspectors reviewed the operator response to the 1A EDG failures on April 24 and May 18,
2023. This review included, in part, interviews with station supervisors, control room operators and
equipment operators involved with the diesel surveillances, a review of the shift narrative logs, and
operating procedures used during the tests.
Regarding the April 24 failure, the inspectors concluded that station personnel operated the 1A
EDG in accordance with their site-approved procedures, adequately monitored performance of the
1A EDG, and acted appropriately to shut down the diesel generator in a timely manner when the
engine showed signs of abnormal performance. In particular, the operational crew's pre-job brief
included actions to be taken to rapidly unload the diesel generator, the performance of which may
7
have avoided further damage of the engine. The inspectors did not identify concerns with operator
response or performance.
Regarding the May 18 failure, the 1A EDG automatically shutdown due to the failure of a lube oil
pump and subsequent low lube oil pressure/setpoints being met. The inspectors did not identify
concerns with operator response during the May 18 failure.
3. Evaluate Constellations identification of the failure mode and the troubleshooting approach and
activities that supports the stations understanding and confidence in determination of the direct
cause (April 24, 2023 failure).
The inspectors reviewed licensee procedures MA-AA-716-004, Conduct of Troubleshooting,
Revision 20 and OP-AA-106-101-1001, Event Response Guidelines, Revision 33 to understand
the licensees troubleshooting process and analysis of equipment failure modes, specifically
Failure Modes Causal Tree analysis. Further, the inspectors had consistent engagement with the
root cause team as well as observed in-plant troubleshooting and maintenance and testing
activities that supported the sites determination of the direct cause.
The inspectors reviewed the licensees Failure Modes Causal Tree Analysis for the April 2023
failure of the 1A EDG and noted that, initially, the licensee concluded the most probable cause of
the failure of the 1A2 1B piston was related to an imbalanced fueling and loss of compression
condition. Further, the licensees root cause team ultimately refined and determined that the most
probable cause of the 1A2 1B piston failure was due to excessive accumulation of deposits inside
the fuel injector that led to the loss of control of the fuel injection timing and metering for that
cylinder. The licensee also determined the age of the injector and the lack of adequate
maintenance to monitor the condition of the injector contributed to its failure.
The inspectors observed that the licensee worked through multiple, potential direct causes
throughout the troubleshooting process, including but not limited to piston cracking due to a
manufacturing defect, piston erosion due to an imbalanced fueling condition, before determining
that the most probable direct cause was due to a build-up of deposits inside of the fuel injector
nozzle.
The inspectors determined that the licensees efforts were consistent with station procedures and
processes to identify and affirm a direct cause given the circumstances and information
available. The inspectors observed that the station came to this conclusion, in part, based upon the
results of its root cause efforts, troubleshooting process, maintenance and testing activities, and
vendor engagement conducted over the period of several months, which included aspects of fuel
injector validation testing and analysis at vendor and Constellation laboratories.
The inspectors determined that the likely cause of the failure was related to an abnormal fueling
condition and loss of compression. However, at the conclusion of the SIT, the inspectors noted that
multiple performance aspects related to maintenance of the engine likely contributed to the failure
and the inspectors were not able to determine whether there was one, singular direct cause. The
maintenance performance aspects of the 1A EDG as it relates to the failure on April 24, 2023, are
documented in the Inspection Results section.
8
4. Evaluate Constellations prompt and long-term corrective actions planned/implemented to
address the failure. This review should include an assessment of the adequacy of repair and
testing activities to restore 1A EDG operability.
Given the discovery that occurred following the April 2023 failure as well as additional discovery
and learnings following the May 2023 failure, the licensee implemented numerous corrective
actions over the course of May through early July including but not limited to the following:
- Replaced a significant amount of the mechanical components of the 1A EDG to include the
cylinder liners, pistons and rings, connecting rods, and head assemblies;
- The main crankcase was hand cleaned and all cylinders were visually inspected for
unexpected damage;
- The engines fuel and lube oil systems were inspected and cleaned to include multiple flushing
evolutions and filter changeouts;
- All fuel injectors were spray and pressure tested satisfactorily;
- The fuel racks were adjusted back to a recommended setting from the vendor and governor
linkages were confirmed;
- All four shaft-driven lube oil pumps were replaced; and
4.0-4.4 MW to reduce excess stress on the engine while still satisfying TS requirements.
The licensee further identified corrective actions which include implementing a new fuel receipt
sampling requirement for fuel oil deliveries, utilizing non-nuclear operating experience to help
develop effective preventative maintenance strategies, revising their maintenance templates, and
performing additional training for station personnel regarding development of effective preventative
maintenance.
During this timeframe, the 1A2 1B fuel injector was being evaluated offsite via a vendor and at
Constellation power labs for potential scoring. During this time, the engine manufacturer was made
aware of the material condition of the injector and requested that the injector be sent for in-house
testing and examination. Approximately two minutes into testing at the manufacturers facility, the
fuel injector stuck open and the spray pattern was affected. This injector had previously passed all
spray and pressure tests. In consultation with the manufacturer and with the unsatisfactory bench
test results, the licensee identified that a build-up, inside of the injector nozzle, from diesel fuel
refining byproducts caused scoring internal to the injector and led to increased friction and sporadic
operation. Given the visual and testing evidence, the licensee determined this condition would
cause the over-fueling condition which can lead to the loss of compression due to piston crown
damage from excessive heating.
The licensees root cause team determined that the most probable cause of the 1A2 1B piston
failure in April 2023 was due to excessive buildup of deposits that led to the ultimate loss of control
of the fuel injection timing and metering for that cylinder. The age of the injector and the lack of
adequate maintenance to monitor the condition of the injector contributed to its failure. Based upon
this information, the licensee elected to replace all 32 fuel injector nozzles on the 1A EDG because
they were installed originally in the engine in 1996 and exceeded the vendors recommended
service life.
Based upon information available at the conclusion of this inspection, the inspectors determined
corrective actions were prompt and reasonable to address the likely causes the licensee identified
as well as promptly address new discoveries learned from root cause and vendor support efforts.
This inspection did not assess corrective actions that may be necessary to address underlying
9
human performance or organization factors that contributed to 1A EDG failures. Assessment of
these corrective actions will be reviewed during a subsequent inspection.
5. Evaluate Constellations evaluation of extent-of-condition as it relates to the other EDGs,
including the station blackout diesel generator (0C SBO).
The 1A EDG and 0C SBO diesel generators are the same design and, as such, these engines are
susceptible to the same failure mechanism. With regards to the Fairbanks Morse diesel generators
(1B, 2A, and 2B EDGs), their injector geometry is standard, and they are susceptible to similar
deposit formation, but they would appear to be less susceptible to the same failure mode because
of piston crown material difference and injection pressures. The inspectors noted the Fairbanks
Morse pistons are made from steel versus aluminum which should provide for greater margin to
crown melting in an adverse environment such as experienced on the 1A EDG.
The licensee identified the commonalities with the 0C SBO and are finalizing a plan to replace
injector nozzles on the 0C SBO because it is susceptible to this same failure mechanism. At the
time of the SIT exit, the scheduled date for these injector nozzle replacements was not affirmed
given the on-going engagement with the vendor for parts availability. The licensee determined this
is a time-based failure mechanism and performed a functionality assessment of the 0C SBO to
ensure they did not have an immediate safety concern. The licensee concluded that the 0C SBO is
susceptible to this failure mechanism but has significantly less operational hours (approximately
1100 vs 1600 hours0.0185 days <br />0.444 hours <br />0.00265 weeks <br />6.088e-4 months <br />) and is therefore functional given acceptable performance history as well. The
inspectors reviewed the licensees functionality assessment as well as operational history for the
0C SBO and concluded that, given the failure mode appears to be a time-based mechanism, it is
reasonable to assume that the engine still maintains its functionality but remains vulnerable to a
similar failure until injector nozzles are replaced. The licensee also plans to inspect a sample size
of fuel injectors from the 1B, 2A, and 2B EDGs for the potential of deposit buildup. Those engines
are of a different design manufactured by Fairbanks Morse and operate at a significantly lower
injection pressure.
The inspectors reviewed the licensees extent of condition review and actions and determined
them to be reasonable.
6. Evaluate Constellations causal determinations including the stations review of relevant plant-
specific and industry operating experience. This review should include an assessment of whether
performance issues, both direct and causal, are similar or reflective of performance issues that
contributed to the February 2022 1A EDG failure.
The inspectors reviewed available causal evaluation information for both the April and May 2023
failures as it was developed. The inspectors also reviewed plant specific and industry operating
experience documents as well as consulted with NRC Headquarters subject matter experts to
identify applicable operating experience. The inspectors determined that the licensees causal
determinations, to date, were reasonable given the available information as well as updated to
evaluate continuing discovery information throughout the course of the special inspection.
The inspectors did not identify industry operating experience that appeared directly relevant to the
April 2023 1A EDG failure. In addition to the overall piston damage that occurred during the
February 2022 and April 2023 failures, the inspectors noted that white deposits were found on both
of the failed pistons. The inspectors observed that white deposits were found on several 1A EDG
piston crowns during post-April 2023 failure borescope inspections. The licensees Issue Report
(IR) 04677642, had documented that Prairie Island Nuclear Generating Station (PINGS) identified
10
that similar white deposits had been observed on the PINGS SACM EDGs to some degree since
the installation of the SACM EDGs. The inspectors noted that the 1A EDG vendor did not identify
or appear to provide operating experience regarding similar white deposits. The inspectors
reviewed the licensees evaluation of these deposits, which included its chemical analysis, which
overall documented that the white deposits were likely a lube oil byproduct plating out of
solution. The licensees evaluation determined that these white deposits observed did not
significantly impact engine operation.
The inspectors observed that, during this inspection timeframe, the licensees analysis did not
include a detailed review of diesel fuel oil and the potential contribution to the 1A EDG failure even
though diesel fuel oil with lubricity values in excess of ASTM and site acceptance criteria had
previously been received and introduced into the 1A EDG FOST. Specifically, at the conclusion of
the special inspection, the licensee had not evaluated the potential impacts of higher lubricity
diesel fuel and its contribution to the lack of lubrication within the fuel injector nozzles and resultant
scoring. The inspectors identified that the licensee did not review operating experience for SACM
diesel generators that are utilized in non-nuclear applications. The inspectors noted that since
2008, some early adopters of ultra-low sulfur diesel fuel oil have experienced problems, detailed in
item 10 below, related to particle buildup on fuel injectors.
The station determined that the performance issues that contributed to the February 2022 1A EDG
failure were not directly related to the 1A EDG failure in April 2023. The station determined the
performance aspects of the February 2022 failure were directly related to foreign material that was
found inside the 1A1 3A fuel injector nozzle that was completely foreign to the system. The SIT did
not independently inspect and assess this determination at this time. The underlying performance
aspects of the February 2022 1A EDG failure will be assessed during the upcoming supplemental
inspection, IP 95001, currently scheduled in September 2023.
The inspectors documented a violation in the Inspection Results report section related to the
licensee's failure to sample new diesel fuel oil after receiving fuel oil out of specification.
7. Evaluate Constellations monitoring and maintenance of the 1A EDG systems performance
information including system health reports, maintenance history, and corrective action program
effectiveness for possible trends and overall timeliness of evaluating associated failures,
degradations, and deficiencies.
The inspection team reviewed historical data and maintenance practices to determine the
maintenance history of the 1A EDG and 0C SBO. Overall, the inspectors concluded that the station
did not implement adequate monitoring and maintenance strategies dating back to the original
commissioning of the engines for service in 1996. In particular, the licensee failed to appropriately
implement maintenance strategies and practices consistent with generally accepted SACM UD45
practices, vendor/external consultant recommendations, and station maintenance
procedures/templates. The inspectors concluded these practices and deviations contributed to the
failure of the 1A EDG on April 24, 2023. The details are captured in the Inspection Results section
of this report.
Additionally, in 2017, the licensee elected to perform a modification on the 1A EDG governor
control system. This modification altered how the main engine controller could control the tandem
engine configuration. Prior to the modification the output was controlled by treating the two engines
1A1 and 1A2 independently and sending individual control signals to each engine to achieve the
desired output for the common generator. Following the modification in 2017, the licensee installed
a new control system which treated the 1A1 and 1A2 engines as one and sent a common control
11
signal to both engines to achieve the desired output on the generator. The inspectors noted this
modification did not receive approval from the vendor and is currently recommended by the vendor
to be restored to its original configuration. The licensee determined this modification may have
exacerbated the imbalanced fueling condition which may have contributed to piston erosion
discovered following the April and May failures.
8. Evaluate Constellations assessment of the risk significance of the degraded condition including
evaluation of input assumptions and independently evaluate the risk.
The licensee completed an initial assessment of risk shortly after the April 24th 1A EDG failure
based on early information which was independently reviewed and considered by a regional senior
reactor analyst (SRA) during an independent event risk assessment as part of IMC 0309, Reactive
Inspection Decision Basis for Reactor. As the licensee and inspection team identified and
confirmed facts regarding the degraded condition(s), the licensee developed a significance
determination process (SDP) analysis of the 1A EDG failures which is documented in CA-SDP-
006, Revisions 0 and 1. The teams SRA reviewed the licensees analysis and determined it to be
appropriate and based on reasonable assumptions. One issue was noted with respect to exposure
time during unit outages which was quickly addressed by Constellation and not considered
impactful to the risk conclusion. The SRA performed an independent detailed risk evaluation (DRE)
for the two 1A EDG failures. The DRE and further details of the teams review of Constellations
assessment are documented in the Attachment of this report.
9. Review the circumstances associated with the 1A EDG failure to identify potential common
failure modes and generic safety concerns.
The licensee completed a root cause investigation and concluded that the most probable cause of
the 1A2 1B piston and engine failure was due to the loss of compression due to an incorrect fueling
condition caused by a buildup of deposits found inside the nozzle of the fuel injector which led to
increased friction, scoring, and excessive wear. This condition can cause the fuel injector to
behave intermittently or sporadically as seen by the testing completed by the vendor in June 2023.
The inspectors determined this failure mechanism could pose a potential common failure mode for
high pressure fuel injectors. The inspectors noted that the formation of these deposits inside the
fuel injectors forms in a high pressure, high temperature environment which is common for fuel
injector nozzles. The majority of the deposits have been identified as trace impurities in the diesel
fuel as a byproduct of the refining process of Ultra Low Sulfur diesel. Leaving the fuel injectors
installed since original installation likely contributed to the build-up and excessive scoring identified
inside the nozzle on the 1A2 1B injector. Following replacement of the 32 injector nozzles, the
licensee did identify deposit build up and scoring in a number of the other injectors on the 1A EDG
albeit to a lesser extent.
The inspectors documented a violation in the Inspection Results report section related to the
licensee's failure to establish adequate maintenance practices and work instructions regarding
maintenance of the 1A EDG contributing to the 1A EDG Failure on April 24, 2023.
At the conclusion of the SIT, the inspectors did not identify any generic safety concerns.
12
10. Evaluate whether circumstances, to include refinement of the risk analysis, warrant an
escalation of the special inspection to an augmented inspection after the first week of on-site
information gathering and inspection.
The team, in consultation with the senior risk analysts, determined the circumstances did not
warrant an escalation to an augmented inspection.
11. Review the circumstances surrounding the May 18, 2023, LO-LO lube oil pressure trip on the
1A EDG to identify any association to the 1A EDG failure on April 24, 2023, and related
maintenance and test activities.
The cause of the failure of the May 18, 2023, LO-LO lube oil pressure trip was directly related to
ineffective cleaning and inspecting of engine components that come into contact with the lube oil
system. Following the April 24, 2023, 1A EDG failure, the licensee failed to ensure damaged
engine debris from the April failure was appropriately flushed and removed from the 1A EDG lube
oil system. Specifically, one of the lube oil pump internal cooling ports became clogged with debris
from the piston failure which subsequently led to overheating and a high cycle fatigue failure
resulting in the complete shearing of one of the two shaft driven lube oil pumps. The inspectors
noted the lube oil pumps take a direct unfiltered suction from the lube oil crankcase sump in which
debris accumulated after the April failure.
The inspectors documented a violation in the Inspection Results report section below related to the
licensee's failure to provide adequate work instructions to ensure debris did not adversely affect
the shaft driven lube oil pumps on the 1A EDG following the April 24, 2023, piston failure.
INSPECTION RESULTS
Failure to ensure applicable guidance regarding allowable piston crack lengths was updated
in the stations vendor technical manual for SACM diesel generators
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Green
Open/Closed
[H.1] -
Resources
93812
The inspectors identified a finding (FIN) of very low safety significance (Green) when
Constellation failed to incorporate applicable guidance into the stations vendor technical
manual Diesel Generator SACM Manual, 18002-091 VTM, Revision 124 consistent with CC-
AA-204, Control of Vendor Equipment Manual, Rev 13. Specifically, Constellation failed to
evaluate and update allowable piston crack length guidance regarding the stations SACM
EDG into their station procedures and technical manual. As such, the station was unaware of
the appropriate criteria and did not act or evaluate adverse conditions associated with piston
crown cracking observed on the 0C SBO in May 2023.
Description: The 1A EDG and 0C SBO are identical SACM UD45 tandem engines. 1A EDG
is a Unit 1 safety-related EDG and the 0C SBO is the augmented quality station blackout
diesel that can be aligned to either unit as required. As part of the stations corrective action
response to the 1A EDG April 2023 failure, as an extent of condition, the station created two
action tracking items to borescope the 0C SBO engine cylinders and record all 32 pistons
serial numbers.
13
On May 10, 2023, maintenance personnel borescoped the 0C SBO to determine if there was
any damage to the pistons, liners, or cylinder heads. Inspectors observed borescoping of the
pistons and noted that multiple pistons were borescoped on the same flywheel location.
Procedurally, the piston was to be inspected, engine barred, and steps repeated until all
required inspection requirements were completed. Inspection results for each piston are
recorded in the table as satisfactory or unsatisfactory with comments. Without following the
procedural guidance, there is no guarantee that all inspection requirements are captured.
Technicians used a self-made sheet to ensure all inspection requirements were met, but
these sheets are not documented in WOs nor maintained with records. IR 04681527 captures
this issue, to incorporate all inspection requirements into the procedure and allow pistons to
be inspected in parallel.
Crack indications to varying degrees were found on multiple pistons and cylinder heads. The
following pistons had indications: 0C1 A2 and 0C2 B1, B2, A8, and B8. The pistons with
indications were all original pistons except for 0C1 A2 which was installed in May 2005.
Indications were dispositioned in accordance with the stations vendor technical manual
(VTM) 18002-091-1049, Rev 001. VTM 18002-091-1049 has been in use since the engine
installation and states, in part:
Cracks may appear in piston head lips. These cracks are not dangerous if length is less than
about 19 mm (3/4).
All indications noted in 0C SBO pistons were less than 0.75 inches with the largest length of
0.454 inches on 0C1 A2.
A Technical Bulletin, NT 1760, Rev B, was issued by vendor on June 10, 2010, updating
piston crack guidance that stated, in part:
It is necessary to replace the piston before scheduled maintenance including pistons
dismantling if: a crack is longer than approximately 7mm, two close cracks converge on to
another and can separate a piece of the lip, or the number of crack is higher than 10.
The new crack guidance would have required two pistons, 0C1 A2 and 0C2 B2, to be
replaced because they had cracks greater than 0.276 inches (7mm) in length. 0C1 A2 had
two indications that spanned .307 inches and 0.454 inches in length. 0C2 B2 had one crack
at 0.290 inches. In 2017, the 0C borescope inspection revealed that 0C1 A2 indications that
were 0.132 inches and 0.164 inches and 0C2 B2 had no cracks.
Historical review of pistons revealed that 0C1 B4 piston had an indication of 0.350 inches in
2017. 0C1 B4 piston had indications since 2011. The exact length was not always
documented in their respective WO. 0C1 B4 piston was replaced in 2019.
In May 2017, MPR, an engineering consultant company, prepared MPR-4300, SACM UD45
Diesel Engine Maintenance Technical Basis Report, for Calvert Cliff and Prairie Island
nuclear plants. The report references an email from the vendor dated August 17, 2004, that
stated, in part:
Cracks that are more than 1/2: long or less than 1 apart should be inspected by a Wrtsil/
SACM Technical Representative or the piston should be replaced.
14
Additionally, the MPR report states, in part, current [Calverts] practice, 25% of the
reciprocating assemblies are removed every four years, so that 16 years would elapse
between piston replacement.
Contrary to the above report and vendor guidance, the station analyzed piston cracks with
acceptance criteria of 0.75 inches. Although no piston cracks exceeded 0.5 inches,
Constellation failed to update their piston criteria when contrary information was provided in a
report prepared for them. Additionally, the licensee failed to replace pistons within the 16-year
recommendation as discussed in MPR-4300 which could have prevented cracks from
exceeding vendors guidance.
1A EDG root cause team contacted the vendor on May 16, 2023, for any updated piston
crack guidance. The vendor provided the Technical Bulletin NT1760 to the root cause team.
Two pistons contained cracks greater than the guidance of 7mm. One of the pistons is an
originally installed piston. Therefore, according to the vendors guidance the station should
replace those pistons prior to next scheduled maintenance. Due to the ongoing 1A EDG
failure, the station had limited piston availability. As a result, the licensee reached back to the
vendor for further evaluation. In an email from the vendor, dated May 20, 2023, the vendor
stated that the pistons could still function as they are, given their locations on the piston
crown, but must be replaced as soon as possible.
The inspectors requested a list demonstrating the difference between the stations VTM and
vendor bulletins. The provided list had approximately 125 vendor updates that were not
incorporated into the licensees VTM with the earliest bulletin released in 2001. CC-AA-204,
Control of Vendor Equipment Manual, Rev 13, contained requirements for receiving and
processing vendor technical information, including updating VTMs. The provided list
indicated only seven updates were incorporated. Four updates were incorporated in June
2014 and three updates were incorporated post 1A EDG April 2023 failure.
Corrective Actions: Constellation performed a technical evaluation, ECP-23-000212, to
document the functionality of the 0C SBO due to pistons with cracks exceeding vendor
length. ECP-23-00212 recommended replacement of the 0C1 A2 and 0C2 B2 pistons during
the next available work window which is currently being scheduled. The licensee planned to
validate all applicable vendor technical documents and incorporate them into the licensees
VTM.
Corrective Action References: IR 04677182, 04679091, 04681527
Performance Assessment:
Performance Deficiency: Inspectors determined the failure to incorporate vendor updates in
the stations VTM was a performance deficiency that was reasonable within licensees ability
to foresee and correct. Contrary to CC-AA-204, approximately 125 vendor updates, including
an update for allowable piston crack length, were not incorporated into the licensees VTM
18002-091.
Screening: The inspectors determined the performance deficiency was more than minor
because it was associated with the Equipment Performance attribute of the Mitigating
Systems cornerstone and adversely affected the cornerstone objective to ensure the
availability, reliability, and capability of systems that respond to initiating events to prevent
undesirable consequences. Specifically, piston cracks on the 0C SBO that exceeded the
15
vendor allowed crack length were not replaced. This required Constellation contact the
vendor for additional technical guidance to ensure reasonable confidence of functionality for
the 0C SBO. Additionally, this issue is similar to IMC 0612, Appendix E, Example 1.c
Significance: The inspectors assessed the significance of the finding using IMC 0609,
Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Per
Exhibit 2 - Mitigating Systems Screen Questions, the inspectors answered yes to question
one, "If the finding is a deficiency affecting the design or qualification of a mitigating SSC,
does the SSC maintain its operability or PRA functionality", and screened the finding as
Green.
Cross-Cutting Aspect: H.1 - Resources: Leaders ensure that personnel, equipment,
procedures, and other resources are available and adequate to support nuclear safety.
Specifically, the 0C pistons cracks were not evaluated with up-to-date vendor guidance.
Enforcement: Inspectors did not identify a violation of regulatory requirements associated
with this finding.
Failure to provide adequate work instructions to ensure debris, as a result of the damage
during the April 24, 2023, 1A EDG failure, did not adversely affect 1A EDG following
restoration to operable
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Green
Open/Closed
[H.12] - Avoid
Complacency
93812
A self-revealing Green finding and associated non-cited violation (NCV) of Calvert Cliffs
Nuclear Power Plant, Unit 1 TS 5.4.1, Procedures, was identified for Constellations failure
to establish and implement adequate maintenance work instructions to inspect and assess
the condition of the shaft driven lube oil pump gears and bearings following the 1A EDG
failure on April 24, 2023. Specifically, WO C93909573 failed to include sufficient guidance to
inspect and verify the lube oil system, in particular the shaft driven lube oil pumps on the 1A
EDG were free from debris and damage because of the April 2023 failure. As a result, 1A
EDG failed approximately 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into a 12-hour run on May 18, 2023.
Description: On May 18, 2023, while running the 1A EDG per OI-21A, 1A Diesel Generator,
the 1A EDG tripped off on Lube-Oil Pressure LO-LO. At 0047 alarm windows SL103 LUBE-
OIL PRESS "LO" and SL104 LUBE-OIL PRESS "LO-LO" for the 1A2 engine were received.
Immediately after these alarms, the 1A EDG tripped and alarm window SL45 EMERGENCY
SHUTDOWN SIGNAL came in. This was approximately 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into the 12-hour run.
Normal lube oil system pressure while operating the engine is approximately 105 psig. The
LO-LO lube oil pressure alarm is triggered at 43.5 psig (43 to 44 psig) on 2 of 3 pressure
switches. Following the shutdown, it was discovered that pressure switch 1-PS-10218 did not
actuate due to being out of calibration low but that did not impact the ability of the alarm to
actuate and successfully shutdown the engine as a protective measure since the other 2
pressure switches actuated properly.
During troubleshooting activities, the station discovered that when the crankshaft was barred
over, one of the two-motor driven lube oil pumps (Pump 1) did not spin freely. Maintenance
personnel further investigated the condition and concluded that the pumps shaft had
completely sheared. The station sent the pump to Constellations offsite laboratory (Power
16
Labs) for a detailed failure analysis and concluded that the pump had a significantly blocked
lube oil port, which led to overheating and subsequent shaft shearing. The failure mechanism
of the pump was determined to be high cycle fatigue failure of the lube oil pump due to the
debris material affecting internal pump alignment. Power Labs analyzed the debris
material found in the lube oil port and concluded that the material was most similar to the
pump casing material. Upon inspection and disassembly, Power labs noted beech marks and
striations where the pump motor gears were rubbing on the pump casing and wearing
material away. This damage and misalignment of the pumps internal gears was caused from
debris from the melted 1A2 1B piston failure on April 24, 2023.
The inspectors noted that the 1A2 EDG and supporting systems were cleaned and flushed
multiple times and visually inspected for debris and related damage following the April 24,
2023, 1B piston failure under WO C93909573. However, the inspectors determined, given the
catastrophic failure of the piston and debris, the work instructions for cleaning and inspecting
did not include sufficient instructions to inspect or verify that critical components in the lube oil
system, such as the shaft driven lube oil pumps, were free of debris and damage. The
inspectors did not identify documented information or other related engineering information
that would support that an inspection of the lube oil pumps was not warranted. Additionally,
the inspectors noted these lube oil pumps exceeded service life recommendations (27 years
in-service vice recommended replacement of 12-15 years). That information should have also
influenced the stations approach to inspecting the lube oil system and pumps were free from
damage and debris given the age of the pumps. The inspectors also noted that these pumps
take an unfiltered direct suction from the engines crankcase where the majority of the
aluminum shavings and debris was found. It is likely that during the failure on April 24, 2023,
these pumps consumed large quantities of debris. The inspectors concluded that the failure
to include instructions to inspect and assess the shaft driven lube oil pumps were not
subjected to stresses/debris from the piston failure constituted a performance deficiency that
was within the licensees ability to have foreseen and that should have been prevented.
Corrective Actions: The station replaced all four-shaft driven lube oil pumps, two on the 1A1
engine and two on the 1A2 engine. Constellation also completed a detailed borescope
inspection of lube oil passages that are not easily accessible, such as lube oil coolers and
piping that leads to the lube oil pressure relief valves. Following completion of repairs and
post-maintenance testing the engine was restored to operable on June 7, 2023.
Corrective Action References: IRs 04678953 and 04679941
Performance Assessment:
Performance Deficiency: Constellations failure to establish and implement adequate
maintenance work instructions to inspect and assess the condition of the shaft driven lube oil
pump gears and bearings following the 1A EDG 1A2 1B piston failure on April 24, 2023, was
a performance deficiency. The inspectors concluded that given the observed damage/debris
of the 1A EDG, the fact that these pumps take a direct unfiltered suction from the crankcase
where most of the debris was found, and the extended age of the pumps, it was reasonable
to verify and assess critical components such as the lube oil pumps were not damaged or
impacted for continued operation.
Screening: The inspectors determined the performance deficiency was more than minor
because it was associated with the Equipment Performance attribute of the Mitigating
Systems cornerstone and adversely affected the cornerstone objective to ensure the
availability, reliability, and capability of systems that respond to initiating events to prevent
17
undesirable consequences. Specifically, the inspectors determined that the licensees failure
to inspect or replace the shaft driven lube oil pump on the 1A EDG due to the potential that
debris caused internal damage contributed to the mechanical failure of the 1A2 engine.
Significance: The inspectors assessed the significance of the finding using IMC 0609,
Appendix A, The Significance Determination Process (SDP) for Findings At-Power.
Inspectors used IMC 0609, Exhibit 2, for Mitigating Systems and determined a DRE was
required since the finding was a deficiency affecting the design or qualification of a mitigating
system, structure, and component that did not maintain its probabilistic risk assessment
(PRA) functionality, and the degraded condition represented a loss of the PRA function of one
train of a multi-train TS system for greater than its TS allowed outage time.
A Region I SRA performed a DRE. The finding was determined to be of very low safety
significance (Green). The risk important core damage sequences were dominated by internal
events and loss of offsite power events. The dominant sequences involved loss of offsite
power events with failure of both EDGs (or variations of common cause failure), failure to
align the 0C SBO EDG to supply power to the safety busses, failure to recover an
EDG/declare extended loss of AC power, and subsequent failure to recover offsite power
leading to core damage. Fire (external events) was also a major contributor and dominated
by scenarios associated with fires in the 45 level switchgear room (fire area 430). See
Attachment 2, May 18, 2023, 1A EDG Failure Detailed Risk Evaluation, for a detailed review
of the quantitative and qualitative criteria considered in the risk determination.
Cross-Cutting Aspect: H.12 - Avoid Complacency: Individuals recognize and plan for the
possibility of mistakes, latent issues, and inherent risk, even while expecting successful
outcomes. Individuals implement appropriate error reduction tools. Specifically, Constellation
did not implement appropriate work order instructions to ensure that latent issues, such as a
lube oil pump exceeding its service life, and the inherent risk that a lube oil pump consuming
aluminum shaving could cause unwanted damage and failure.
Enforcement:
Violation: Calvert Cliffs Nuclear Power Plant, Unit 1 TS 5.4.1, Procedures, requires, in part,
that written procedures shall be established, implemented, and maintained as covered in
Regulatory Guide 1.33, Revision 2, Appendix A, February 1978. Regulatory Guide 1.33,
section 9, covers procedures for performing maintenance that can affect the performance of
safety-related equipment and states it should be properly pre-planned and performed in
accordance with written procedures, documented instructions, or drawings.
Contrary to this requirement, following the failure of the 1A EDG on April 24, 2023, adequate
work instructions/orders were not established or adequately implemented such that the 1A
EDG critical components, such as the lube oil system, were properly cleaned and inspected
to be free of damage and debris. Specifically, WO C93909573 or other appropriate work
instruction did not provide steps to adequately inspect or evaluate the condition of the shaft
driven lube oil pumps. As a result, the 1A EDG subsequently failed again on May 18, 2023,
due to debris related failure of the shaft lube oil pump.
Enforcement Action: This violation is being treated as a non-cited violation, consistent with
Section 2.3.2 of the Enforcement Policy.
18
Failure to sample stored fuel oil to ensure Emergency Diesel Generator Technical
Specification 3.8.3, Diesel Fuel Oil, requirements were met
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Green
Open/Closed
[P.1] -
Identification
93812
The inspectors identified a Green finding and associated non-cited violation (NCV) of Calvert
Cliffs Nuclear Power Plant, Unit 1 TS 3.8.3, Diesel Fuel Oil, Condition E, for the licensees
failure to perform the required action within the completion time. Specifically, the licensee
failed to perform the required action of TS 3.8.3, Condition E when upon receipt, and
subsequent addition to the 1A EDG FOST, of new fuel oil not within limits, the licensee failed
to ensure the 1A FOST fuel oil properties were within limits within the 30-day allowed
completion time.
Description: On March 8, 2022, the licensee received a new fuel oil shipment of
approximately 7500 gallons of fuel oil and subsequently added the fuel oil to the 1A FOST.
Samples were taken from the fuel oil tanker prior to delivery as required by licensee
procedure, CY-CA-100-226, Specification and Surveillance - Diesel Fuel Oil, Revision
3. The sample results were received on March 25, 2022, and documented a lubricity value of
614µm. The inspectors reviewed ASTM-D975, Standard Specification for Diesel Fuel, and
noted that the acceptance criteria for lubricity is less than 520µm. The inspectors reviewed
licensee procedure CY-CA-100-226, Specification and Surveillance - Diesel Fuel Oil,
Revision 4 and noted that a lubricity value in excess of 520µm is an Action Level 2 value and
requires notification of the Shift Manager that corrective actions should be taken per
applicable TSs. The inspectors also reviewed licensee procedure CY-AA-120-500, "Fuel Oil
Chemistry," Revision 9 and noted that it requires an IR be written for any unacceptable fuel
oil chemistry results. The inspectors noted that neither of these actions were taken by the
licensee. The inspectors reviewed IR 04498630 and noted that it documented a new fuel oil
shipment that was later received on April 26, 2022, and added to a different FOST, with a
lubricity value in excess of 520µm. This second shipment prompted the licensee to generate
IR 04498630, created on May 9, 2022, documented a corrective action to sample all FOSTs
on site and analyze for lubricity. The sample for the 1A FOST was analyzed on June 2, 2022,
with a documented lubricity value of 396µm, which was within acceptable limits.
The inspectors reviewed Calvert Cliffs Nuclear Power Plant TS Limiting Condition for
Operation (LCO) 3.8.3 and noted it requires that the stored diesel fuel oil shall be within limits
for each required EDG. To meet that LCO, Surveillance Requirement (SR) 3.8.3.2 requires
verification that the properties of new and stored fuel oil are tested in accordance with the
Diesel Fuel Oil Testing Program described in TS 5.5.13. The inspectors reviewed TS 5.5.13
and noted that it requires testing new fuel oil and stored fuel oil to verify it is within ASTM
standards. Some of the parameters including, specific gravity, flash point, kinematic viscosity,
water, and sediment, must be verified before the fuel is added to the storage tanks as they
pose immediate hazard to proper EDG operation. Additional parameters, which include
lubricity, must be verified within 31 days of addition to the storage tanks. While these
parameters don't pose an immediate hazard to the EDG, they can adversely impact reliability
of the engine. A high lubricity value can adversely impact the operation of the fuel injection
system. The inspectors noted that TS 3.8.3, Condition E applies when one or more EDGs
with new fuel oil properties are not within limits, the required action is to restore stored fuel oil
properties to within limits within a completion time of 30 days.
19
The inspectors reviewed Calvert Cliffs TS Basis 3.8.3, Diesel Fuel Oil, Revision 47 and
noted it referenced the 1996 version of the ASTM-D975 standard which did not contain
requirements for fuel oil lubricity. The inspectors also reviewed ECP-11-000373, Changing
from Low-Sulfur Diesel Fuel Oil to Ultra-Low Sulfur Diesel Fuel Oil for the EDGs, effective
September 6, 2012, which was prepared in anticipation of low-sulfur diesel fuel oil becoming
unavailable for sale in the United States. This engineering change package established a limit
for lubricity based on the 2011 ASTM-D975 standard which was incorporated into site
chemistry procedure CP-226 but was not incorporated into the TS bases.
The inspectors concluded that the licensee failed to notify the Shift Manager that corrective
actions should be taken per applicable TSs as required by CY-CA-100-226 when new fuel oil
sample parameter was in excess of the Action Level 2 value. The inspectors concluded that
the licensee failed to perform the actions of TS 3.8.3, Condition E when upon receipt, and
subsequent addition to the 1A FOST, of new fuel oil not within limits, no action was taken to
restore the 1A FOST oil properties to within limits in the 30-day allowed completion time.
Corrective Actions: As noted above, the licensee sampled the 1A FOST to ensure lubricity
was within acceptable limits. The licensee also performed a corrective action program
evaluation (IR 04674511) which created a corrective action to update TS bases 3.8.3 to
reference the 2011 version of the ASTM-D975 standard which includes lubricity as a required
parameter for testing.
Corrective Action References: IR 04498630, 04674511
Performance Assessment:
Performance Deficiency: The inspectors determined that the licensee's failure to properly
implement procedure CY-CA-100-226, Specification and Surveillance - Diesel Fuel Oil,
Revision 4, was a performance deficiency. Specifically, from March 25, 2022, until June 2,
2022, the licensee failed to advise the Shift Manager that corrective actions should be taken
per applicable TS.
Screening: The inspectors determined the performance deficiency was more than minor
because it was associated with the Equipment Performance attribute of the Mitigating
Systems cornerstone and adversely affected the cornerstone objective to ensure the
availability, reliability, and capability of systems that respond to initiating events to prevent
undesirable consequences. Specifically, the failure to ensure that diesel fuel oil lubricity was
in specification adversely impacted the licensee's ability to ensure reliability of the EDG
because high lubricity diesel fuel can cause excessive wear on high pressure fuel
components such as injector and pumps. Additionally, the inspectors reviewed IMC 0612,
Appendix E, Examples of Minor Issues, issued on January 1, 2021, and found example 3.d
to be similar. Specifically, the EDG was run fully loaded for approximately 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> between
March 8, 2022, and June 2, 2022. As a result, the fuel oil sample analyzed on June 2, 2022,
would not be representative of the fuel oil consumed by the EDG during that time period.
Significance: The inspectors assessed the significance of the finding using IMC 0609,
Appendix A, The Significance Determination Process (SDP) for Findings At-Power. The
inspectors answered all questions as No in Exhibit 2 - Mitigating Systems Screen Questions,
Section A - Mitigating SSCs and PRA Functionality, and therefore determined the finding is of
very low safety significance (Green).
20
Cross-Cutting Aspect: P.1 - Identification: The organization implements a corrective action
program with a low threshold for identifying issues. Individuals identify issues completely,
accurately, and in a timely manner in accordance with the program. Specifically, the licensee
failed to write an IR as required by licensee procedure CY-AA-120-500, "Fuel Oil Chemistry,"
Revision 8 contributed to the delay in sampling the 1A FOST.
Enforcement:
Violation: Calvert Cliffs Nuclear Power Plant TS LCO 3.8.1 requires, in part, while in
Modes 1, 2, 3, and 4, two diesel generators each capable of supplying one train of the onsite
Class 1E AC Electrical Power Distribution System be OPERABLE.
TS 3.8.3 requires, in part, that when the associated diesel generator is required to be
operable, the stored diesel fuel oil shall be within limits for each required diesel generator.
SR 3.8.3.2 requires verification that fuel oil properties of new and stored fuel oil are tested in
accordance with the Diesel Fuel Oil Testing Program.
TS 5.5.13, Diesel Fuel Oil Testing Program, requires within 31 days following addition of
new fuel oil to the storage tanks, verify that the properties of the new fuel oil, other than those
addressed in 5.5.13.a, are within limits for ASTM 2D fuel oil.
The TS 3.8.3 ACTION E requires that when one or more diesel generators with new fuel oil
properties are not within limits, restore stored fuel oil properties to within limits within 30 days.
ACTION F requires that with ACTION E and associated completion time not met, declare the
associated diesel generator inoperable immediately.
Contrary to the above, on March 25, 2022, after identifying the new fuel oil properties of the
1A EDG were not within limits, Constellation did not restore the fuel oil properties within limits
within 30 days and did not declare the associated diesel generator inoperable. Specifically,
Constellation identified the lubricity of the 1A EDG fuel oil exceeded the limits for ASTM 2D
fuel on March 25, 2022, and did not verify the stored fuel oil was within limits until June 2,
2022 (a period greater than 30 days) and did not declare the 1A EDG inoperable
immediately.
Enforcement Action: This violation is being treated as a non-cited violation, consistent with
Section 2.3.2 of the Enforcement Policy.
Failure to establish adequate maintenance practices and work instructions regarding
maintenance of the 1A EDG contributing to the 1A EDG Failure on April 24, 2023
Cornerstone
Significance
Cross-Cutting
Aspect
Report
Section
Mitigating
Systems
Preliminary White
Open
[H.12] - Avoid
Complacency
93812
A self-revealing finding and apparent violation of Calvert Cliffs Nuclear Power Plant, Unit 1
TS 5.4.1, Procedures, was identified because activities affecting quality for the 1A EDG
were not adequately prescribed and/or accomplished by Constellation in accordance with
documented instructions and procedures of a type appropriate to the circumstances.
Specifically, Constellation failed to adequately establish and implement maintenance
instructions and practices that reasonably ensured the reliability, availability, and operability
21
of the 1A EDG. The 1A EDG failed on April 24, 2023, while performing a monthly surveillance
due to the 1A2 B1 piston failure requiring the engine to be placed in an emergency shutdown
condition. The most probable direct cause of the failure was due to loss of compression in the
1A2 1B cylinder due to an improper fueling condition.
Description: The 1A EDG is a safety-related 4.16kV, three-phase, 60-cycle, tandem-diesel
engine generator set which has a continuous rating of 5400kW. The 1A EDG consists of two
UD45 SACM engines (1A1 and 1A2) that are connected to a common generator. The
engines were installed at Calvert Cliffs Unit 1 in 1996.
On April 24, 2023, while conducting the monthly run of the 1A EDG, the 1A2 engine
developed significant lube oil leakage. Approximately 1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into a 2-hour run the 1A2
Exhaust Gas Temp "HI" alarm came in. While investigating, the operators noted a significant
amount of white smoke coming from the engine in addition to oil leaks from both ends of the
engine and shortly thereafter the control room received the 1A2 lube oil high differential
pressure alarms. The 1A EDG was emergently shutdown. Following the emergency
shutdown of the engine, the 1A2 engine was assessed for damage. The 1A2 EDG was
identified to have significant damage to the 1A2 B1 piston crown, the skirt of the piston was
fully scratched and showed signs of melted aluminum, the piston liner was severely scored,
large amounts of aluminum debris were found inside the crankcase sump, and both lube oil
filters were found clogged with aluminum shavings.
As a result, the station performed corrective maintenance on the 1A2 engine which included
flushing the fuel oil and lube oil systems, filter replacements, crankcase cleaning, and
replacing the 1A2 B1 cylinder liner, connecting rod, piston and rings, head assembly, fuel
pump, fuel injector, wrist pin bearing and pin, as well as corresponding replacements on the
1A2 A1 cylinder. In addition to the replacements, maintenance personnel borescoped all 16
cylinders and all 16 fuel injectors were pressure and spray tested satisfactorily. Following the
corrective maintenance on the 1A2 engine the station performed a 12-hour break-in run for
the new pistons, performed an additional oil change and borescoped the pistons for
unexpected wear. Operators then performed a loaded 12-hour run for post-maintenance
operability and declared the engine operable on May 2, 2023 at 0413.
Subsequently, on May 18, 2023, while running the 1A EDG per OI-21A, 1A Diesel
Generator, the 1A EDG tripped off on Lube-Oil Pressure LO-LO alarms on the 1A2
engine. This was approximately 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into a 12-hour run. Station personnel identified the
cause of the low lube oil pressure was due to the failure of one of the two shaft-driven lube oil
pumps on the 1A2 engine. The station elected to replace all four lube oil pumps on the 1A
EDG. Maintenance personnel also identified, at this time, significant erosion on six piston
crowns and elected to replace 20 of the 32 pistons in the 1A EDG which were from the same
lot in 2017. Of the remaining 12 pistons, 4 were replaced following the February 2022 and
April 2023 failures and the remaining 8 were replaced during scheduled maintenance in 2020.
During this maintenance and in consultation with the vendor, it was identified that the fuel
racks-to-fuel pump linkages were out-of-specification which could have the potential to lead to
an imbalanced fueling condition which could cause piston erosion and potential failure. The
station worked to rebalance the fuel rack linkages to ensure that the minimum fuel stroke
length was uniform on all 32 cylinders with a value of 3.4mm. Following replacement of the
pistons and rebalancing of the fuel racks-to-fuel pump interface, station personnel performed
a 12-hour break-in run loaded at 4.0MW-4.4MW and performed a subsequent borescope and
lube oil change. Operators then successfully performed a 4-hour post-maintenance
operability surveillance of the 1A EDG on June 7, 2023, and the 1A EDG was declared
22
The inspectors determined that the maintenance instructions, procedures, and practices were
not adequate in a number of areas that likely caused, or significantly contributed to, the
April 24, 2023, failure and, more specifically, to the loss of compression and imbalanced
fueling condition. Namely, the inspectors determined that station procedures and work
instructions did not incorporate appropriate guidance and common practices consistent with
maintenance templates, vendor technical manuals/guidance, and general UD45 EDG
maintenance practices applicable to the 1A EDG.
The inspectors identified that the Wartsila SACM Diesel Technical Manual, Section 3-0,
Maintenance Schedule, specifies every ten to twelve years, removal of diesel engine for
general overhaul in workshop or on site with Wrtsil SACM Diesel assistance.
Wrtsil/SACM factory engine overhauls are characterized as requalification efforts, with the
goal of rebuilding the engine to a condition comparable to a new unit. These efforts include, in
part, replacing lube oil pumps, fuel injector pumps, fuel injectors, rocker arms, push rods and
inspection of the crankshaft and camshaft. However, the inspectors identified the licensee
failed to replace fuel injectors, ensure fuel rack fuel pump stroke lengths, check valve timing,
check injection pump delivery rate, and inspect cam shafts and cam lobes or replace fuel
injector pumps consistent with this guidance or develop an appropriate site-specific
maintenance plan or instruction to address these recommendations. The performance of
these maintenance tasks should have either been completed during scheduled maintenance
outages or major overhauls such as the 5-6 year overhaul or during the 10-12 year
overhaul. The omission of these critical tasks impacts the long-term reliable operation of the
engine and likely contributed to the failure of the 1A2 engine on April 24, 2023.
The inspectors noted that in 2017, the station contracted a consultant to provide guidance
and a technical basis for the continued operation of major engine components in SACM
Model UD45 diesel engines in nuclear standby service in the United States. The contractor
performed an analysis and provided recommendations to justify deviations from the vendors
recommended service and provided a revised preventative maintenance template which
included a number of recommendations. One noteworthy recommendation was Check the
fuel pump rack settings for uniformity on a 24-month frequency. This recommendation was
incorporated into the corporate maintenance template, SACM Diesel Generator, Revision 1
but it was not translated into EDG-13, 24 Month Inspection of SACM Diesel Generator,
Revision 17 and, therefore, never performed. The contractors analysis did not provide a
recommended time-based frequency to replace fuel injectors but, instead, recommended a
testing frequency for pressure and spray tests at an interval of 24 months, which was
incorporated into EDG-13. The inspectors noted this maintenance approach does not have
the ability to consistently detect long-term degradation mechanisms or intermittent failure
modes.
Further, Constellation failed to follow their own maintenance template for the UD45 engine.
Specifically, the maintenance template SACM Diesel Generator, Revision 1 described
actions to check and adjust valve timing, fuel rack length, and fuel rack settings every 24
months for this type of EDG and these maintenance items had not been completed or
evaluated for acceptability in not completing these tasks. In addition to that action, the
maintenance template also prescribed to replace fuel rack linkages swivel joints and bearings
at a frequency of 20 years and this action has never been performed which could have
contributed to the fuel racks being out of specification. The omission of these critical tasks
impact long-term reliable operation of the engines and likely contributed to the failure of the
1A2 engine on April 24, 2023.
23
Additionally, the inspectors observed that, after the April and May 2023 failures, the licensee
continued to evaluate the parts removed from the engine with offsite vendors and
Constellation Power Labs. During those evaluations, it was determined that the removed 1A2
B1 fuel injector exhibited scoring on the spindle and nozzle body. The injector had previously
passed all spray and pressure tests prior to this discovery. The licensee shared this
information with Wrtsil/SACM and the injector was sent to France to be tested at their
repair center. The injector failed testing approximately 2 minutes into testing and stuck open
intermittently. Based upon the results of this testing, the licensee elected to replace all 32
injector nozzles in July 2023 and successfully performed post-maintenance testing
surveillance and declared the engine operable. The inspectors noted that the majority of
the fuel injectors removed were original and well exceeded the replacement frequency that
was recommended by Wrtsil/SACM and common general EDG maintenance practices
applicable to the UD45 engine. This gap in performance likely caused or contributed to an
intermittent fueling condition that would have been reflective of the damage seen in the April
2023 failure.
Corrective Actions: As documented above, the licensee has essentially replaced 32 pistons
and supporting components (1A1 and 1A2 engines) as well as replaced all 32 injector nozzle
assemblies as of July 17, 2023. Further, the licensee took corrective actions to ensure fuel
and lube oil systems met quality specifications per vendor recommendations. Constellations
review is still on-going as it relates to evaluation of underlying maintenance and
organizational performance factors that may have contributed to the failure.
Corrective Action References: IRs 04672350, 04673770, 04688495, 04684643
Performance Assessment:
Performance Deficiency: The inspectors determined that the licensee's failure to properly
develop and implement adequate maintenance instructions and practices was a performance
deficiency reasonably within their control. Adequate instructions, guidance, and part
replacement frequencies relevant to the 1A EDG, in part, would have ensured appropriate
preventive maintenance actions for the 1A EDG, including lube oil and fuel oil systems. The
failure to perform these maintenance tasks consistent with station and vendor guidance
should have been completed, or evaluated for acceptability, during major overhauls before
the April 2023 failure. These performance deficiencies caused or contributed to an
imbalanced fuel condition on the 1A2 EDG, ultimately resulting in its failure on April 24 and
May 18, 2023.
Screening: The inspectors determined the performance deficiency was more than minor
because it was associated with the Equipment Performance attribute of the Mitigating
Systems cornerstone and adversely affected the cornerstone objective to ensure the
availability, reliability, and capability of systems that respond to initiating events to prevent
undesirable consequences. Specifically, the failure to properly implement adequate
maintenance instructions and practices contributed to the 1A2 engine damage and
mechanical failure of the 1A EDG.
Significance: The inspectors assessed the significance of the finding using IMC 0609,
Appendix A, The Significance Determination Process (SDP) for Findings At-Power. The
inspectors reviewed IMC 0609, Attachment 4, Initial Characterization of Findings, and
determined the finding affects the Mitigating Systems cornerstone. Inspectors used IMC 0609, Exhibit 2, for Mitigating Systems and determined a DRE was required since the finding
24
was a deficiency affecting the design or qualification of a mitigating system, structure, and
component that did not maintain its PRA functionality, and the degraded condition
represented a loss of the PRA function of one train of a multi-train TS system for greater than
its TS allowed outage time.
A Region I SRA performed a DRE. The finding was preliminarily determined to be of low-to-
moderate safety significance (White). The SRA also determined that the PRA exposure time
for this failure was 216 days for Unit 1 which contributed to the preliminary White risk
determination. The risk important core damage sequences were dominated by internal events
and loss of offsite power events. The dominant sequences involved loss of offsite power
events with failure of both EDGs (or variations of common cause failure), failure to align the
0C SBO EDG to supply power to the safety busses, failure to recover an EDG/declare
extended loss of AC power, and subsequent failure to recover offsite power leading to core
damage. Fire (external events) was also a major contributor and dominated by scenarios
associated with fires in the 45 level switchgear room (fire area 430). See Attachment 1,
April 24, 2023, 1A EDG Failure Detailed Risk Evaluation, for a detailed review of the
quantitative and qualitative criteria considered in the preliminary risk determination.
Cross-Cutting Aspect: H.12 - Avoid Complacency: Individuals recognize and plan for the
possibility of mistakes, latent issues, and inherent risk, even while expecting successful
outcomes. Individuals implement appropriate error reduction tools. Specifically, Constellation
did not incorporate available vendor guidance for maintenance and, instead, assumed their
own maintenance practices were adequate.
Enforcement:
Violation: Calvert Cliffs Nuclear Power Plant, Unit 1 TS 5.4.1, Procedures, requires, in part,
that written procedures shall be established, implemented, and maintained covering the
activities recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978.
Regulatory Guide 1.33, section 9, recommends procedures for performing maintenance that
can affect the performance of safety-related equipment and states it should be properly pre-
planned and performed in accordance with written procedures, documented instructions, or
drawings appropriate to the circumstances.
Contrary to this requirement, prior to April 24, 2023, and as early as 1996 (since 1A EDG
installation), Constellation failed to establish and implement appropriate procedures and
instructions for performing maintenance that can affect the performance of the safety-related
1A EDG. Specifically, the licensee did not incorporate into its maintenance procedures and
work orders recommendations from the vendor technical manual and requirements from the
station maintenance template to inspect and/or replace specified components to ensure
reliable operation of the EDG. This ultimately led to an imbalanced fueling condition and loss
of compression on the 1A EDG.
Enforcement Action: This violation is being treated as an apparent violation pending a final
significance (enforcement) determination.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
On August 31, 2023, the inspectors presented the special inspection results to Patrick
Navin, Site Vice President and other members of the licensee staff.
25
On July 27, 2023, the inspectors presented the debrief inspection results to Patrick
Navin, Site Vice President and other members of the licensee staff.
On May 5, 2023, the inspectors presented the debrief inspection results to Patrick Navin,
Site Vice President and other members of the licensee staff.
26
DOCUMENTS REVIEWED
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
93812
Corrective
Action
Documents
Corrective
Action
Documents
Resulting from
Inspection
Engineering
Evaluations23-005
1A EDG
0 23-212
0C EDG
0 23-215
SACM White Deposits
0 23-232
1A EDG Fuel pump stroke
0 23-303
0C Fuel Injector Functionality
0
CAL-1-2023-
0117
1
OP-AA-108-
Adverse Condition Monitoring
17
27
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
111
and Contingency Plan
Miscellaneous
18002-091
VTM
Diesel Generator SACM Manual
124
CCN-63536
Laboratory Inspection of 1A EDG
Cylinder Liners
06/15/2023
Maintenance
Template
SACM Diesel Generator
1
Wartsila Service Work Report
05/05/2023
Procedures
1A DG Local Control Panel Alarm
Manual
9
Control of Vendor Equipment
Manual
13
CY-CA-100-
226
Specification and Surveillance -
Diesel Fuel Oil
3
EDG-13
24 Month Inspection of SACM
Diesel Generator
17
MA-AA-716-
004
Conduct of Troubleshooting
20
OP-AA-106-
101-1001
Event Response Guidelines
33
OP-CA-108-
106-0001
Calvert Cliffs Operability and
Maintenance Testing
1
Work Orders
C120023979
C93844580
C93899648
C93909573
C93909739
C93909762
C93909782
C93909840
C93910090
C93910222
Attachment 1
ATTACHMENT 1: APRIL 24, 2023, 1A EDG FAILURE DETAILED RISK EVALUATION
The SRA evaluated the finding using the Calvert Cliffs Standardized Plant Analysis Risk (SPAR)
model, version 8.81, and System Analysis Program for Hands-On Integrated Reliability
Evaluations (SAPHIRE), version 8.2.8. This model incorporates the most recent failure
probability and initiating event data, incorporates internal events all-hazards, and models post-
Fukushima diverse and flexible coping strategies (i.e., FLEX). The SPAR model was not used to
evaluate the fire risk.
Summary of Significance Determination:
a.
Screening and Detailed Risk Evaluation (DRE), Logic Process, and Results
The inspectors evaluated the finding in accordance with IMC 0609, Appendix A, The
Significance Determination Process (SDP) for Findings At-Power, Exhibit 2. The
inspectors concluded the finding required a DRE because 1A EDG was determined
not to be able to perform its 24-hour PRA function for greater than its TS allowed
outage time. The DRE results indicate the finding has a preliminary White safety
significance.
b.
Influential Assumptions
The 1A EDG would not continue to run for its mission time based on observed
damage.
The 1A EDG was not recoverable within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> based on observed damage.
The assumed exposure time for 1A EDG functionality was based upon successful
operation of the EDG for a mission time of 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. Calculation of the exposure
time (216.3 days; at-power) was determined by summing the past successful run
times. The basis for this approach was the assumption that the impact of
inadequate maintenance on the engines resulted imbalance fueling of the engines
and subsequent piston melting, was highly runtime dependent.
The assumed Unit 2 exposure time for 1A EDG functionality was 196.3 days to
account for its 2023 refueling outage.
Consistent with the above assumption, offsite power non-recovery probabilities
were adjusted for each runtime segment to reflect the duration of successful EDG
operation over the past 24-hour runtime. Appropriate SPAR model changes were
made with the assistance of the Idaho National Laboratory staff.
SPAR model adjustments of EDG common cause failure (CCF) probability were
made to limit common cause to the 1A and 0C SACM diesels since the PD
impacted only SACM maintenance inadequacies. A change set was used to
maintain all CCF control groups to nominal, except CCF of 1A and 0C (EPS-
DGN-CF-FR1A0C = 1.380E-2)
The 1A EDG and 0C EDG are SACM EDGs with aluminum pistons. The other
EDGs (1B, 2A, and 2B) are Fairbanks Morse EDGs and not considered impacted
by the PD.
Initiating Event Loss of Offsite Power (LOOP) frequencies were updated in the
SPAR.
FLEX credit was assumed (initially deployed N equipment only) using 2022
generic industry failure data as best-available information.
A1-2
Exposure Time
The SRA reviewed Constellations condition report documenting the April 24th failure
(IR 04672350), inspection team results, and the body of information provided to the
team from the licensee, including the licensees SDP analysis for the April 24th and
May 18th (2023) failures. This documented 1A/0C engine maintenance and operation
timelines and observed damage to the 1A EDG and other 0C engine inspection results.
It also included descriptions of troubleshooting efforts and repairs/replacements made
over the 5.3-day repair window. The team determined improper maintenance resulted,
in part, to an imbalanced fueling condition and contributed to degradation of the SACM
engines. This resulted in melted/degraded pistons, displaced piston material, and
subsequent engine failure. Calvert Cliffs first experienced a similar issue (melted piston)
in 2007. Recently in 2022, another piston melt condition occurred caused by foreign
material in a fuel injector on the 1A1 engine (ADAMS Accession No. ML22320A043).
Based on these experiences, Calvert Cliffs demonstrated SACM diesels have a low
margin between piston operating temperature and the melting point of aluminum. The
0C and 1A EDGs are SACM air-cooled and have aluminum-crowned piston heads. The
Fairbanks Morse EDGs (1B, 2A, 2B) do not have aluminum-crowned pistons.
It was determined by the inspection team that the most probable cause of the 1A2
engine is inadequate maintenance leading to an imbalanced fueling condition which
resulted in subsequent engine failure. The SRA considered the degradation mechanism
to be primarily dormant while in standby and active when the engine was running. The
EDGs at Calvert Cliffs are operated approximately 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> every 30 days. A review of
plant computer logs and information from the licensee identified the exposure to lead
back to September 25, 2022, as the time when the 24-hour mission time would have
been satisfied. This was no refueling outage window for Unit 1. Hence, Unit 1 exposure
time was best represented from 1A2 engine failure (4/24/2023) back to when it could
operate for its PRA mission time (i.e., 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />) (9/25/22). This results in 210.7 days,
plus 5.6 days to repair (1A EDG made functional on 4/29/23), with a total exposure of
216.3-days. This is consistent with Risk Assessment Standardized Project (RASP)
Volume I, Section 2.5, Exposure Time for Component Run Failures, example A.
Unit 2 completed an approximate 20-day outage during the exposure window hence its
exposure was 196.3 days. (9/25/23 thru 4/29/23, minus 20 days).
SPAR Model Modifications
Consistent with RASP, Volume 1, for Component Run Failures, the SRA modified the
Unit 1 SPAR model to account for the successful EDG run times by revising the offsite
power non-recovery probabilities and was developed in consultation with Idaho
National Laboratory. The method used maintained convolution adjustments and the
potential for random EDG failures during the 24-hour mission time.
Model adjustments were made to account for diminished decay heat following
successful EDG operation. With decay heat rates lowering with time after shutdown
there are lower demands on mitigating equipment over time. To partially account for this
condition, SPAR model post-processing rules were written for each interval between
surveillance test runs. The rules were invoked for the dominant sequences which
included offsite power non-recovery events at 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />, 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />, and 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.
A1-3
The 1A EDG surveillance tests completed prior to the April 24th failure, were broken into
exposure time intervals. These intervals were then added up until the 24-hour mission
time was met. For exposure Interval 1, 1A EDG was assumed to operate for 2.2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />
and then fail. Recovery rules and fault tree logic modifications were made to credit
these 2.2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> of operation. Separate delta core damage frequency (CDF) risk
calculations were performed for each of seven intervals, comprising the cumulative
exposure time back to when 1A EDG would have been capable of completing its 24-
hour mission time (9/25/22). Each of these exposure intervals had revised post-
processing rules written to reflect adjustments to offsite power non-recovery. This was
incorporated as part of the best-estimate case.
Table 1: (At-Power) 1A EDG Run Time Intervals for Calculated 24-hour Mission Time
Interval
Dates
Duration
(days)
Runtime
(hours)
Cumulativ
e Runtime
(hours)
1
4/24/23 to 3/24/23
30.61
2.2
2.2
2
3/24/23 to 2/17/23
35.23
3.4
5.6
3
2/17/23 to 1/24/23
23.84
5.0
10.6
4
1/24/23 to 12/19/22
36.07
3.0
13.6
5
12/19/22 to 11/23/22
26.18
3.1
16.7
6
11/23/22 to 10/25/22
29.07
3.2
19.9
7
10/25/22 to 9/25/22
29.67
3.1
23.0
--
9/25/22
--
3.1
26.1
Common Cause Failure Assumption
The Calvert Cliff EDGs are of two types; 0C and 1A are SACM air-cooled; the
remaining (1B, 2A, 2B) are water-cooled Fairbanks Morse (FBM). The SPAR model has
four common cause control groups (CCCGs) and SAPHIRE uses the alpha-factor
method to automatically adjust CCF factors depending on the number and type of
failures of components in each CCCG. Below is a summary of the version 8.81 model
CCCGs and their nominal values when 1A EDG fail-to-run (FTR) is set to TRUE;
included are the values parameters used by the SRA for the Best Estimate case to limit
CCF to the SACM diesel generators as the PD directly affects SACM DGs:
Table 2: CCF Factors Related to EDGs in Calvert Cliffs Unit 1 SPAR Model 8.81
Group
Description
Nominal
Value
(No failure)
CCF Value
with 1A EDG
Failed to Run
Best Estimate
Case
(Only SACM DGs)
CCCG1
FR1AB
Unit 1 EDGs Only
(1 SACM, 1 FBM)
3.696E-04
1.380E-02
3.696E-04
CCCG2
FR1AB0C
Unit 1 EDGs + 0C
(2 SACM, 1 FBM)
1.049E-04
3.914E-03
1.049E-04
CCCG3
FR1A0C
SACM EDGs Only
(1A, 0C)
3.696E-04
1.380E-02
1.380E-02
CCCG4
ALL5FR
Unit 1 & 2 EDGs + 0C
(2 SACM, 3 FBM)
1.357E-05
5.064E-04
1.357E-05
A1-4
Updated Initiating Event (IE) Frequencies for Loop of Offsite Power (LOOP)
The model IE LOOP frequencies were updated to reflect information in Tables 16 and
17 from INL/RPT-22-68809, Analysis of Loss-of-Offsite-Power Events - 2021
Update, dated August 2022. In addition, the IE-LOOP frequency for grid-related
events was updated to NERC-Region Central East values from the same reference.
FLEX Credit
When FLEX credit is appropriate and applied, it was toggled via a change set and
increased by a factor of three to account for the quality of the failure data. In 2022,
the NRC received, and reviewed, FLEX failure information provided by industry via
the Pressurized Water Reactor Owners Group (PWROG) which is documented in
report PWROG-18042-NP, Revision 1, FLEX Equipment Data Collection and
Analysis (ADAMS Accession No. ML22123A259). This information is considered
best available and was used to update failure to start and run for various FLEX
related equipment. This was incorporated into a separate change set and adjusted for
24-hour mission time. In addition, the SRA only credited the first set of deployed
equipment (i.e., N vs N+1 equipment).
c.
Internal Risk Calculation
Unit 1 At-Power Internal Risk
The base case was set for each interval consistent with each case (FLEX and/or offsite
power adjustments). For the condition case EPS-DGN-1A-FR was set TRUE.
Best Estimate Case - 1A FTR / Limited CCF / FLEX (N-Only & PWROG Data)/ Offsite
Power Recovery (OPR) Adjustment
Sensitivity Case 1 - Best Estimate case without OPR binning (direct solving)
Sensitivity Case 2 - Best Estimate case without OPR binning (ECA module solving,
upper bound)
Table 3 - CDF/year for At-Power Exposure Period for Internal Events for 1A EDG FTR
Best Estimate
(mean)
Sensitivity Case 1
(mean)
Sensitivity Case 2
(mean)
1.88E-6
2.23E-6
3.38E-6
Table 4 - Best Estimate CDF/year (per Interval) for At-Power Internal Events (1A FTR)
Period No.
(Runtime
prior to FTR)
Duration
(days)
Base Case
(per year)
Cond Case
(per year)
Delta CDF
(per year)
Delta CDF
(per interval)
1
(2.2hours)
30.61
1.25E-05
1.58E-05
3.31E-06
2.78E-07
2
(5.6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />)
35.23
1.25E-05
1.56E-05
3.11E-06
3.00E-07
3
(10.6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />)
23.84
1.25E-05
1.55E-05
3.03E-06
1.98E-07
4
(13.6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />)
36.07
1.24E-05
1.55E-05
3.13E-06
3.09E-07
A1-5
5
(16.7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br />)
26.18
1.24E-05
1.55E-05
3.13E-06
2.25E-07
6
(19.9 hours1.041667e-4 days <br />0.0025 hours <br />1.488095e-5 weeks <br />3.4245e-6 months <br />)
29.07
1.24E-05
1.56E-05
3.21E-06
2.56E-07
7
(23.0 hours0 days <br />0 hours <br />0 weeks <br />0 months <br />)
29.67
1.25E-05
1.57E-05
3.23E-06
2.63E-07
(26.1 hours1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />)
--
--
--
--
Sum = 1.83E-6
Repair Interval Risk Increase
The repair interval is assessed from the time of failure (4/24/23) until 1A EDG was
made functional (4/29/23). The risk increase for 5.6 days was assessed similar to
Sensitivity Case 1 (no OPR binning). However, during the repair window, the station
blackout (SBO) 0C DG was pre-aligned to safety bus 11 and was credited by adjusting
the human error probability to minimum during this window. This credit resulted in only
a minor impact based on the short exposure time. The resultant repair interval risk
increase contribution in delta CDF was 5.3E-8/year.
(Best Estimate Internal Events = 1.83E-6/yr (210.7days) + 5.3E-8/yr (5.6days) = 1.88E-
6/year).
Unit 2 At-Power Internal Risk
Though the finding is assigned to Unit 1, the 1A EDG is modeled in Unit 2s SPAR. In
accordance with RASP guidance, a screening SDP was conducted on Unit 2 to
determine the limiting unit risk. The base and condition cases were established as in
Unit 1 Best Estimate case, but no offsite power recovery adjustments were made
(Sensitivity Case 1). Also, exposure was set to 196.3 days. Based on the shutdown risk
assessment for Unit 1 in 2022, the risk increase from during a 20-day outage is
estimated at 1E-7/yr. The total internal events risk increase is estimated at 3.1E-7/year
plus 1E-7/yr (S/D) = 4.1E-7/yr. This screening determined that risk from this finding
was bounded by Unit 1 risk increase.
d.
Dominant Cutsets
For internal events, the dominant sequences involve LOOP events with failure of both
EDGs (or variations of CCFs), failure to align the 0C SBO to supply power to the
safety busses, failure to recover an EDG/declare extended loss of AC power and
subsequent failure to recover offsite power leading to core damage.
For external events, a dominant scenario involves fires associated with the 45 level
switchgear room (fire area 430). The dominant sequence involves the loss of
component cooling water and failure of operators to trip reactor coolant pumps
resulting in a reactor coolant pump seal loss of coolant accident.
e.
Risk-Insights
Events resulting in a LOOP constitute most of the risk associated with EDG
operations. The Calvert Cliff units are highly cross-tied and capable of flexible
alignments to support the other unit. The station does have an SBO (0C) EDG that is
credited as a backup AC power source which is very reliable (though impacted by the
PD) and does result in a significant reduction in overall risk since it can be aligned to
any of the units safety busses. Since the 0C EDG is a SACM EDG and part of the
A1-6
directly affected CCCG that includes the 1A and 0C EDGs; results are sensitive to its
CCF increase, and more importantly, the basic events related to its alignment and/or
recovery. The flexibility of the 0C SBO EDG alignment does affect internal risk to both
units and reflected in the CCF increase when any of the EDGs fail.
f.
Uncertainty and Sensitivity Studies
The SRA reviewed and evaluated sensitivities related to increases in CCFs for only
the SACM EDGs versus all EDGs, FLEX credit, and the reduction of offsite power
non-recovery failure as the 1A EDG culminated 24-hour runtime as documented in
the 2022 DRE (ADAMS Accession No. ML22320A043). Those sensitivities were not
repeated here but assisted in establishing the current Best Estimate case parameter.
Two additional sensitivities were performed to assess the impact of offsite power
recovery binning. These sensitivities highlighted a reduction of risk against the best
estimate case.
Sensitivity Case 1 - No OPR Binning with Direct Solving
This case determined that the risk results are sensitive to OPR binning. A 19 percent
increase in risk was noted without binning. This case was done with the direct solve
method used in the best estimate case using the modified SPAR that supports
binning/OPR shifting. The sensitivity case indicates that with the without binning
applied, the risk would remain White risk significance.
Sensitivity Case 2 - No OPR Binning using ECA module
This case determined that the risk results are sensitive to OPR binning and method.
A 34 percent increase in risk was noted from sensitivity case 1. This case was done
to assess the upper bound estimator for internal events. The sensitivity case
indicates that with the without binning applied using the ECA module method, the risk
would remain White risk significance.
g.
Contributions from External Events (Fire, Flood, and Seismic)
External fire risk was determined to be the dominant external risk contributor for this
performance deficiency. High winds, tornados, hurricanes, and seismic were also
evaluated. Only hurricane contribution was noteworthy but minor in comparison to fire
risk.
Fire Risk Contribution
The SPAR model for Calvert Cliffs does not include fire (external) events. Constellation
used their Fire PRA to estimate risk for their 2023 updated 1A EDG FTR SDP risk
evaluation. The analyst reviewed the Fire PRA results from their Fire model
CC020DFire. The model included several conservatisms the licensee identified during
their 2022 SDP review. Some items had been updated but other conservatisms
remained until further work is completed onsite. Constellation did indicate a 24 percent
reduction in Unit 1 fire risk since 2022 because of updated fire area modeling. The
dominant fire areas that contributed the most were 45 switchgear room (430) and 27
cable spreading room (306) area.
The analyst performed a sensitivity check on the two dominant fire scenarios using the
SPAR model internal events as a surrogate for the events. The analyst compared this
to the Calvert Cliffs fire model result cutsets for the 1A EDG failure to run. The analyst
A1-7
noted the resulting contribution was close to the licensees Fire PRA estimate for the
increase in contribution due to this fire scenario. The analyst did not perform the same
assessment for Unit 2.
Based on the above reviews, the analyst considered the use of the licensees
calculated annual change in CDF, given the 1A EDG failure, to appropriately represent
a conservative increase in fire risk. The annual change in CDF calculated by
Constellation with their fire model was 6.91E-6/yr (Unit 1) and 4.35E-6/yr (Unit 2; not
updated for 2023) for the 1A EDG failure. Using the licensees annual fire risk values for
the unit-specific exposure time results in increase due to fire risk:
Unit 1 = 6.91E-6/yr x 216.3/365 = 4.1E-6/yr
Unit 2 = 4.35E-6/yr x 196.3/365 = 2.3E-6/yr
Flooding
The analyst reviewed the Calvert Cliffs Individual Plant Examination External Events
and did not consider flooding to be a significant risk contributor for this performance
deficiency.
Seismic and High Winds (including Tornado and Hurricane)
The Calvert Cliffs SPAR models are an all-hazards models and can estimate seismic
and high winds. Below is a summary:
Unit 1 Wind/Tor/Hur: 3.2E-7/yr
Unit 2 Wind/Tor/Hur: 1.5E-8/yr
Unit 1 Seismic: 8.2E-8/yr
Unit 2 Seismic: 4.3E-9/yr
h.
Potential Risk Contribution Due to Large Early Release Frequency (LERF)
The SRA used insights from IMC 0609, Appendix H, Containment Integrity
Significance Determination Process, to evaluate the estimated change in LERF
associated with this finding. The failure of 1A EDG to meet its 24-hour mission time
was classified as a Type A finding per Appendix H. The SRA screened out LERF based
on SAPHIRE LERF estimation <1E-7 based zero delta CDF for events with non-zero
LERF factors. However, Calvert Cliffs is a Combustion Engineering plant and hence
requires a consequential steam generator tube rupture assessment per IMC 0609,
Appendix H, Section 5. With guidance from NUREG-2195, the analyst estimated
consequential steam generator tube rupture (C-SGTR) at Unit 1 at (3.87E-7/yr) x
216.3/365 = 2.3E-7/yr (for 216.3 days).
Constellations estimated LERF (Unit 1) = 4E-7/yr, and (Unit 2) = 2E-7/yr. Both NRC
and licensee estimated values are consistent with a White risk range based on LERF.
i.
Total Estimated Change in Core Damage Frequency (CDF) and Large Early
Release Frequency (LERF) and Qualitative Risk Considerations
The best estimate total change in Unit 1 CDF is 6E-6/yr - 8E-6/yr, (preliminary White).
The estimated change in Unit 1 LERF is 2E-7/yr, (preliminary White).
(Consequential Steam Generator Tube Rupture)
A1-8
Table 5 - Risk Summary for April 24, 2023, 1A EDG FTR
Best Estimate
Sensitivity Case 1
Sensitivity Case 2
Internal Risk
1.9E-6/yr
2.2E-6/yr
3.4E-6/yr
Ext. Risk from Winds
3.2E-7/yr
3.2E-7/yr
3.2E-7/yr
Ext. Risk from Seismic
8.2E-8/yr
8.2E-8/yr
8.2E-8/yr
Ext. Risk from Fire
4.1E-6/yr
4.1E-6/yr
4.1E-6/yr
Total Risk for Unit 1
6.4E-6/yr
6.7E-6/yr
7.9E-6/yr
Qualitative Risk Considerations
Offsite power non-recovery adjustments were made for a partial set of dominant
event tree top events (1-hour, 2-hour, 4-hour, 6-hour only). A full 24-hour offsite
power non-recovery reduction was not implemented. Only the hours in the
dominant sequence top events (LOOP and SBO). Making these adjustments would
lower risk estimate.
No other adjustments to mitigating systems to account for EDG runtime were
made.
A review of the licensees fire risk evaluation indicated several remaining
conservative assumptions regarding zone of influence and cable damage
assumption that would further reduce the fire risk contribution.
The licensees fire analysis did not include EDG runtime binning (or comparable)
method, which would reduce the fire risk further.
During risk result discussions with licensee PRA staff three SPAR modeling
enhancements were discussed as it pertains to 1) cross-unit 125VDC systems,
2) late LOCA (including RCP LOCA) sequence re-aligned of equipment, and 3)
opposite unit auxiliary feedwater crosstie credit. Resolution of these issues would
potentially lower the risk estimate.
j.
Licensees Risk Evaluation
Constellation modeled this event as a failure to run of the 1A EDG and determined
exposure time based on its history of past (24-hour) runtime capability, similar to the
NRC, and documented their SDP evaluation of both Units 1 and 2 in CA-SDP-006,
Revision 0. Revision 1 was issued to correct the NRC concern that the outage window
was applied to the incorrect unit. The licensee does not include repair time in their
analysis since it considers pre-alignment of the 0C to the safety-bus as functionally
equivalent. The licensee used Calvert Cliffs Units 1 and 2 (National Fire Protection
Association-805 Plant) Fire PRA (CC020DFire) as part of their assessment. The
licensee used a 210.7 day exposure for Unit 1 and 192.6 day for Unit 2.
Constellation made various changes within their application specific model to represent
the plant alignment during the event, including other mitigating system adjustments to
account for a successful 1A EDG runtime. Constellation considered CCF only between
the SACM EDGs similar to the NRC. Also, human error probabilities for 0C SBO
realignment (to alternate busses) and component cooling water heat exchanger
realignments were reassessed based on additional available time gained by at least 1
A1-9
hour of 1A EDG runtime. The SRA did not incorporate any specific licensee model
adjustment since the SRA application of partial offsite power non-recovery adjustment
was reasonably representative of the licensee adjustments. The licensee Internal
Events result increase for their exposure time was:
Unit 1 CDF = 8.1E-7/yr (210.7 days); Unit 2 CDF = 4.7E-7/yr (192.6 days).
During fire area walkdowns in 2022 the licensee identified additional conservatisms that
could be adjusted in the model related to various cable routing and fire damage
assumptions. Several fire areas had updated modeling, but the majority have yet to be
addressed and remain conservative and not incorporated in the licensees SDP;
additional follow-up and review were needed by the licensee and remain in Calvert
Cliffs PRA configuration control database (URE-DB). The SRA considered the
licensees fire risk result a realistic bounding estimate and incorporated the licensees
result into the NRCs total risk result. The licensee fire risk result increase for 1 year,
Unit 1 CDF = 6.91E-6/yr; Unit 2 CDF = 4.36E-6/yr, which results in Unit 1 CDF = 4.0E-
6/yr (210.7 days); Unit 2 CDF = 2.3E-6/yr (192.6 days).
Seismic, wind, and LERF impacts were determined qualitatively and considered to not
significantly change the findings of their result. Licensee did not assess potential
shutdown risk on Unit 1.
The total increase in risk estimate determined by licensee:
Unit 1 CDF = 4.8E-6/yr, LERF = 4.1E-7/yr;
Unit 2 CDF = 2.8E-6/yr, LERF = 2.1E-7/yr
The licensees risk increase values are also consistent with a preliminary White safety
significance.
k.
Summary of Results and Impact
The NRCs preliminary quantitative risk increase assessment (internal and external
CDF contributions) for this finding was determined to be in the range of 6E-6/yr - 8E-
6/yr, or of low to moderate safety significance, (preliminary White). Sensitivity analyses
demonstrated a high confidence in this quantitative risk estimate. Constellations value
was calculated to be 5E-6/yr (low to moderate safety significance). The SRA based the
preliminary risk evaluation of Unit 1 on the exposure time assessed by the cumulative
EDG run time for its 24-hour mission time.
Attachment 2
ATTACHMENT 2: MAY 24, 2023, 1A EDG FAILURE DETAILED RISK EVALUATION
The SRA evaluated the finding using the Calvert Cliffs SPAR consistent with, and references,
assumptions and methods used in Attachment 1 for the April 24th 1A EDG failure; however, no
EDG runtime binning was performed. This is appropriate since the 1A EDG had not
accumulated 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> of runtime since its restoration from the April 24th repair. Specific
considerations were given as documented below with respect to the separate performance
deficiency.
Summary of Significance Determination:
a.
Screening and Detailed Risk Evaluation (DRE), Logic Process, and Results
The inspectors evaluated the finding in accordance with IMC 0609, Appendix A, The
Significance Determination Process (SDP) for Findings At-Power, Exhibit 2. The
inspectors concluded the finding required a DRE because 1A EDG was determined
not to be able to perform its 24-hour PRA function for greater than its TS allowed
outage time. The finding was determined to be of very low safety significance
(Green).
b.
Influential Assumptions (same as April 24th Failure DRE in Attachment 1, except as
noted below)
The 1A EDG would not continue to run for its mission time based on main lube oil
system failure of the 1A2 engine.
The assumed exposure time for 1A EDG functionality was based upon the time
when the EDG was last made functional (4/29/2023) with the latent degraded
condition until the failure on 5/18/2023.
Repair time is not negligible.
The repeat failure is not caused by the first repair.
No SPAR modifications were made.
Unit 1 represents the dominant risk (Unit 2 will not be assessed)
Exposure Time
The SRA reviewed Constellations condition report documenting the May 18th failure (IR
04678953), inspection team results, and the body of information provided to the team
from the licensee, including the licensees SDP analysis for the April 24th and May 18th
(2023) failures.
The 1A2 engine lube system failure (lo-lo pressure) was a result of a sheared shaft on
one of the lube oil pumps caused by blocked cooling oil ports. The SRA considered the
degradation mechanism to be primarily dormant while in standby and active when the
engine was running. The EDG was made functional on 4/29/2023 (inception date) and
accrued approximately 13 hours1.50463e-4 days <br />0.00361 hours <br />2.149471e-5 weeks <br />4.9465e-6 months <br /> of post-maintenance testing runtime before being
declared operable on 5/2/2023. The EDG ran loaded approximately 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> before
failure on 5/18/2023. The SRA reasonably assessed repair time at 4.2 days based on
work order reviews and team discussion as additional maintenance was performed in
addition to the lube pump replacements.
A2-2
Hence, exposure time was best represented from 4/29/2023 to 5/18/2023 (18.4 days)
plus repair (4.2 days); 22.6 days total. This is consistent with RASP Volume I,
Section 2.5, Exposure Time for Component Run Failures, example B.
c.
Internal Risk Calculation
Unit 1 At-Power Internal Risk
The base case and condition used change sets as noted. For the condition case EPS-
DGN-1A-FR was set TRUE.
Best Estimate Case - 1A FTR / Limited CCF / FLEX (N-Only & PWROG Data)
Sensitivity Case 1 - Best Estimate with extended exposure time (28.1 days)
Table 1 - CDF/year for At-Power Exposure Period for Internal Events for 1A EDG FTR
Best Estimate
(mean)
Sensitivity Case 1
(mean)
3.5E-7
4.3E-7
d.
Uncertainty and Sensitivity Studies
Sensitivity Case 1 - Best Estimate with extended exposure time (28.1 days)
This case was assessed to determined impact of establishing the inception date at
time of 1st failure (4/24/2023) versus the return to functionality date (4/29/2023). The
sensitivity indicates that with the extended exposure and taking qualitative aspects
into account the total risk would remain Green significance.
e.
Contributions from External Events (Fire, Flood, and Seismic)
External fire risk was determined to be the dominant external risk contributor for this
performance deficiency. High winds, tornados, hurricanes, and seismic were also
evaluated. Only hurricane contribution was noteworthy but minor in comparison to fire
risk. Using the external risk inputs from the April 24th Failure DRE, the risk
contribution from the May 18th failure are as follows:
Fire: 6.91E-6/yr x 22.6/365 = 4.3E-7/yr
Seismic: 1.38E-7 x 22.6/365 = 8.5E-9/yr
High Winds (including Tornado / Hurricane): 4.16E-7 x 22.6/365 = 2.6E-8/yr
f.
Potential Risk Contribution Due to Large Early Release Frequency (LERF)
The SRA used IMC 0609, Appendix H, Containment Integrity Significance
Determination Process, to evaluate the estimated change in LERF associated with this
finding. The failure of 1A EDG to meet its 24-hour mission time was classified as a
Type A finding per Appendix H. The SRA screened out LERF based on SAPHIRE
LERF estimation <1E-7 based zero delta CDF for events with non-zero LERF factors.
With guidance from NUREG-2195, the analyst estimated C_SGTR at Unit 1:
(3.87E-7/yr) x 22.6/365 = 2.4E-8/yr (for 22.6 days).
A2-3
Constellations estimated LERF (Unit 1) = 3.5E-8/yr. Both NRC and licensee estimated
values are consistent with Green risk based on LERF.
g.
Total Estimated Change in Core Damage Frequency (CDF) and LERF and
Qualitative Risk Considerations
The best estimate total change in Unit 1 CDF is 8.1E-7, (Green).
The estimated change in Unit 1 LERF is 2E-8/yr, (Green). (C-SGTR)
Table 2 - Risk Summary for May 18, 2023 1A EDG FTR
Best Estimate
Internal Risk
3.5E-7/yr
Ext. Risk from Winds
2.6E-8/yr
Ext. Risk from Seismic
8.5E-9/yr
Ext. Risk from Fire
4.3E-7/yr
Total Risk for Unit 1
8.1E-7/yr
Qualitative Risk Considerations (same as April 24th failure DRE, including below)
The 1A EDG successfully operated for approximately 18.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> prior to failure (13
hours during functional runs and 5.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> during a loaded surveillance run). This
demonstrated EDG runtime was not accounted for in this DRE and would further
reduce risk further as the EDG would have supported safety-related loads for that
period.
h.
Licensees Risk Evaluation
Constellation modeled this event as a failure to run of the 1A EDG and embedded
there SDP in the same document as the April 24th failure (CA-SDP-006, Revision 0.
Revision 1). Constellation used an 18.1 day exposure time (no repair time) since it
considers pre-alignment of the 0C to the safety-bus as functionally equivalent.
The total increase in risk estimate determined by licensee:
Unit 1 CDF = 4.2E-7/yr, LERF = 3.5E-8/yr;
Unit 2 CDF = 2.7E-7/yr, LERF = 2.0E-8/yr
The licensees risk increase values are consistent with Green risk significance.
i.
Summary of Results and Impact
The NRCs quantitative risk increase assessment (internal and external CDF
contributions) for this finding was determined to be 8.1E-7/yr, or very low safety
significance, (Green).
Attachment 3
ATTACHMENT 3 - SPECIAL INSPECTION TEAM CHARTER
MEMORANDUM TO J. DEBOER FROM B. WELLING
CALVERT CLIFFS NUCLEAR POWER PLANT, UNIT 1 EMERGENCY DIESEL GENERATOR
FAILURE, REVISION 1
MAY 30, 2023
This special inspection team is chartered to assess the circumstances surrounding the 1A
emergency diesel generator (EDG) failure that occurred at Calvert Cliffs, Unit 1 on April 24,
2023. The special inspection will be conducted in accordance with Inspection Procedure 93812,
Special Inspection. The Inspection team will conduct an entrance and exit meeting and
document the inspection findings and conclusions in a Special Inspection Team final report
within 45 days of inspection completion. Revision 1 is issued to incorporate the May 18, 2023,
trip of the 1A EDG on LO-LO lube oil pressure.
A. Basis
Calvert Cliffs has five total EDGs. Unit 1 has a tandem-engine Societe Alsacienne de
Constructions Mecaniques de Mulhouse (SACM) EDG that consists of two diesel engines
(1A1 and 1A2) connected to a common generator and one Fairbanks-Morse opposable
piston EDG. Unit 2 has two Fairbank-Morse Diesels (2A and 2B). The station blackout diesel
(0C) is a SACM which can be manually lined up to either units vital buses.
On April 24, 2023, during the monthly surveillance run of the 1A SACM diesel generator at
Calvert Cliffs, Unit 1, the 1A2 diesel engine developed a significant lube oil leak.
Approximately 1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into the 2-hour run, the 1A2 exhaust gas temperature hi alarm
came in. A significant amount of white smoke was observed coming from the engine, and
operators witnessed lube oil spraying from the engine. The lube oil filter high differential
pressure alarm was received as well. During manual emergency shutdown of the engine,
significant knocking sound was reported as being heard from the engine.
The 1A EDG was declared inoperable. Unit 1 entered the applicable technical specification
action statement for one EDG out of service. As of April 26, 2023, during troubleshooting
activities thus far, the licensee found that the 1A2 (1B piston and cylinder liner) were found
damaged with an appearance that indicated degradation/loss of metal on the
piston/liner. Lube oil samples contained aluminum, iron, and copper further indicating
damage of the piston/liner. Further, Constellation identified similar debris in the crankcase
volume.
The same 1A EDG experienced a similar failure in February 2022, when the 1A EDG tripped
on 1A1 engine high crankcase pressure after the high exhaust gas temperature alarm came
in. Large amounts of lube oil were reported coming from a 1A1 engine block relief valve as
well as smoke from the engine. Constellation disassembled the injector and discovered
three pieces of foreign material within the injector nozzle. That failure was the subject of the
White Finding documented in 3Q2022 with an expected supplemental inspection in June
2023.
Both failures would have prevented recovery of the 1A EDG in an event.
A3-2
B. Scope
The team is expected to perform information gathering and fact-finding in order to
address the following areas:
1. Develop a sequence of events of the 1A EDG failure to include follow-up actions taken
by Constellation. The review should consider licensee-identified timelines and
applicable computer data and logs, as applicable.
2. Evaluate the adequacy of operator response to the 1A EDG failure. This should
include a review of abnormal and alarm response procedure adherence and
technical specification compliance.
3. Evaluate Constellations identification of the failure mode and the troubleshooting
approach and activities that supports the stations understanding and confidence in
determination of the direct cause.
4. Evaluate Constellations prompt and long-term corrective actions
planned/implemented to address the failure. This review should include an
assessment of the adequacy of repair and testing activities to restore 1A EDG
operability.
5. Evaluate Constellations evaluation of extent-of-condition as it relates to the
other EDGs, including the station blackout diesel generator.
6. Evaluate Constellations causal determinations including the stations review of
relevant plant-specific and industry operating experience. This review should include
an assessment of whether performance issues, both direct and causal, are similar or
reflective of performance issues that contributed to the February 2022 1A EDG failure.
7. Evaluate Constellations monitoring and maintenance of the 1A EDG systems
performance information including system health reports, maintenance history,
and corrective action program effectiveness for possible trends and overall
timeliness of evaluating associated failures, degradations, and deficiencies.
8. Evaluate Constellations assessment of the risk significance of the degraded
condition including evaluation of input assumptions and independently evaluate the
risk.
9. Review the circumstances associated with the 1A EDG failure to identify potential
common failure modes and generic safety concerns.
10. Evaluate whether circumstances, to include refinement of the risk analysis, warrant
an escalation of the special inspection to an augmented inspection after the first
week of on-site information gathering and inspection.
11. Review the circumstances surrounding the May 18, 2023, LO-LO lube oil pressure
trip on the 1A EDG to identify any association to the 1A EDG failure on April 24,
2023, and related maintenance and test activities.
A3-3
C. Team Members
-
Joe DeBoer, Team Leader
-
Brian Dyke, Team Member
-
Rodney Clagg, Team Member
-
Eugene DiPaolo, Senior Resident
-
Anthony Tran, Resident Inspector
-
Carey Bickett, Senior Reactor Analyst
-
Dave Werkheiser, Senior Reactor Analyst
D. Guidance
This memorandum designates you as the special inspection team leader. Your duties will be
as described in Inspection Procedure 93812, "Special Inspection." The team composition
has been discussed with you directly. During performance of the special inspection activities
assigned to them, designated team members are separated from their normal duties and
report directly to you. The team is to emphasize fact finding in its review of the
circumstances surrounding the event, and it is not the responsibility of the team to examine
the regulatory process. Safety concerns identified that are not directly related to the event
should be reported to the Region I office for appropriate action.
You should notify the licensee and the team should begin inspection activities on or before
May 1, 2023, based on the licensees schedule of activities. You should conduct an
entrance meeting with the licensee at the appropriate time at the site. A report documenting
the results of the inspection, including findings and conclusions, should be issued within 45
days of the exit meeting conducted at the completion of the inspection. While the team is
active, you will provide periodic status briefings to Region I management.
This Charter may be modified should the team develop significant new information that
warrants review. Should you have any questions concerning this Charter, contact Brice
Bickett, Team Manager, Division of Operating Reactor Safety, at (610) 337-5312.