Text

Special Inspection Reactive Report 05000346/2021050 and Apparent Violation
	ML21321A365
Person / Time
Site:	Davis Besse
Issue date:	11/19/2021
From:	Hayes M; Division of Reactor Safety III
To:	Tony Brown; Energy Harbor Nuclear Corp
References
	IR 2021050
	Download: ML21321A365 (65)
	v • d • e

November 19, 2021

SUBJECT:

DAVIS-BESSE NUCLEAR POWER STATION - SPECIAL INSPECTION REACTIVE REPORT 05000346/2021050 AND APPARENT VIOLATION

Dear Mr. Brown:

On July 22, 2021, the U.S. Nuclear Regulatory Commission (NRC) completed its initial assessment of multiple diesel generator failures that took place from July 2019 to June 2021, as well as a reactor trip with multiple complicating equipment issues, which occurred on July 8, 2021, at Davis-Besse Nuclear Power Station. Based on this initial assessment, the NRC sent an inspection team to your site on July 26, 2021.

On October 7, 2021, the NRC completed its special inspection and discussed the results of this inspection with you and other members of your staff. The results of this inspection are documented in the enclosed report.

The results of the inspection concluded there were five findings. Of the five, two are pending while undergoing additional NRC reviews to assess the safety significance of the performance deficiency. A preliminary determination of safety significance for both issues is in progress and is expected to be issued soon. The other three findings have been determined to be of very low safety significance (Green) per the NRCs significance determination process. Two of these findings have associated violations, which we are treating as non-cited violations (NCVs)

consistent with Section 2.3.2 of the Enforcement Policy.

Section 93812 of the enclosed report discusses two findings with associated apparent violations for which the NRC has not yet reached a preliminary significance determination. These findings involved the apparent failure of licensee personnel to inspect the Emergency Diesel Generator Field Flash Selector Switch and the apparent failure to install an Emergency Diesel Generator Speed Switch that was suitable to the application. Since the NRC has not made a final determination in these matters, Notices of Violation are not being issued for these inspection findings at this time. In addition, please be advised that the characterization of the apparent violations described in the enclosed inspection report may change as a result of further NRC Review.

We intend to issue our final safety significance determinations and enforcement decisions, in writing, within 90 days from the date of this letter. The NRCs significance determination process (SDP) is designed to encourage an open dialogue between your staff and the NRC; however, neither the dialogue nor the written information you provide should affect the timeliness of our final determination. We are currently evaluating the significance of these findings and will notify you in a separate correspondence once we have completed our preliminary significance review. You will be given an opportunity to provide additional information prior to our final significance determinations unless our review concludes that these findings are of very low safety significance (i.e., Green).

If you contest the violations or the significance or severity of the violations documented in this inspection report, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region III; the Director, Office of Enforcement; and the NRC Resident Inspector at Davis-Besse Nuclear Power Station.

If you disagree with a cross-cutting aspect assignment or a finding not associated with a regulatory requirement in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region III; and the NRC Resident Inspector at Davis-Besse Nuclear Power Station.

This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding.

Sincerely, Signed by Hayes, Michelle on 11/19/21 Michelle W. Hayes, Acting Deputy Division Director Division of Reactor Safety Docket No. 05000346 License No. NPF-3

Enclosure:

As stated

Inspection Report

Docket Number: 05000346 License Number: NPF-3 Report Number: 05000346/2021050 Enterprise Identifier: I-2021-050-0000 Licensee: Energy Harbor Nuclear Corporation Facility: Davis-Besse Nuclear Power Station Location: Oak Harbor, OH Inspection Dates: July 26, 2021 to October 7, 2021 Inspectors: K. Barclay, Reactor Inspector D. Kimble, Senior Resident Inspector K. Miller, Senior Electrical Engineer D. Reeser, Operations Engineer L. Rodriguez, Operations Engineer J. Winslow, Reactor Systems Engineer Approved By: Laura C. Kozak, Acting Branch Chief Engineering Branch 3 Division of Reactor Safety Enclosure

SUMMARY

The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a special inspection at Davis-Besse Nuclear Power Station, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.

List of Findings and Violations

Failure to Appropriately Classify the Digital Electrohydraulic Control Uninterruptible Power Supply Battery Bank Cornerstone Significance Cross-Cutting Report Aspect Section Initiating Events Green [H.3] - Change 93812 FIN 05000346/2021050-01 Management Open/Closed A self-revealed Green finding of very low safety significance was identified for the licensees failure to appropriately classify the digital electrohydraulic Control (DEHC) uninterruptible power supply (UPS) battery bank as Non-Critical as required by component classification procedures NOP-ER-1001 and NOBP-ER-3901. Specifically, when the licensee decided to use the DEHC UPS batteries as the sole source of control power for the automatic voltage regulator (AVR) automatic transfer switch (ATS), they failed to upgrade the classification of the battery bank from Non-Impact to Non-Critical as required by the component classification procedures, since the battery banks failure could now induce a critical component (e.g.,

the AVR) failure as was seen during the July 8, 2021, reactor trip.

Failure to Provide Procedural Instructions for Transferring Gland Sealing Steam from Main Steam to Auxiliary Steam Following a Reactor Trip Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green [H.11] - 93812 Systems NCV 05000346/2021050-02 Challenge the Open/Closed Unknown A self-revealed finding of very low safety significance (Green) and an associated NCV of Technical Specification (TS) 5.4.1(a) were identified for the licensees failure to establish and implement procedural guidance for transferring the gland sealing steam supply from the main steam system to the auxiliary steam system following a reactor trip. Specifically, the guidance in licensee procedure DB-OP-06205, Turbine Generator and Main Feedwater Pump Turbine Gland Steam and Turbine Drains, for transferring the gland steam header supply from the main steam header to the auxiliary steam header did not provide instructions to ensure that the pressure in the gland steam header would be maintained low enough so as not to produce an excessive steam demand on the main steam system during the transfer. This lack of procedural guidance allowed control room operators to fully open valve GS2385, Steam Seal Feed Bypass, pressurizing the gland steam header above the normal control range, which led to an excessive steam demand on the main steam system, a low-pressure condition in SG No. 2, and a steam and feedwater rupture control system (SFRCS)actuation.

Failure to Install Correct Part for Limit Switch ZS101B Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green None 93812 Systems NCV 05000346/2021050-03 Open/Closed A self-revealed Green finding of very low safety significance and an associated non-cited violation (NCV) of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, were identified for the licensees failure to have a procedure of a type appropriate to the circumstances for the replacement of Main Steam Line No. 1 Isolation Valve Limit Switch ZS101B. Specifically, in 2008, the licensee failed to include appropriate instructions in safety-related Work Order (WO) 200205458 to ensure Limit Switch ZS101B was replaced with the correct part. As a result, the wrong part was used when replacing Limit Switch ZS101B.

Failure to Inspect Emergency Diesel Generator Selector Switch Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Pending None 93812 Systems AV 05000346/2021050-04 Open On May 27, 2021, a self-revealed finding with its safety significance as yet to be determined (TBD) and an associated Apparent Violation (AV) of TS 5.4.1.a were identified for the licensees apparent failure to develop a preventive maintenance schedule for the inspection of emergency diesel generator (EDG) field flash selector switches (FFSSs).

Failure to Install Emergency Diesel Generator Parts That Were Suitable to the Application Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Pending [H.6] - Design 93812 Systems AV 05000346/2021050-05 Margins Open On September 4, 2020, a self-revealed finding with its safety significance as yet TBD and an associated AV of 10 CFR Part 50, Appendix B, Criterion III, Design Control, and TS 3.8.1,

Conditions B.1 and B.3, were identified for the licensees apparent failure to select an EDG speed switch replacement that was suitable to the application. This resulted in the unavailability and inoperability of the Division 2 EDG when it was being relied upon for plant safety.

Additional Tracking Items

Type Issue Number Title Report Section Status URI 05000346/2021050-06 Potential Adverse Changes 93812 Open During 2006 Emergency Diesel Generator Modifications URI 05000346/2021050-07 Potential High Emergency 93812 Open Diesel Generator Field Currents When Adding Manual Loads

INSPECTION SCOPES

Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.

Starting on March 20, 2020, in response to the National Emergency declared by the President of the United States on the public health risks of the coronavirus (COVID-19), inspectors were directed to begin telework. In addition, regional baseline inspections were evaluated to determine if all or a portion of the objectives and requirements stated in the IP could be performed remotely. If the inspections could be performed remotely, they were conducted per the applicable IP. In some cases, portions of an IP were completed remotely and on site. The inspections documented below met the objectives and requirements for completion of the IP.

OTHER ACTIVITIES

- TEMPORARY INSTRUCTIONS, INFREQUENT AND ABNORMAL 93812 - Special Inspection Team In accordance with the attached Special Inspection Team (SIT) Charter, the Team conducted a detailed review of five diesel generator failures that have occurred at Davis-Besse Nuclear Power Station (hereafter referred to as Davis-Besse) within the last two years. Additionally, the Team also conducted a detailed review of the complicated reactor/plant trip that occurred on July 8, 2021. As detailed in the SIT Charter, the following items were reviewed:

Diesel Generator Failures

1. Identify a timeline for all diesel failures within the last two years (including the station

blackout generators). Include relevant and major plant conditions, system lineups, and operator actions in response to the failures, and any applicable maintenance or component replacements prior to each failure.

2. Review licensees troubleshooting efforts and follow-up evaluations for each of these

failures to confirm the adequacy of the licensees assessments including extent-of-condition.

3. Review identified failed components for acceptability of design, including environmental

conditions and appropriate surge protection, where applicable.

4. Assess the licensees operability, functionally and/or availability assessment associated with

the Division 1 EDG June 24, 2021, failure, focusing on the resulting high voltage condition and its impact on the availability of the diesel-powered components.

5. Considering the cause information from the March 13, 2020, failure of the Division 1 EDG to

start during the integrated safety features actuation system (SFAS) test, evaluate the licensees past operability and functionality assessment of the Division 1 Diesel Generator following the semiannual fast start test failure on May 27, 2021. This assessment should focus on the start circuitry for potential common issues that could have caused these failures.

6. Assess the licensees evaluation and response to all applicable Part 21 reports.

7. Assess the licensee's monitoring of the diesel generator systems performance information

including the licensees system health reports, corrective action program for possible trends and overall timeliness and effectiveness of evaluating associated failures, degradations, and deficiencies.

8. Perform an independent search of a sample of industry information associated with the

sites diesel generators and validate that the licensee reviewed the issues and that their reviews were adequate.

9. Promptly identify and convey potential generic safety concerns to regional management and

NRR/DRO/IOEB when appropriate, who will initiate appropriate follow-up actions.

Complicated Trip 10. Identify a timeline for the significant events associated with the July 8, 2021, complicated reactor trip, including the initial plant conditions, the relevant activities on-going prior to the trip, and the significant alarms and indication, operator actions and equipment major performance until the plant was placed in a stable condition.

11. Review and evaluate equipment performance during the event with particular emphasis in the following areas:

a. Atmospheric Vent Valve Operation after Main Steam Isolation Valve (MSIV) closure; b. Start Up Feedwater Valves SP7A and SP7B; and c. Uninterruptable Power Supply failure.

12. Review and evaluate operator response during the event with particular emphasis in the following areas:

a. Conditions that resulted in SFRCS actuation and MSIV closure; and b. Response to initial overcooling event.

INSPECTION RESULTS

Assessment 93812 Charter Item No. 1 Timeline of Diesel Generator Failures Summary From July 3, 2019, to present, Davis-Besse experienced five failures of components on their diesel generators. The site has two safety-related EDGs and one non-safety-related station blackout (SBO) diesel generator that are all General Motors, Electro-Motive Division models.

In four cases, the safety-related EDGs were rendered inoperable due to the component failures.

The first component failure occurred on July 3, 2019, when the licensee detected an electrical ground on the SBO Diesel Generator control circuit. The licensees troubleshooting determined that the electrical ground was associated with a failed speed switch. The failed component was replaced on July 11, 2019, and tested satisfactorily on July 12, 2019.

The second component failure occurred on March 13, 2020, when during integrated SFAS testing, the EDG No. 1 output breaker did not close; operators performed an emergency shutdown of EDG No. 1 and declared the EDG inoperable. The most likely cause was determined to be a failed flyback diode across the K2 relay which caused the field not to flash. The diode was replaced and EDG No. 1 was retested satisfactorily on March 15, 2020, and declared operable.

On September 4, 2020, the third component failure was discovered when EDG No. 2 failed to start and experienced a lockout during a scheduled monthly surveillance test. EDG No. 2 was declared inoperable. The licensee determined the issue to be due to a failed speed switch. The speed switch was replaced and EDG No. 2 was declared operable after testing on September 7, 2020. Additional licensee analysis determined that the speed switch failure occurred on September 1, 2020, due to grounds identified on the licensees direct current (DC) system. The Teams review of the station's ground readings found that a hard positive ground came in on August 25, 2020, but shifted between log readings on August 28th at 8:00 p.m. and August 29th at 8:00 a.m., to a hard neutral ground or a reduced positive ground. The Team concluded that the second hard ground (neutral bus) came in between log readings and caused thermal damage to components on the speed switch.

The fourth component failure occurred on May 27, 2021, when a scheduled fast start of EDG No. 1 for a 184-day surveillance test was unsuccessful. The EDG failed to reach steady state voltage and frequency. The licensee identified that the FFSS that selected between 800 rpm (for an idle EDG start) and 400 rpm (for a fast EDG start) was making intermittent contact when the 400 rpm position was selected. The FFSS was replaced and EDG No. 1 was retested satisfactorily and declared operable on May 28, 2021.

The fifth component failure occurred on June 24, 2021. During a monthly 800 rpm idle start test of EDG No. 1, voltage exceeded the acceptance criteria. Attempts to manually adjust voltage were unsuccessful. EDG No. 1 was shut down and declared inoperable. The licensee identified during troubleshooting that the microprocessor-based reference adjuster (RA-70) in the voltage regulator had failed. The RA-70 was replaced and EDG No. 1 was tested satisfactorily and declared operable on June 26, 2021.

NRC Region III dispatched a SIT on July 26, 2021, to review the five diesel generator component failures in accordance with the SIT Charter.

Detailed Sequence of Events 10/16/2017. Condition Report (CR) CR-2017-10465 was written by the licensee associated with operating experience involving an EDG speed switch failure at another site.

10/26/2017. A 10 CFR Part 21 notification was submitted to the NRC by Engine Systems, Inc. (ESI). The notification identified Davis-Besse as a customer that was supplied two speed switches by ESI that applied to the notification.

10/30/2017. CR-2017-10873 was written by the licensee for the 10 CFR Part 21 notification on the EDG speed switches from ESI. The licensees evaluation determined that the two speed switches identified in the notification were installed in the SBO Diesel Generator on July 11, 2017.

2/16/2018. The licensee performed measurements of the voltage drop across the common and normally open contacts of the SBO Diesel Generator speed switches to determine if transient voltages exceed voltage specifications provided by ESI as part of their evaluation of the 10 CFR Part 21 notification. No transient voltages exceeded the voltage specifications provided.

7/3/2019. A significant electrical ground was detected on the SBO Diesel Generator.

Availability and functionality of the SBO Diesel Generator was maintained.

7/10/2019. Ground detection investigation commenced, and the ground was determined to be on speed switch ESS 1 on the SBO Diesel Generator. The failed speed switch ESS 1 was one of the two speed switches identified in the notification under 10 CFR Part 21 notification submitted on October 26, 2017.

7/11/2019. The failed speed switch ESS 1 was replaced on the SBO Diesel Generator with a new speed switch.

7/12/2019. The SBO Diesel Generator was run for a monthly test satisfactorily.

3/11/2020. A fast start and 8-hour run of EDG No. 1 was performed satisfactorily.

3/13/2020. During integrated SFAS testing on Channel 1, which includes a fast start of EDG No. 1, the output breaker did not close, and operators performed an emergency shutdown for EDG No.1.

3/14/2020. Troubleshooting identified the most likely cause of the failure was a failed flyback diode across the K2 relay. However, the diode was shorted when technicians applied a reverse polarity during troubleshooting and no further failure analysis could be done. A new diode was installed, and integrated SFAS testing was reperformed on Channel 1 satisfactorily.

8/7/2020. A fast start of EDG No. 2 was performed as part of the 184-day surveillance test.

The test was performed satisfactorily.

8/25/2020. Operators received a Bus 2 DC system ground alarm with indications of a hard ground on DC Motor Control Center (MCC) 2. Two-hour monitoring of the DC MCC 2 ground meter by operators was directed and troubleshooting for the ground commenced.

8/26/2020. As part of troubleshooting, the High-Pressure Injection (HPI) Pump No. 2 DC lube oil pump breaker, D206, was opened with no changes on ground indication.

8/29/2020. Operators were directed to extend DC MCC 2 ground meter checks to four-hour monitoring. The Teams review of the station's ground readings found that the hard positive ground on DC MCC 2 that came in on August 25, 2020, shifted between log readings on August 28th at 8:00 p.m. and August 29th at 8:00 a.m., to a hard neutral ground or a reduced positive ground. The Team concluded that the second hard ground (neutral bus) came in between log readings and caused thermal damage to components on the speed switch.

9/1/2020. Licensee troubleshooting determined the most likely cause of the hard ground on DC MCC 2 was a neutral ground on the HPI Pump No. 2 DC lube oil pump breaker, D206.

9/4/2020. During routine monthly testing of EDG No. 2, a lockout of EDG No. 2 occurred.

Troubleshooting determined that the cause of the lockout was a failed speed switch on EDG No. 2. The failed speed switch was previously installed on October 4, 2019, as part of a preventive maintenance activity.

9/7/2020. The failed speed switch on EDG No. 2 was replaced. Post maintenance testing on EDG No. 2 was completed satisfactorily and EDG No. 2 was declared operable.

11/9/2020. The speed switch on EDG No. 1 was replaced as part of a preventive maintenance activity.

11/12/2020. EDG No. 1 was fast started for the 184-day surveillance test. Testing was completed satisfactorily.

11/25/2020. The licensee identified that the speed switch removed from EDG No. 1 as part of the preventive maintenance activity showed signs of overheating.

2/12/2021. The licensee determined that a past operability evaluation needed to be performed on EDG No. 2 for the speed switch failure discovered on September 4, 2020. A failure analysis of the failed speed switch by a vendor determined that the most likely cause was a voltage surge on the 125 Volts direct current (Vdc) line, but identified a component failure as a possible cause also.

5/27/2021. During a fast start of EDG No. 1 for the 184-day surveillance test, EDG No. 1 failed to reach steady state voltage and frequency. Troubleshooting for the cause of the failure commenced.

5/28/2021. Troubleshooting identified an intermittent failure on the FFSS on EDG No. 1. The switch is a General Electric SBM type switch that had no continuity through the 400 rpm (fast start) contacts approximately 1 of 5 times. The licensee replaced the FFSS, satisfactorily re-performed the 184-day surveillance test, and declared EDG No. 1 operable.

6/8/2021. The licensee wrote CR-2021-04520 for the negative trend in EDG reliability for the current operating cycle. They identified three failures that resulted in the EDGs not performing their design function (failures on 3/13/2020, 9/4/2020, and 5/27/2021).

6/24/2021. During a routine monthly run of EDG No. 1, the licensee identified an overvoltage condition on the EDG and declared EDG No. 1 inoperable.

6/25/2021. Troubleshooting identified a failed RA-70 in the EDG No. 1 voltage regulator.

The failed RA-70 was originally installed on September 23, 2014, as part of a preventive maintenance activity. The RA-70 was replaced, the monthly run of EDG No. 1 was satisfactorily performed, and EDG No. 1 was declared operable.

Assessment 93812 Charter Item No. 2 Review of Diesel Generator Troubleshooting Station Blackout Diesel: July 3, 2019 The licensees troubleshooting efforts found that the DC ground on the SBO Diesel Generator on July 3, 2019, was caused by a failed capacitor in one of the two SBO Diesel Generator speed switches. The speed switch was the same model discussed in 10 CFR 50, Part 21 Report 2017-50-00 (ADAMS Accession Number ML17311A150) and was susceptible to high voltage transients from externally connected relay coils. The licensee had previously performed testing on the SBO DC circuit, confirmed that the speed switch output contacts were not subject to the high transient voltages, and concluded that it was appropriate to keep the speed switches installed in the SBO Diesel Generator. The SBO Diesel Generator would have functioned properly even with the failed capacitor because it only created a single ground on the DC circuit.

The licensee replaced the failed speed switch with a model that was not susceptible to high voltage transients. One speed switch that was manufactured prior to the Part 21 report noted above remains installed in the SBO Diesel Generator; however, based on the functional configuration its failure would not affect the ability of the SBO Diesel Generator to start and load.

These speed switch models were not used on the Davis-Besse safety-related EDGs and the licensee no longer maintains any speed switches manufactured prior to the Part 21 report in stock. Future SBO speed switch replacements will be the upgraded models that are not susceptible to failures from high transient voltages applied to the output contacts. The Team concluded that the licensees troubleshooting efforts and extent-of-condition reviews were reasonable.

Division 1 Emergency Diesel Generator: March 13, 2020 The licensees troubleshooting efforts and evaluations concluded that the most probable cause for the failure of the Division 1 EDG on March 13, 2020, was a failed diode in the field flash circuit. Specifically, a shorted diode that was connected in parallel with the field flash contactor (FFC) created a low resistance current path around the FCC coil and prevented it from energizing and closing. The function of the FFC is to control the DC current to the generator field during the EDG startup sequence. Current from the licensees DC distribution system is supplied to the generator field until the generator is capable of supplying its own field current.

The licensee made preventive maintenance strategy changes for the EDG excitation system to replace the contactor diodes at a ten-year frequency. Conversely, the failed diode had been installed in the Division 1 EDG for approximately 15 years. In sum, the Team concluded that the licensees troubleshooting efforts and extent-of-condition reviews were reasonable.

Division 2 Emergency Diesel Generator: September 4, 2020 The licensees troubleshooting efforts and evaluations found that the failure of the Division 2 EDG on September 4, 2020, was caused by a failed speed switch. Specifically, the speed switch failed from being exposed to steady state voltage levels in excess of the ratings of a metal oxide varistor within the speed switch.

The licensee identified and replaced the failed speed switch in a timely manner; however, they did not identify the cause of the speed switch failure until July 26, 2021. Two items drove the licensee to identify the cause. The first was their EDG reliability study, which was being performed in response to multiple EDG failures at the station over the past two years.

The second item was an NRC question provided weeks earlier associated with speed switch sub-component voltage ratings and their ability to withstand inductive voltage transients from nearby relay coils.

The Team determined, once the cause of the speed switch failure was identified, the licensees extent-of-condition reviews were reasonable. Additional details associated with this speed switch failure are documented in a finding and associated apparent violation in the Results Section of this report.

Division 1 Emergency Diesel Generator: May 27, 2021 The licensees troubleshooting efforts and evaluations found that the failure of the Division 1 EDG on May 27, 2021, was caused by a failed FFSS. The licensee quickly identified that the EDG FFSS contacts had failed, which prevented the EDG from flashing its generator field during the starting sequence. The licensee replaced the FFSS and restored the EDG to an operable status. Their initial investigation concluded that the switch contacts failed from fouling on the contacts. The licensees condition report stated, No PM is currently performed on the switch, leading to fouling that is present on the contacts. Part of the resolution will be to create a PM to clean the contacts on these switches. Additional details associated with the FFSS failure are documented in a finding and associated apparent violation in the Results Section of this report.

The Team reviewed the licensees extent-of-condition and found that it was narrowly scoped.

The licensees extent-of-condition looked for similar EDG switches that would not have given indication of contact position through equipment operation or indicating lights. As part of its review, the Team had identified a similar switch that was installed as part of the same modification that had installed the FFSS. The switch was a test switch used in the 4160 Vac breakers for the reactor coolant system makeup pumps. The licensee reviewed the Teams concern and found that the switch in question was exposed to the same degradation mechanism as the FFSS. The switch provided no immediate feedback on its position, was not normally operated under load, and was approximately the same age as the failed EDG FFSS. The licensee entered the issue into their corrective action program as CR-2021-06146, 2021 NRC SIT: Switch DST for Makeup Pump Time Delay Relay 2Y1 Isolation Potentially Exposed to Similar Degradation Mechanisms as EDG FFSS.

Division 1 Emergency Diesel Generator: June 24, 2021 The licensees troubleshooting efforts and evaluations found that the high voltage condition on the Division 1 EDG on June 24, 2021, was caused by an internal circuit failure of the RA-70 microprocessor-based reference unit. The RA-70 provides a resistance reference value to the automatic voltage regulator and it had failed to its high limit setpoint. The high reference setpoint raised the EDG output voltage to almost 110 percent of its nominal setpoint.

The licensee sent the failed RA-70 reference unit to be independently evaluated and at the conclusion of the inspection period the formal evaluation had not been provided to the Team.

The licensee provided preliminary conclusions from their vendor, which determined that a resistor had failed open. Inspection of the resistor found no discoloration or indication of heating and no mechanical damage was identified. The vendor did not identify a failure trend for this resistor and concluded that the failure was random.

The licensee planned to replace the RA-70 reference unit for the Division 2 EDG in September 2021; however, the technicians identified a questionable rattling noise in the replacement part and, out of an abundance of caution, postponed the replacement. The Team noted that the RA-70 reference unit is not used in the SBO Diesel Generator. The Team concluded that the licensees troubleshooting efforts and extent-of-condition reviews were reasonable.

Assessment 93812 Charter Item No. 3 Review of Diesel Generator Failed Components Station Blackout Diesel: July 3, 2019 The Team noted the similarities between the SBO speed switch failure and the failure discussed in Part 21 Report 2017-50-00, but concluded that the licensees previous justification for leaving the affected switch installed was reasonable. Specifically, the licensee measured transient voltages and confirmed that the speed switch output contacts were not being subject to the high transient voltages that caused the failure that was documented in the Part 21 report.

The Team also confirmed that the SBO Diesel Generator was not susceptible to the same steady state high voltage condition that caused the September 2020 Division 2 EDG speed switch failure. The DC system for the SBO Diesel Generator had a traditional configuration and did not have an excessively high voltage potential between the positive DC connection and ground.

Division 1 Emergency Diesel Generator: March 13, 2020 The Team reviewed the March 2020 Division 1 EDG failure, which was found to be caused by a failed diode, and did not identify any design, environmental, or surge protection concerns.

Division 2 Emergency Diesel Generator: September 4, 2020 The Team found that the speed switch design for the Division 1 and 2 EDGs was not compatible with the licensees DC distribution system. Specifically, on July 26, 2021, the licensee identified and documented the design concern, which was associated with a high voltage potential between the stations positive DC connection and ground. The high voltage potential was caused by the connection location of the stations DC ground detection system, combined with the hybrid design of the stations DC distribution system. The hybrid DC distribution system allowed for loads to be supplied by 125 Vdc or 250 Vdc depending on how loads were connected to the three-wire DC distribution system. The licensees long-term corrective action for the design issue is to install a new compatible speed switch design in both EDGs. The licensee implemented interim corrective actions, which included replacing the speed switches on the safety-related EDGs and modifying their DC ground hunting procedure to prioritize checking the EDGs when DC grounds are detected.

The Team also reviewed the room temperature calculations for the Division 1 and 2 EDGs and found that the listed maximum temperature that the speed switch was exposed to was non-conservatively low. Specifically, the maximum exposure temperature listed for the speed switches was the maximum room temperature and not the warmer temperature within the cabinets that the speed switches were installed. The licensee documented the concern in their corrective action program, and determined that with the existing margin the speed switches remained qualified for the higher temperature.

Division 1 Emergency Diesel Generator: May 27, 2021 The Team reviewed the May 2021 Division 1 EDG failure, which was found to be caused by failed FFSS contacts. The Team did not identify any environmental or surge protection concerns.

The Team reviewed the modification that installed the FFSS in 2006 and found that prior to the modification the 400 RPM speed switch contacts, which flash the generator field when the EDG is in standby and aligned for automatic start, were tested every month. The modification created a parallel path around the 400 RPM speed switch contacts that allowed the licensee to align the field flash circuit to operate from the 800 RPM speed switch contacts for monthly testing, with the 400 RPM path aligned when the EDG was in standby and ready for automatic start. The design modification changed the testing frequency for the 400 RPM speed switch contacts from monthly to every six months, which potentially allowed a degraded condition to exist for substantially more time. The Team found that the licensee screened out the modification as not requiring the performance of a safety evaluation under 10 CFR 50.59. The Team concluded that this issue requires further NRC review in order to determine whether a safety evaluation under 10 CFR 50.59 should have been performed, or even possibly that the change in testing frequency should have had prior NRC approval before being implemented. As a result, the issue is being documented as an Unresolved Item in the Results Section of this report.

Division 1 Emergency Diesel Generator: June 24, 2021 The Team reviewed the June 2021 Division 1 EDG high voltage issue, which was caused by a failed RA-70 reference unit, and did not identify any design, environmental or surge protection concerns. The Team did verify that the RA-70 reference unit was appropriately rated for the calculated high temperatures it could experience within its cabinet, and that it was not affected by the same DC high voltage potential that caused the Division 2 speed switch failure in September 2020.

Assessment 93812 Charter Item No. 4 Assessment of Division 1 Emergency Diesel Generator Failure that Occurred on June 24, 2021 The Team reviewed and evaluated corrective action program documents associated with the Division 1 EDG failure that occurred on June 24, 2021. Specifically, the Team reviewed condition reports, past operability assessments, maintenance rule evaluations and design documents.

The cause of the high voltage condition observed on June 24, 2021, was found to have been an internal circuit failure of the RA-70 microprocessor-based reference unit. The RA-70 provides a resistance reference value to the automatic voltage regulator, and it had failed to its high limit setpoint. The high reference setpoint had raised the EDG output voltage to almost 110 percent of its nominal setpoint.

The Team reviewed the licensees maintenance rule functional failure assessment, which concluded that the high voltage condition would not have prevented the EDG from performing its design function and it was not a maintenance rule functional failure. The licensee reviewed the design change documents associated with the installation of the updated EDG excitation system and found that the specific failure mode of the RA-70 failing to its maximum resistance setting had been assessed in a failure modes and effects analysis. That analysis concluded that the EDG would be able to start and load, and while the EDG voltage would be higher than normal, it would be within an acceptable range for long-term operation.

The licensee also performed a separate evaluation to validate the conclusions of the failure modes and effects analysis. Specifically, the evaluation assessed the effects of the high voltage condition on the operation of loads connected to the EDG. Examples of evaluated components included battery chargers, transformers, induction motors, motor-operated valves and the EDG exciter. The evaluation concluded that some of the components would age at an accelerated rate, but did not anticipate that any of the components would fail due to the higher than normal voltage condition.

The Team independently reviewed the evaluation and found that the licensees conclusions associated with the downstream EDG loads were reasonable. However, the Team found that the evaluation did not completely address the effects of manually adding additional loads to the EDG and the potential increase in generator exciter current levels. The Team questioned the licensee whether the increased generator exciter currents could cause an exciter component failure and/or prevent proper operation of the EDG. The licensee revised the evaluation to address the Teams questions associated with high generator exciter currents; however, this revised evaluation was not completed before the conclusion of the Teams inspection activities and will be documented as an Unresolved Item in this report, pending completion of the NRCs review and assessment.

Assessment 93812 Charter Item No. 5 Assessment of Potential Division 1 Emergency Diesel Generator Common Start Circuitry Issues The Team evaluated the causal information from the Division 1 EDG failures that occurred on March 13, 2020, and May 27, 2021, to evaluate whether there was a common cause between the two failures.

The March 2020 and the May 2021 EDG failures were both associated with the FFC failing to energize. The function of the FFC is to control the DC current to the generator field during the EDG startup sequence. Specifically, current from the stations essential DC distribution system is supplied to the generator field until the generator is capable of supplying its own field current. The May 2021 diesel failure was found to be caused by failed contacts on the FFSS, which prevented circuit current from energizing the FCC coil. The March 2020 diesel failure was found to be a shorted diode that was connected in parallel with the FCC. The shorted diode created a low resistance current path around the FCC coil and prevented it from energizing and closing. The Team reviewed the licensees troubleshooting efforts and causal information for both failures and determined the licensees conclusions were reasonable.

Subsequently, the Team concluded that the two events were independent failures and that the March 2020 failure was not associated with FFSS. The Teams review found that the FFC functioned properly during the diode replacement post maintenance test and also during fast start surveillance testing in May 2020 and November 2020. Additionally, during the licensees troubleshooting efforts for the March 2020 EDG failure, the licensee performed direct readings of the FFCC contacts and confirmed they had functioned properly.

Assessment 93812 Charter Item No. 6 Assessment of Licensee Response to Applicable Diesel Generator Defect Reports Made Under 10 CFR Part 21 The Team reviewed applicable EDG defect reports made to the NRC under 10 CFR Part 21.

The Team identified one defect report made on October 26, 2017, by ESI that was applicable to the licensees EDGs (ADAMS Accession No. ML17311A907).

On October 26, 2017, ESI reported to the NRC that they began an evaluation on September 12, 2017, upon notification of a potential issue with a speed switch supplied to another site that had failed in service. An analysis performed by another vendor determined the failure was due to a shorted capacitor that was installed on the speed switchs relay output contacts to ground. ESI concluded their evaluation on October 25, 2017, and determined the issue was a reportable defect under 10 CFR Part 21.

CR-2017-10465 was written on October 16, 2017, after the licensee identified operating experience associated with the failure that ESI discussed in their notification On October 30, 2017, CR-2017-10873 was written to evaluate ESIs notification under 10 CFR Part 21. The licensee identified that the two applicable speed switches were installed in the SBO Diesel Generator on July 11, 2017. No additional applicable speed switches were identified as installed in the plant or in stock. The licensee evaluated these two CRs for applicability of the operating experience.

ESI provided additional information that the shorted capacitor was rated for a 200 Vdc working voltage and a withstand voltage of 500 Vdc. As part of their evaluation, on February 16, 2018, the licensee performed measurements of the voltage drop across the common and normally open contacts of the SBO Diesel Generator speed switches to determine if transient voltages exceeded voltage specifications provided by ESI as part of their evaluation of the 10 CFR Part 21 notification. No transient voltages exceeded the voltage specifications provided. The licensee determined that no further actions were needed. The two speed switches were not replaced or sent to the vendor to address the identified issue. On July 11, 2019, speed switch ESS 1 on the SBO Diesel Generator was replaced due to a failure of the speed switch. The replacement speed switch was a redesigned speed switch. The other speed switch is currently installed in the SBO Diesel Generator.

The Team did not identify any concerns with the licensees evaluation and response to the applicable defect report made under 10 CFR Part 21.

Assessment 93812 Charter Item No. 7 Assessment of Diesel Generator Systems' Performance Monitoring Information The Team assessed the licensees monitoring of the diesel generator systems performance information. Diesel generator system health reports from 2019 through 2020 were reviewed along with any condition reports the licensee generated for possible trends for the EDGs and the SBO Diesel Generator. Additionally, the overall timeliness and effectiveness of evaluating associated failures, degradations, and deficiencies for the EDGs and SBO Diesel Generator were reviewed.

On June 8, 2021, the licensee wrote CR-2021-04520 that identified a negative trend in EDG reliability during the current operating cycle. The condition report was to document and further investigate the unsatisfactory trend in EDG performance after three EDG failures. A corrective action for this condition report was to perform an evaluation of EDG reliability. The evaluation included a fourth EDG failure that occurred on June 24, 2021.

The Team reviewed the evaluation that assessed any commonalities in the failures and gaps that may have led to the negative trend in performance. The licensee identified two main gaps during the assessment associated with the preventive maintenance strategy for EDG components, and the insufficient evaluation of the design of some EDG components to ensure they were appropriate for their application. The licensee recommendations from this assessment included routine review of recommended maintenance programs for EDG systems to identify any changes to the programs, and evaluation and/or creation of preventive maintenance activities for identified EDG components.

The Team noted the conclusions in the licensees assessment associated with overall EDG component reliability, preventive maintenance strategies, and evaluation of the design requirements of some EDG components. The Team did not identify any additional concerns with the licensees monitoring of the diesel generator systems performance information.

Assessment 93812 Charter Item No. 8 Review of Applicable Diesel Generator Industry Information The Team performed an independent search of a sample of industry information associated with the sites diesel generators and validated that the licensee reviewed the issues and that their reviews were adequate.

The Team reviewed the licensees procedures associated with operating experience, evaluations of operating experience included in CRs, maintenance rule evaluations, and a list of operating experienced reviewed as part of the licensees in progress EDG reliability evaluation.

The Team determined the licensees use of operating experience was appropriate and did not identify any concerns associated with the licensees use of operating experience.

Assessment 93812 Charter Item No. 9 Review for Potential Diesel Generator Safety Concerns The Team reviewed each of the diesel generator component failures for any safety concerns with the potential for generic applicability.

The Team did not identify any additional concerns with these diesel generator component failures other than what was discussed in Charter Item No. 3 that would result in potential generic safety concerns.

Assessment 93812 Charter Item No. 10 Timeline of Significant Events Associated with the July 8, 2021, Complicated Reactor Trip Summary On July 8, 2021, with the Unit in Mode 1 at 100% power, the reactor automatically tripped due to a trip of the main turbine. The main turbine trip was caused by a loss of power to the main generators AVR which is normally fed by MCC E32 (includes MCC E32A and its cascaded MCC, E32B). Power to MCC E32 was lost due to a failure of its normal power supply breaker (BE306) to close when transferring the MCC back from its alternate power supply (through breaker BF306) after planned post maintenance testing. Although the AVR contains an ATS which should have automatically swapped the AVR power supply to its alternate, the ATS lost control power and was, therefore, unable to perform the swap. The ATS control power should have been supplied by the DEHC D3201B UPS battery bank. However, the DEHC UPS is also powered from MCC E32, and a defective battery cell caused the UPS to be unable to supply its loads when MCC E32 lost power.

Following the reactor trip, an overcooling event was caused by valve MS199, Moisture Separator Reheater No. 1 Reheater Steam Supply Second Stage Source Valve, remaining open. The valve is powered by MCC E32, and due to its loss of power, it failed to close following the turbine trip. During this transient, Feedwater Startup Valve SP7B should have automatically been opened by the integrated control system (ICS) in response to a low SG No. 1 level. However, the automatic controller for the valve did not respond as expected causing SG No. 1 level to lower to the steam feed rupture control system (SFRCS) initiation setpoint. The SFRCS initiation caused both turbine-driven auxiliary feedwater (AFW) pumps to start. The on-watch operations shift crew (hereafter referred to as simply the crew)subsequently manually started the high-pressure injection (HPI) pumps, low pressure injection (LPI) pumps, and the standby makeup (MU) pump, per procedure, in response to the unanticipated overcooling event. Valve MS199 was closed a short time later by manual operator actions locally at the valve, allowing the crew to stabilize the plant.

With the plant stabilized and decay heat removal via the main condenser, the crew entered the stations reactor trip recovery procedure. The crew returned the HPI, LPI, and MU pumps to the normal standby alignment, and also secured Main Feedwater Pump (MFW) No. 1 and AFW Pump No. 1 to reduce main steam loads since they were not needed at the time. The auxiliary boiler was also started to allow main steam loads to be transferred to the auxiliary steam system, to further reduce main steam loads.

Approximately two hours after the initial reactor trip, while attempting to transfer gland steam from main steam to auxiliary steam, a low-pressure condition was experienced in SG No. 2 resulting in a second overcooling event and another SFRCS actuation. This SFRCS actuation, on low steam generator pressure vice low steam generator level as with the first case, caused the main steam and feedwater lines to isolate. This isolation removed the crews ability to use the main condenser as a heat sink and resulted in the need to use the SG atmospheric vent valves (AVVs) for decay heat removal. The overcooling event was caused by an excessive main steam demand through the gland steam (GS) system when attempting to transfer GS from main steam to auxiliary steam. An elevated GS header pressure caused a combination of gland sealing steam relief valves and/or unloader valves to open, which created the large main steam demand.

With the main steam lines isolated, the AVVs were being used for decay heat removal. The AVVs should have automatically been opened by the plants ICS to control SG pressures.

However, the AVVs did not respond as expected and the operators had to manually control them from the main control room in order to maintain adequate SG pressures. Subsequently, AVV No. 2 failed partially open and could not be controlled from the main control room. The crew isolated the motive force to the valve (instrument air) to force it to reposition to the closed position. The crew then stationed an equipment operator locally in the plant to operate the valve with its handwheel, as was specified in applicable plant procedures and per their training, in order to adequately control SG No. 2 pressure. These actions allowed the crew to stabilize the plant once again and continue with reactor trip recovery procedures.

Detailed Sequence of Events (all times listed are Eastern Daylight Time)7/8/2021 - The Unit was in Mode 1 at 100% power.

7/8/2021 - 2151. The crew performs functional test of BF306 - MCC E32 Alternate Feeder.

7/8/2021 - 2154. The crew attempts to return MCC E32 power supply from BF306 Alternate Feeder breaker to BE306 Normal Feeder breaker. Breaker BE306 fails causing a loss of power to MCC E32. Due to the loss of power to MCC E32, power is lost to the AVR (normal power), DEHC UPS, and valve MS199, among other loads. A failed cell in the DEHC UPS battery bank causes a loss of control power to the AVR ATS. The loss of normal power to the AVR, and its inability to receive power from its alternate power source due to the loss of control power to the AVR ATS, causes a total loss of power to the AVR, which leads to a main turbine trip. The turbine trip causes a reactor trip via the anticipatory reactor trip system (ARTS). The crew enters Emergency Procedure DB-OP-02000, RPS, SFAS, SFRCS Trip, or SG Tube Rupture.

7/8/2021 - 2158. The crew diagnoses the overcooling event and implements DBOP02000, Section 7.0, Overcooling, actions. The crew also starts to place HPI, LPI, and MU pumps in service per DBOP02000, Attachment 8, Place HPI/LPI/MU In Service, as directed by Step 7.1 of DBOP02000.

7/8/2021 - 2201. The crew takes manual control of SP7B from the control room in attempt to restore SG No. 1 level, but level is unable to be restored before automatic SFRCS initiation occurs due to the SG No. 1 low level condition. Both AFW pumps automatically start.

7/8/2021 - 2202. The crew terminates the overcooling event by manually closing valve MS199 in the plant, which had been previously identified as the source of overcooling using DBOP02000, Attachment 20, Isolate or Control Potential Sources of Overcooling.

7/8/2021 - 2205. The crew completes the HPI/LPI/MU actions of DBOP02000, Attachment 8 to place the pumps in service.

7/8/2021 - 2218. The crew enters procedure DBOP06910, Trip Recovery. The crew commences securing and re-aligning HPI/LPI/MU pumps to the standby condition. The crew also commences actions to reduce steam loads on the main steam system.

7/8/2021 - 2235. The crew secures MFP No. 1.

7/8/2021 - 2321. The crew completes return of the HPI/LPI/MU pumps to standby alignment.

7/8/2021 - 2332. The crew re-establishes manual control of SP7B valve from the control room because it is not responding as expected to automatically maintain SG No. 1 level as AFP No.1 flow is reduced to the SG.

7/8/2021 - 2340. The crew secures AFP No. 1.

7/8/2021 - 2345. The crew begins transfer of gland sealing steam from main steam to auxiliary steam using procedure DBOP06205, Turbine Generator and Main Feedwater Pump Turbine Gland Steam and Turbine Drains.

7/8/2021 - 2350. The crew identifies a second overcooling event, and re-enters DBOP02000, Section 7.0, Overcooling.

7/8/2021 - 2356. The crew receives notification from security and equipment operators in the plant that gland steam relief valve(s) may be lifting. Steam is seen issuing from the gland steam combined relief tailpipe on the roof of the plant.

7/8/2021 - 2358. An automatic SFRCS initiation occurs due to low steam pressure on SG No. 2. This automatic initiation results in a main steam line isolation, which removes the crews ability to use the main condenser as a heat sink.

7/9/2021 - 0008. The crew identifies that the AVVs are not automatically controlling SG pressures as expected and takes manual control of the AVVs at their controllers in the control room.

7/9/2021 - 0215. AVV No. 2 fails to operate as expected from the main control room and remains partially open. The crew subsequently isolates air to the valve to force it to the closed position.

7/9/2021 - 0240. The crew establishes manual local control of AVV No. 2 using an equipment operator stationed in the plant at the valve handwheel and, using established plant procedures, effectively controls SG No. 2 pressure.

7/9/2021 - 0515. The crew re-establishes manual control of SP7B valve from the control room because it is not responding as expected to automatically maintain SG No. 1 level as AFP No.1 flow is reduced to the SG.

Assessment 93812 Charter Item No. 11 July 8, 2021, Complicated Reactor Trip Equipment Performance The Team reviewed and evaluated equipment performance during the July 8, 2021, complicated reactor trip with a focus on the equipment below. The Team performed walkdowns of the equipment, reviewed documentation associated with the equipment, and interviewed site personnel regarding equipment performance.

a. Atmospheric Vent Valve Operation As discussed in Charter Item No. 10, approximately two hours after the initial reactor trip, an SFRCS actuation occurred which closed the main steam lines. The AVVs failed to open automatically, as expected, to control SG pressures and remove decay heat. The crew had to take manual control of both AVV No. 1 (ICS11B associated with SG No. 1) and AVV No. 2 (ICS11A associated with SG No. 2) from the control room in order to open the valves. The AVVs failed to automatically open following the isolation of the main steam lines because the ICS logic for transferring SG pressure control from the turbine bypass valves to the AVVs was not completed. As discussed in the station' Updated Safety Analysis Report (USAR),

Section 7.7.1.2.9, when either of the two MSIVs are less than 90% open (one out of two logic), the ICS is supposed to transfer SG pressure control from the turbine bypass valves to the AVVs.

During troubleshooting activities, the licensee determined the two limit switches providing position indication for the MSIVs to the ICS had failed to perform their function. The actuating arm for Limit Switch ZS100B (associated with MSIV MS100) fell off the switch due to the loosening of its set screw, as described in CR-2021-05279, causing it to incorrectly indicate an open MSIV position. The licensee repaired the ZS100B limit switch by reinstalling its actuating arm on July 10, 2021 (Notification 601324629). The other limit switch ZS101B (associated with MSIV MS101) was found to be the incorrect part for the application, as described in CR-2021-05271, causing it to also indicate an incorrect open MSIV position.

The incorrect part (86900806) used for Limit Switch ZS101B is a spring-return limit switch.

Although it momentarily indicated a closed valve position, it then returned back (spring-returned) to indicate an open valve position as the MSIV continued going closed. The correct part (86901006) does not have a spring-return and would have continued indicating a closed valve position as the MSIV continued going closed. The licensees failure to install the correct part for Limit Switch ZS101B was determined to be a self-revealed Green finding of very low safety significance and an associated non-cited violation (NCV) of 10 CFR Part 50, Appendix B, Criterion V, as described in the Results Section of this report.

Approximately two hours after the AVVs were placed in service following the SFRCS actuation that isolated the main steam lines, AVV No. 2 failed partially open. Attempts to close the valve from the main control room were unsuccessful. The crew ultimately had to isolate instrument air to the valve to get it to close (the valve fails closed on a loss of air).

Once closed, the crew stationed an equipment operator in the plant to locally operate the valve with its handwheel to control SG No. 2 pressure, as was specified in applicable plant procedures and per their training. Subsequent troubleshooting identified a failure of the AVV No. 2 controllers feedback arm. The failure of the feedback arm during valve operation caused the controller to be unable to close the valve, although it was being demanded closed by the crew. The licensee completed Work Order (WO) 200858490 on July 11, 2021, to repair the valve.

Other than the finding and associated NCV for the failure to install the correct part for Limit Switch ZS101B, the Team did not identify any additional performance deficiencies related to the performance of the AVVs during the July 8, 2021, reactor trip.

b. Start Up Feedwater Control Valves SP7A and SP7B Following a reactor trip, the rapid feedwater reduction (RFR) circuitry of the ICS is designed to throttle the SP7A and SP7B, Startup Feedwater Valves, to approximately a 15-20% open position. The SP7A valve feeds SG No. 2 and the SP7B valve feeds SG No. 1. The RFR circuitry is designed to then release control of each valve to the ICS for SG level control when its associated SG lowers to 40 inches or after two-and-a-half minutes have passed, whichever comes first. At that point, the ICS is supposed to maintain SG levels at the 40-inch setpoint with the SP7A and SP7B valves. After the reactor trip, both SG levels were lowering but above the 40-inch setpoint at the two-and-a-half minute mark, when the RFR circuitry released control of the valves. At that point, both valves went to their closed positions as expected, since SG levels were above the setpoint. Both SG levels continued lowering, but SG No. 1 level reached the 40-inch setpoint sooner because of the overcooling caused by MS199 which is associated with SG No. 1. The crew noted the SP7B valve did not open as expected. The crew attempted to place the SP7B controller in hand control to re-open the valve, but SG No. 1 level could not be restored before the SFRCS initiation setpoint (at 23.5 inches) was reached. SG No. 2 level never got below the 40-inch setpoint, therefore SP7A never opened, nor was it expected to open at that time.

On two separate occasions after the initial reactor trip and SFRCS initiation, the crew attempted to place the SP7B controller back in automatic to maintain SG No. 1 level.

However, the controller failed to control SG No. 1 level in automatic on both occasions (in addition to its original failure to control in automatic) and the crew had to revert back to manually controlling the SP7B valve from the control room. The licensee believed the failure of the SP7B controller to maintain SG No. 1 level was due to an ICS module (ICSFW5403)failure. The licensee captured this issue as an operational challenge to the crew and subsequently replaced the ICS module under WO 200859743 on September 2, 2021.

During the 2nd overcooling event which led to the 2nd SFRCS initiation, SG No. 2 level began dropping and the SP7A valve was slow to respond. It started automatically opening when SG level approached 37 inches, instead of beginning to open at the expected 40-inch SG level setpoint. As the valve was automatically opening, the crew took manual control of the valve and continued opening the valve in order to re-establish SG No. 2 level. Shortly after taking manual control of the valve, the 2nd SFRCS actuation occurred causing the SP7A valve to close. Although the SP7A valve controller may not have opened the valve as quickly as expected prior to the crew taking manual control of the valve, it was in fact responding to the lowering SG level. Therefore, the licensee concluded the controllers slow response was due to a calibration issue with the controller module (ICSFW5503). The licensee also replaced this ICS module under WO 200859743 on September 2, 2021.

The Team did not identify any performance deficiencies associated with the performance of the SP7A and SP7B valves.

c. Digital Electrohydraulic Control System Uninterruptable Power Supply On July 8, 2021, the failure of breaker BE306, MCC E32 Normal Feeder, caused a loss of power to MCC E32 (includes MCC E32A and its cascaded MCC E32B). On a loss of power to MCC E32A, the DEHC D3201B UPS battery bank was expected to continue providing power to the DEHC system and control power to the AVR ATS. However, the DEHC UPS battery bank failed resulting in the loss of one of two redundant power supplies to the DEHC system, and a complete loss of control power to the AVR ATS. During the licensees subsequent troubleshooting (WO 200858488), a failed battery cell in the UPS battery bank was identified. The battery cell was found to have positive terminal post growth, leakage around the post, and a reduced individual cell voltage compared to the remainder of the cells in the battery bank. The failed battery cell caused the DEHC UPS battery bank to be unable to supply the required voltage when demanded, causing the battery output breaker to open.

The licensee replaced the failed battery cell (WO 200858529) on July 9, 2021, to place the DEHC UPS back in service.

Although the DEHC UPS (D3201A) was classified as Non-Critical and had a general inspect and clean preventive maintenance activity associated with it, the DEHC UPS battery bank (D3201B) was originally classified as Non-Impact and had no specific preventive maintenance activities associated with it. The DEHC UPS battery bank shouldve been classified as Non-Critical because it provided the sole source of control power for the AVR ATS and its failure could induce the failure of the AVR, a critical component, as was seen during the July 8, 2021, reactor trip. A Non-Critical component classification would have required the licensee to develop preventive maintenance activities for the DEHC UPS battery bank to enhance its reliability. The Team determined the failure to appropriately classify the DEHC UPS battery bank as Non-Critical was a self-revealed Green finding of very low safety significance as described in the Results Section of this report.

In addition to the issue with the DEHC UPS battery bank classification, the licensees causal evaluation (CR-2021-05250) performed for the July 8, 2021, reactor trip identified a gap in their engineering change process. Specifically, the causal evaluation found that component classification and single point vulnerability reviews were not being performed for new equipment installed as part of engineering changes. This issue is captured in the licensees Corrective Action Program (CAP) as CR 2021-05916. A total of 2,637 currently installed components were identified which have not been classified per NOBP-ER-3901, Component Classification ER Workbench Module 1. Most of the components appear to be associated with the digital control rod drive system, plant computer, DEHC, station air compressor, and security systems, among other systems. The Team did not review any of these components in detail, as this was outside of the scope of the SIT Charter. However, the Team communicated the importance of correctly classifying and evaluating the breadth of installed components to the licensee to ensure appropriate preventive maintenance activities are in place for them.

d. Gland Steam Components Approximately two hours after the initial reactor trip on July 8, 2021, while the crew was attempting to transfer gland steam from main steam to auxiliary steam using procedure DB-OP-06205, Turbine Generator and Main Feedwater Pump Turbine Gland Steam and Turbine Drains, a low-pressure condition was experienced in SG No. 2 resulting in an overcooling event and SFRCS actuation. The licensee attributed the overcooling event to a failure of valve GS2338, Gland Seal Header Pressure Regulating Valve, to fully close. The excessive steam supply to the GS system caused an elevated GS header pressure, which in turn caused a combination of GS relief valves and/or unloader valves to open.

Although the licensee believes the GS 2338 failed open during the July 8, 2021, trip, the issue was not entered into the licensees CAP until July 12, 2021, (CR-2021-05330) during the subsequent reactor startup. At that time, GS2338 was isolated to prevent over-pressurizing the GS header because it was stuck in a throttled open position (approximately 60% open). During troubleshooting activities, the licensee discovered a degraded o-ring in the actuator which caused the valve to stay open when demanded closed.

The GS2338 valve was repaired (WO 200858874) on July 22, 2021, at which time it was returned to service. The GS2338 valve positioner was also replaced as part of the WO 200858874 activities.

The Team reviewed other components associated with the GS system which could have contributed to its performance during the July 8, 2021, reactor trip. The GS header pressure indicated in the control room is provided by PI2340, which has a 0 to 10 pounds per square inch (psig) range. During the attempted transfer of GS from main steam to auxiliary steam on July 8, 2021, the indicated GS header pressure never exceeded 3.5 psig, as observed by the crew. However, it is known that at least one of the GS safety relief valves lifted to relieve GS header pressure. GS1929 and GS1930, Steam Seal Relief Valves, are set to relieve pressure at 120 psig, and GS1931 and GS1932, Steam Seal Relief Valves, are set to relieve pressure at 20 psig. Therefore, some sections of the GS header must have been significantly above the indicated GS header pressure in order to lift the GS safety relief valves. The Team believes it was most likely the section past GS2385, Steam Seal Feed Bypass Valve, which experienced the elevated pressure causing the GS1929 and GS1930 relief valves to open.

The licensee verified the calibration of pressure transmitter PT2340 which feeds PI2340 on July 15, 2021, and did not identify any calibration issues. In addition, the licensee performed thermography scans of the GS relief valves on July 16, 2021, looking for signs of leakage after a relief lift. A potentially small steam leak was identified past GS1929, but the relief valve was considered seated given the minor leakage. The licensee did not identify anything else that would indicate the GS relief valves had opened below their setpoints.

Therefore, given the known facts, the indicated GS header pressure in the control room is insufficient to identify an overpressure condition of some sections of the GS header to prevent lifting the GS relief valves during GS transfers. As a result of questions from the Team, the licensee generated Standing Order No.21-003, Gland Steam - Local Operator During GS2385 Transfers, on August 6, 2021, to address this issue by requiring a local operator to be stationed at the GS2338 valve location and to monitor for GS relief valve lifting during GS transfers. The licensee subsequently incorporated this guidance into a Revision 25 of DB-OP-06205 as a final corrective action.

The Team also reviewed a sample of CAP documents associated with the GS system and noted there were known open issues with the GS1938 and GS1939, Steam Packing Unloading Valves. CR-2020-02730, Steam Packing Unloader Valves not Controlling Correctly, documents valve GS1938 was fully open and valve GS1939 was half open with a GS header pressure of 2.7 psig. Valves GS1938 and GS1939 are designed to control GS header pressure below their opening setpoints at 3.5 psig and 4 psig, respectively.

Therefore, since it is known that valves GS1938 and GS1939 were opening below their setpoints, it is reasonable to conclude they contributed to the GS system steam demand (similar to the opening of a GS relief valve) that led to a low-pressure condition in SG No. 2 on July 8, 2021, and an eventual SFRCS actuation which isolated the main steam lines.

During the Teams review of the CAP documents, an open issue with PIC2338, Steam Seal Feed Regulating Valve Controller, which is the controller for the GS2338 valve, was also noted. During troubleshooting efforts documented in Notifications 601266808 and 601302699 related to CR-2020-02730, PIC2338 was found to be degraded. Specifically, with an increase in gland steam header pressure, PIC2338 output did not increase as expected to close GS2338. PIC2338 had to be mechanically agitated on January 27, 2021, to get it to function properly to close the GS2338 valve. The issues with GS1938, GS1939, and PIC2338 have yet to be corrected. The licensee is relying on the instructions provided in Standing Order No.21-003, and subsequently incorporated into Revision 25 of DB-OP-06205, to prevent these equipment issues from causing another SFRCS initiation due to an excessive steam demand from the GS system, as was experienced on July 8, 2021.

Other than the finding of very low safety significance and associated NCV related to the adequacy of the GS transfer procedure (DB-OP-06205) described in the Results Section of this report, no additional findings were identified by the Team related to the performance of the GS components.

Assessment 93812 Charter Item No. 12 Review and Evaluation of Operator Response/Performance During the July 8, 2021, Complicated Reactor Trip The Team reviewed and evaluated the performance of operations personnel following the trip of the reactor on the evening of July 8, 2021. The Team performed walkdowns of the main control room and areas of the plant where response actions were performed, reviewed the plants documentation of the event response, and interviewed operations personnel responsible for the event response. The primary focus of this review centered around the operator performance during the from the time of the reactor trip at 9:54 p.m. on July 8, 2021, through the early morning hours of July 9, 2021, with special focus on the time period immediately following the reactor trip and the time leading up to the SFRCS initiation and isolation approximately two hours later.

a. Response to the Reactor Trip and Subsequent Overcooling The reactor tripped at 9:54 p.m. on July 8, 2021, directly from a main turbine/generator trip that resulted from the loss of power to the main generator automatic voltage regulator and DEHC. The loss of power also deenergized motor-operated valve MS199, Second Stage Source Valve, causing the valve to remain open following the main turbine trip and resulting in an excessive post trip steam demand on SG No. 1. The excessive steam demand coupled with the failure of Startup Feedwater Valve SP7B to respond in automatic as designed, resulted in automatic initiation and injection of auxiliary feedwater to both steam generators due to actuation of the SFRCS on low water level in SG No. 1 at approximately 10:01 p.m.

Between 9:54 p.m. and 9:58 p.m., the crew entered and began implementation of emergency operating procedure (EOP) DB-OP-02000, RPS, SFAS, SFRCS Trip, or SG Tube Rupture, including completion of the immediate actions, evaluation of specific rules, diagnosis of an overcooling event, and then transitioned to the Overcooling Section of DB-OP-02000. Upon entering the Overcooling Section of the EOP, the crew manually aligned and started HPI/LPI/MU injection, promptly determined that the overcooling was likely due to failure of MS199 to close, and dispatched personnel to manually close the valve.

At approximately 10:02 p.m., the crew was informed that MS199 had been shut, terminating the excess steam demand on SG No. 1. However, due to the auxiliary feedwater injection of cold water, both SGs continued to depressurize until SG levels stabilized at the auxiliary feedwater automatic level control setpoint of 49 inches. Due to the activity level in the main control room and limited personnel resources, as well as the unexpected rate of decrease in SG No. 1 water level from the excess steam demand, the crew did not immediately recognize that SP7B was not responding to automatically control water level in SG No. 1. They were, subsequently, unable to establish manual control of SP7B before the SFRCS actuation. In the Teams assessment, while it was possible that the crew could have prevented the low SG water level actuation of the SFRCS, it is unlikely that the SFRCS actuation could have been prevented because of the speed at which the events unfolded.

Approximately 20-25 minutes after the reactor trip, with primary and secondary system conditions stabilized, the crew entered DB-OP-06910, Trip Recovery, and exited DB-OP-02000. Prompt diagnosis and response by the crew terminated the overcooling event and primary system parameters remained above their setpoints for automatic actuation of the SFAS.

The Team concluded that there were no significant operator performance issues identified during the initial stabilization of the primary and secondary plant following the reactor trip and operator performance was evaluated as acceptable for this time interval.

b. Steam and Feedwater Rupture Control System Actuation and Main Steam Isolation Valve Closure With the primary and secondary plants stabilized following the reactor trip, the crew began to implement DB-OP-06910, Trip Recovery. The initial focus was on reducing unnecessary steam loading on the SGs, including shutdown of Main Feed Pump No. 1, restoring the normal feedwater lineup to SG No. 1, shutdown of Auxiliary Feedwater Pump No. 1, transferring the auxiliary steam supply from main steam to the auxiliary boiler, and transferring the GS supply from main steam to auxiliary steam. Shutdown of Main Feed Pump No. 1 was completed successfully. Restoration of the normal feed lineup to SG No. 1 and shutdown of Auxiliary Feedwater Pump No. 1 were complicated by the failure of SP7B to automatically control water in SG No. 1 but were completed with SG No. 1 water level being controlled manually with SP7B. Auxiliary steam was transferred to the auxiliary boiler after resolving issues with the tripping of one of the auxiliary boiler feed pumps. These activities were completed over a period of about eighty minutes (10:18 p.m. to approximately 11:40 p.m.)

At approximately 10:45 p.m., the crew began to transfer the GS supply from main steam to auxiliary steam using system operating procedure DB-OP-06205, Turbine Generator and Main Feedwater Pump Turbine Gland Steam and Turbine Drains. This process involved:

- Bypassing the Steam Seal Feed Control Valve, GS2338, by throttling open the Steam Seal Feed Bypass Valve, GS2385, until GS header pressure was being manually controlled at approximately 3 psig with GS2338 fully closed.

- Closing the Main Steam Seal Steam Supply Valve, GS2384.

- Opening Auxiliary Supply Steam Seal Valve, GS2380.

- Throttling closed GS2385 until GS2338 regained automatic control of GS header pressure at approximately 2.5 - 3.0 psig.

In the main control room, there were very limited indications for monitoring GS system operation during this transfer. These included a 0 - 10 psig GS header pressure indicator, an amber indicating lamp that is illuminated when GS2338 is not fully closed, a GS header low-pressure alarm that is set at 1.5 psig, and position indicating lamps for motor-operated valves GS2380, GS2384, and GS2385. At the start of the evolution, GS header pressure was being maintained at approximately 2.5 psig, as indicated on the 0 - 10 psig pressure indicator.

Procedure DB-OP-06205 contained a global precaution (applicable to the entire procedure),which cautioned the operator to take care when manually controlling GS header pressure using GS2385 to prevent over-pressurization of the GS system. In the sections of the procedure used to transfer GS system supply between main steam and auxiliary steam, there were also informational NOTES informing the operator that a slight rise in pressure would ensure that GS2385 would control the GS header pressure once GS2384 was closed. The procedure step controlling the initial operation of GS2385 directed the operator to throttle open the valve to obtain a GS header pressure of approximately 3.0 psig. There were also informational NOTES within these procedure sections informing the operator that it was desired to control GS header pressure low [< 3.5 psig] in the normal pressure band (2.5 - 4.5 psig) to prevent unnecessary operation of steam packing unloading valves, GS1938 and GS1939, and excessive loading on the auxiliary boiler. The unloading valves are designed to throttle open and closed in a staggered manner over a control range of 3.5 and 6.0 psig (with both valves fully closed at 3.5 psig and fully open at 6.0 psig). The unloading valves discharge to Low Pressure Feedwater Heaters, E1-1 and E1-2, when the main turbine is in operation and to the main condenser when the main turbine is shutdown.

Prior to the reactor trip on July 8, 2021, there were known equipment performance issues associated with GS2338, GS1938, and GS1939. These performance issues are discussed in more detail in the Charter Item No. 11 Assessment Section.

Additionally, there are four relief valves installed in the GS system to protect system components from over-pressurization. Two relief valves (set at approximately 120 psig) are installed on the GS feed bypass line (supplied by main steam) down stream of GS2385, and two relief valves (set at approximately 20 psig) are installed on the GS header itself. There are no direct indications in the main control room available to the crew to monitor relief valve status. Operation of the relief valves was typically indicated by reports of observations, from outside the main control room, indicating that steam was emanating from the relief valve exhaust line located on the roof of turbine building, or from the interconnected turbine building floor/equipment drain system. The Team noted that one or both 120 psig relief valves had a history of operating when GS2385 was initially operated during the transfer of the steam supplies between main steam and auxiliary steam. Following operation of the 120 psig relief valve(s) in 2011, DB-OP-06205 was revised to:

- Add a transfer section prerequisite to ensure that the GS header pressure was being maintained between 2.5 and 2.7 psig.

- Add the informational NOTEs (previously mentioned), related to the slight pressure rise, to ensure the ability to use GS2385 for GS header pressure control.

- Lower the initial target pressure from 4.5 psig to 3.0 psig.

Members of the crew, with a particular focus on the personnel who performed the GS supply transfer, were interviewed by the Team to assess the crews performance and system response during the transfer evolution. The member of the crew tasked with the transfer was an experienced licensed reactor operator who had previously performed the transfer evolution several times. The Team confirmed that the crew was implementing Section 3.6, Transfer Gland Steam from Main Steam to Aux Steam, of procedure DB-OP-06205. The Team confirmed that the prerequisites for performing the transfer evolution were verified to be satisfied and that the crew had ensured that condensate was drained from the auxiliary steam supply header prior to opening GS2385. Team interviews with the crew confirmed that GS2385 was throttled open by repeated operation of the momentary-contact OPEN pushbutton. Through crew interviews, the Team confirmed that when GS2385 positioned off its closed seat that the GS header pressure indication spiked to about 3.5 psig then lowered to near the starting pressure. Momentary operation of the OPEN pushbutton was then apparently repeated several times with similar spikes in the pressure indication, with pressure stabilizing at a slightly higher pressure each time. These indicated pressure transients were consistent with the control action of the steam packing unloading valves each time GS header pressure rose above the 3.5 psig setpoint then reaching an equilibrium GS2385 was fully opened. The unloader valves discharge to the main condenser and the steam flow through them is the most likely explanation for the SG2 depressurization.

Procedure DB-OP-06205 directed GS2385 to be throttled open to obtain approximately 3.0 psig, as indicated on the GS header pressure indicator, or until the GS2338 amber indicating lamp was extinguished, indicating that GS2338 was fully closed. During the crew interviews, the Team determined that it was typical, based on the previous experience of the crew, that the GS2338 amber indicating lamp was expected to be extinguished after the first few manipulations of the GS2385 OPEN pushbutton. Crew personnel went on to explain to the Team that while they had expected the amber indicating light to have extinguished, they assumed that due to the lower-than-normal main steam pressure that continued operation of the OPEN pushbutton was necessary to achieve the condition necessary for GS2338 to close. Rather than pausing to investigate why GS2338 was not responding as expected, crew personnel assigned to perform the transfer evolution continued to operate the OPEN pushbutton until GS2385 was fully open. The GS2338 amber indicating lamp remained energized and was never extinguished, since the GS2338 had become stuck in a mid-position as discussed in more detail in the Charter Item No. 11 Assessment Section.

GS header pressure had been raised by approximately 0.5 psig to approximately 3 psig, high enough that GS2338 should have been closed. Pressure continued to spike with each operation of the GS2385 OPEN pushbutton, stabilizing at a slightly higher pressure each time. While the crew stated that they could not recall having observed the pressure indicator reading any higher than 3.5 psig during the transfer evolution, the Team determined through review of plant process computer data that pressure spikes as high as approximately 4.2 psig had been experienced by the system at the GS2338 valve controller. GS header pressure eventually stabilized at approximately 3.0 - 3.2 psig, most probably under control of steam packing unloading valves, GS1938 and GS1939.

After GS2385 had been fully opened, the crew observed that SG pressure was lowering in both SGs and the operator performing the transfer of the GS supply from main steam to auxiliary steam was directed to assist in response to the lowering SG pressures. At 11:50 p.m., the crew reentered DB-OP-02000 and again routed to the Overcooling Section of the EOP. The crew reestablished HPI/LPI/MU injection and began looking for the source of the overcooling. At 11:55 p.m., the crew took manual control and throttled auxiliary feedwater flow to SG No. 2 to minimize cooling from the cold-water injection; the action had minimal impact on lowering the rate of pressure decrease. At 11:56 p.m., reports were received from both security personnel and crew field operators that steam was exhausting from GS system relief valve vent pipe and observed in the lower level of the turbine building from the interconnected turbine building floor/equipment drain system. Approximately two minutes later, at 11:58 p.m., SFRCS Actuation Channel No. 2 actuated on low-pressure in SG No. 2 before any additional action could be taken to reduce the rate of pressure decrease in the SGs. This action resulted in the isolation of both main steam lines and the associated loss of the main condenser as a heat sink, isolation of main feedwater to both SGs, and the isolation of auxiliary feedwater injection to SG No. 2.

Initially, the depressurization of SG No. 2 was attributed to operation of one or more GS system relief valves. While operation of the relief valves may have contributed to depressurization of SG No. 2, the Team concluded from review of post trip data that operation of one or both steam packing unloading valves was probably the main contributor to the depressurization of SG No. 2 and the resulting low-pressure SFRCS isolation. The Team concluded that the decrease in SG No. 1 pressure was due primarily to the overcooling of the reactor coolant system.

The Team concluded that a contributing factor to the uncontrolled depressurization of SG No. 2 was a failure on the part of the licensees staff, both the on-watch crew as well as other site personnel, to fully understand the flow dynamics of the GS system that could cause operation of the 120 psig relief valves when transferring GS between main steam and auxiliary steam using the Steam Seal Feed Bypass Valve, GS2385. Interviews the Team conducted clearly indicated that some personnel did not have a solid understanding of the interrelated operation of the Steam Seal Feed Bypass Valve, GS2385, the Steam Seal Feed Control Valve, GS2338, the steam packing unloading valves, GS1938 and GS1939, and the four relief valves installed in the GS system to protect system components from over-pressurization.

The Team further concluded that the failure of the operator performing the GS supply transfer to more thoroughly question and/or investigate the response of GS2338, while continuing to open GS2385 when the expected system response was not obtained, was a significant contributor to the resultant uncontrolled depressurization of SG No. 2. Additionally, the Team concluded that the lack of procedure guidance, given the limited ability to monitor operation of the GS system from the main control room, was also a significant contributor to the uncontrolled depressurization of SG No. 2. Both of these factors are discussed in additional detail in a finding and associated NCV that is documented in the Results Section of this report.

Additional contributors to the uncontrolled depressurization of SG No. 2 that were identified by the Team involved the following equipment issues:

- The failure of GS2338 to close upon demand.

- Preexisting unresolved equipment issues associated with the Steam Seal Feed Control Valve, GS2338, and the steam packing unloading valves, GS1938 and GS1939.

As noted earlier, these issues are discussed in more detail in the Charter Item No. 11 Assessment Section.

Failure to Appropriately Classify the Digital Electrohydraulic Control Uninterruptible Power Supply Battery Bank Cornerstone Significance Cross-Cutting Report Aspect Section Initiating Events Green [H.3] - Change 93812 FIN 05000346/2021050-01 Management Open/Closed A self-revealed Green finding of very low safety significance was identified for the licensees failure to appropriately classify the digital electrohydraulic Control (DEHC) uninterruptible power supply (UPS) battery bank as Non-Critical as required by component classification procedures NOP-ER-1001 and NOBP-ER-3901. Specifically, when the licensee decided to use the DEHC UPS batteries as the sole source of control power for the automatic voltage regulator (AVR) automatic transfer switch (ATS), they failed to upgrade the classification of the battery bank from Non-Impact to Non-Critical as required by the component classification procedures, since the battery banks failure could now induce a critical component (e.g.,

the AVR) failure as was seen during the July 8, 2021, reactor trip.

Description:

The DEHC UPS battery bank (D3201B) is a back-up power supply for one of the two redundant power supplies to the stations DEHC loads. It backs up the DEHC power supply coming from MCC E32A, while the redundant DEHC power supply comes from MCC F32A.

It was installed during the 2014 refueling outage as part of engineering change package (ECP) 05-0140. The DEHC UPS is designed to power the DEHC loads for one hour in the event of a normal loss of power from both redundant power supplies (such as during a loss of offsite power event) to allow sufficient time for the turbine to slow to turning gear speed. The licensee had originally classified the DEHC UPS battery bank as a Non-Impact component per NOBP-EP-3901, Component Classification ER Workbench Module 1, Revision 5, because it was considered a third power source for the DEHC loads and not required for plant operations. The Non-Impact designation allowed the UPS battery bank to have no preventive maintenance activities associated with it to enhance its reliability.

During the same outage that the DEHC UPS was installed, the licensee installed a new AVR as part of ECP 12-0134. During the modification process, the licensee decided to use the DEHC UPS as the sole source of control power for the AVR ATS, effectively adding an additional function to the DEHC UPS battery bank. The AVR ATS needs control power to swap the AVR between its redundant power supplies coming from MCC E32A and MCC F33A. During the July 8, 2021, reactor trip, the AVR was being powered by MCC E32.

When MCC E32 lost power, the AVR ATS should have swapped the AVR to its redundant power supply (MCC F33A). However, because the DEHC UPS battery bank failed, the AVR ATS lost control power and was unable to swap the AVR to its redundant power supply.

The AVR therefore lost all power, causing a turbine trip and subsequent reactor trip.

When the licensee decided to use the DEHC UPS as the sole source of control power for the AVR ATS, they failed to properly evaluate the design interface created between the DEHC UPS and the AVR ATS, as documented in CR-2021-0520, Turbine/Reactor Trip on 7/8/2021. Specifically, the licensee failed to identify the DEHC UPS battery bank function of supplying control power to the AVR ATS as an important function to be evaluated for the component in accordance with Step 4.6.1 of NOP-ER-1001, Continuous Equipment Performance Improvement. This led to the misclassification of the DEHC UPS battery bank as a Non-Impact component instead of a higher importance Non-Critical component, as required by Step 4.7 of NOP-ER-1001 and Step 3.4.3 of NOBP-ER-3901. Specifically, the failure of the DEHC UPS battery bank could induce the failure of the AVR, a critical component, as seen during the July 8, 2021, reactor trip. A Non-Critical component classification would have required the licensee to develop preventive maintenance activities for the DEHC UPS battery bank to enhance its reliability.

Corrective Actions: The licensee entered the issue into their corrective action program and replaced the DEHC UPS defective battery cell under WO 200858529 on July 12, 2021, to restore the functionality of the DEHC UPS battery bank as an immediate corrective action.

They also reclassified the DEHC UPS battery bank (D3201B) from Non-Impact to Non-Critical, as required by NOP-ER-1001 and NOBP-ER-3901, on July 27, 2021. The licensee has plans to create preventive maintenance activities for the DEHC UPS battery bank to enhance its reliability which include the creation of periodic testing for the battery bank as well as scheduled replacements for the battery based on vendor recommended intervals.

Corrective Action References:

CR-2021-05220, DEHC D3201A UPS Failed to Power DEHC on Loss of E32A [7/8/21 Reactor Trip,] 7/9/2021 CR-2021-05250, Turbine/Reactor Trip on 7/8/2021, 7/9/2021 CR-2021-05319, Defective Battery Cell Identified at D3201B During Troubleshooting of DEHC UPS [7/9/2021] Reactor Trip], 7/12/2021

Performance Assessment:

Performance Deficiency: The licensees failure to appropriately classify the DEHC UPS battery bank as Non-Critical was contrary to licensee component classification procedures NOP-ER-1001 and NOBP-ER-3901, and constituted a performance deficiency. Specifically, when the licensee decided to use the DEHC UPS batteries as the sole source of control power for the AVR ATS, they failed to upgrade the classification of the battery bank from Non-Impact to Non-Critical as required by the component classification procedures since the battery banks failure could now induce a critical component (AVR) failure as was seen during the July 8, 2021, reactor trip.

Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Equipment Performance attribute of the Initiating Events cornerstone and adversely affected the cornerstone objective to limit the likelihood of events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the incorrect classification of the DEHC UPS battery bank as Non-Impact allowed the licensee to perform no preventive maintenance activities for the battery bank to enhance its reliability. Its failure directly contributed to the July 8, 2021, reactor trip. The Team also compared the finding with the examples listed in IMC 0612, Power Reactor Inspection Reports, Appendix E, Examples of Minor Issues. Example 8.c was found to be similar in that a failure to properly scope components adversely affected cornerstone objectives because effective control of equipment performance was not demonstrated.

Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Using Exhibit 1, Initiating Events Screening Questions, Section B, Transient Initiators, the Team was able to answer no to the question and determined the finding to be of very low safety significance (Green).

Specifically, although the finding caused a reactor trip, it did not result in the loss of mitigation equipment relied upon to transition the plant from the onset of the trip to a stable shutdown condition.

Cross-Cutting Aspect: H.3 - Change Management: Leaders use a systematic process for evaluating and implementing change so that nuclear safety remains the overriding priority.

Specifically, when the licensee decided to use the DEHC UPS as the sole source of control power for the AVR ATS in ECP 12-0134, they failed to properly evaluate the design interface created between the DEHC UPS and the AVR ATS. As documented in CR-2021-0520, Turbine/Reactor Trip on 7/8/2021, this weakness in the design change process still exists.

Enforcement:

Inspectors did not identify a violation of regulatory requirements associated with this finding.

Failure to Provide Procedural Instructions for Transferring Gland Sealing Steam from Main Steam to Auxiliary Steam Following a Reactor Trip Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green [H.11] - 93812 Systems NCV 05000346/2021050-02 Challenge the Open/Closed Unknown A self-revealed finding of very low safety significance (Green) and an associated NCV of Technical Specification (TS) 5.4.1(a) were identified for the licensees failure to establish and implement procedural guidance for transferring the gland sealing steam supply from the main steam system to the auxiliary steam system following a reactor trip. Specifically, the guidance in licensee procedure DB-OP-06205, Turbine Generator and Main Feedwater Pump Turbine Gland Steam and Turbine Drains, for transferring the gland steam header supply from the main steam header to the auxiliary steam header did not provide instructions to ensure that the pressure in the gland steam header would be maintained low enough so as not to produce an excessive steam demand on the main steam system during the transfer.

This lack of procedural guidance allowed control room operators to fully open valve GS2385, Steam Seal Feed Bypass, pressurizing the gland steam header above the normal control range, which led to an excessive steam demand on the main steam system, a low-pressure condition in SG No. 2, and a steam and feedwater rupture control system (SFRCS)actuation.

Description:

As discussed in the Assessment Section for Charter Item No. 10, with the reactor shutdown and in the hot standby condition on July 8, 2021, at approximately 11:58 p.m., the SFRCS actuated as designed on low SG pressure indication for SG No. 2 coincident with control room operators trying to transfer the gland sealing steam supply from the main steam system to the auxiliary steam system following an earlier reactor trip.

Inspection by the Team following the event revealed that the licensees procedural guidance in DB-OP-06205 for transferring the gland steam header supply from the main steam header to the auxiliary steam header did not provide instructions to ensure that the pressure in the gland steam header would be maintained low enough so as not to produce an excessive steam demand on the main steam system during the transfer. The procedure for the transfer involves the control room operator gradually opening motor-operated valve GS2385, Steam Seal Feed Bypass, to force valve GS2338, Steam Seal Feed Valve Controller, to automatically close. At that point, the pressure in the gland steam header is under manual control by the control room operator, via throttling of GS2385 as needed, and the control room operator can transfer the steam supply from the main steam system to the auxiliary steam system. Motor-operated valve GS2385 is then closed in a controlled manner to allow GS2338 to automatically open back up and control gland steam header pressure.

As can be seen in the photograph above, the controls and indications available to the operator in the main control room for this transfer procedure are rudimentary (indications in the picture show the normal operating lineup in Mode 1). The only indication afforded to the operator to provide a status on the position of the GS2338 valve is an amber light that simply indicates that the valve is not closed when illuminated. DB-OP-06205 directs the control room operator to throttle open GS2385 to establish approximately 3 psig in the gland steam header, or until GS2338 indicates closed [amber light extinguished]. During the transfer of the gland sealing steam supply from the main steam system to the auxiliary steam system following the July 8, 2021, reactor trip, the GS2338 valve did not reposition closed as expected when the control room operator began to slowly open motor-operated valve GS2385. Even though the gland steam header pressure, as indicated in the main control room, was high enough to have forced close GS2338, the control room operator incorrectly interpreted the failure of the amber light to extinguish to be due to a reduced main steam header pressure following the reactor trip and subsequent reactor coolant system cooldown. Absent any notes, cautions, or other guidance in DB-OP-06205 to the contrary and unable to ascertain that the GS2338 valve was stuck partially open, the control room operator continued to open motor-operated valve GS2385 until it was fully open. The ensuing steam demand on the main steam system decreased SG No. 2 steam pressure until the setpoint was reached on SFRCS Actuation Channel No. 2 and an SFRCS initiation and isolation occurred. This caused the main feedwater supply to both SGs to be isolated and both main steam isolation valves (MSIVs) to close, which effectively ended the excessive steam demand transient.

Corrective Actions: Following interactions with the Team, the licensee recognized that the rudimentary controls and indications associated with the transfer of gland sealing steam from the main steam system to the auxiliary steam system placed the control room operator in a difficult situation, unaware of certain component responses in the actual plant power block. In response to this, the licensee established an operations standing order (No.21-003, Gland Steam - Local Operator During GS2385 Transfers, dated 8/6/2021) to station an equipment operator in the plant to locally observe the actual components involved in the transfer and report back to the control room operator as necessary. The standing order is slated to remain in effect until the licensee can revise the applicable procedure, DB-OP-06205, to incorporate the new guidance.

Corrective Action References:

CR-2021-05330, GS2338 Not Responding. Stuck OPEN @ 60% [Reactor Trip 7/8/2021]; 07/12/2021 CR-2021-05494, Gland Steam Relief Lift During Transferring of Gland Steam from Main Steam to Aux Steam [Reactor Trip 7/8/2021]; 07/19/2021

Performance Assessment:

Performance Deficiency: The Team reviewed this finding using the guidance contained in Appendix B, Issue Screening Directions, of IMC 0612, Issue Screening. The Team determined that the licensees failure to have provided adequate procedural instructions to plant operators for the transfer of gland sealing steam from the main steam system to the auxiliary steam system following a reactor trip constituted a performance deficiency that was reasonably within the licensees ability to foresee and correct and should have been prevented.

Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Procedure Quality attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences.

Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At-Power. The Team determined that the finding had resulted in the loss of function for non-Technical Specification mitigating systems (i.e., the main feedwater system and the main condenser), and using Exhibit 2-Mitigating Systems Screening Questions, determined that the finding was of very low safety significance (Green). Specifically, because the non-Technical Specification mitigating systems involved had their functionality restored within 24-hours, the finding screened directly to Green without the need for a detailed risk evaluation. The Team determined that the main feedwater system was recovered by 4:08 a.m. on July 9, 2021.

Both MSIVs could have been opened and the main condenser recovered for use as a heat sink, by procedure, at the same time. However, the licensee elected to maintain the valves closed to support various maintenance activities. The MSIV for Main Steam Line No. 2 (MS100) was subsequently opened at 1:35 a.m. on July 10, 2021, and the MSIV for Main Steam Line No. 1 (MS101) was subsequently opened at 2:06 a.m. on July 10, 2021.

Cross-Cutting Aspect: H.11 - Challenge the Unknown: Individuals stop when faced with uncertain conditions. Risks are evaluated and managed before proceeding. Specifically, the control room operator performing the transfer of gland sealing steam from the main steam system to the auxiliary steam system following the reactor trip interpreted the failure of the GS2338 valve to indicate that it had closed and the need for motor-operated valve GS2385 to continue to be opened as being due to a reduced main steam header pressure following the reactor trip and subsequent reactor coolant system cooldown. This was, in fact, an incorrect assumption, as the GS2338 valve has become stuck in mid-position.

Enforcement:

Violation: Technical Specification 5.4.1(a) requires the licensee to establish, implement, and maintain applicable written procedures for the safety-related systems and activities recommended in Regulatory Guide (RG) 1.33, Revision 2, Appendix A, February 1978.

Section 2(c) of RG 1.33, Revision 2, Appendix A, requires procedures for the recovery of the plant from a reactor trip. Similarly, Section 3(i) of RG 1.33, Revision 2, Appendix A, requires procedures governing the proper operation of the main steam system. Contrary to these requirements, the licensee failed to properly prepare and implement technically adequate written procedures for the transfer of gland sealing steam from the main steam system to the auxiliary steam system following a reactor trip, such that the on-watch control room crew lacked sufficient procedural guidance to effectively transfer gland sealing steam from the main steam system to the auxiliary steam system on July 8, 2021, and an SFRCS actuation ensued.

Davis-Besse procedure DB-OP-06205 provides instructions for the transfer of gland sealing steam supply from the main steam system to the auxiliary steam system following a reactor trip.

Contrary to the above, on July 8, 2021, the licensee failed to establish, implement, and maintain applicable written procedures for the transfer of gland sealing steam supply from the main steam system to the auxiliary steam system following a reactor trip. Specifically, licensee procedure DB-OP-06205 did not provide guidance or instructions to limit the excessive demand on the main steam system and the subsequent decrease of steam generator No. 2 pressure to the low pressure setpoint.

Enforcement Action: This violation is being treated as a non-cited violation, consistent with Section 2.3.2 of the Enforcement Policy.

Failure to Install Correct Part for Limit Switch ZS101B Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green None 93812 Systems NCV 05000346/2021050-03 Open/Closed A self-revealed Green finding of very low safety significance and an associated non-cited violation (NCV) of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, were identified for the licensees failure to have a procedure of a type appropriate to the circumstances for the replacement of Main Steam Line No. 1 Isolation Valve Limit Switch ZS101B. Specifically, in 2008, the licensee failed to include appropriate instructions in safety-related Work Order (WO) 200205458 to ensure Limit Switch ZS101B was replaced with the correct part. As a result, the wrong part was used when replacing Limit Switch ZS101B.

Description:

Subsequent to the reactor trip on July 8, 2021, an SFRCS initiation caused the main steam line isolation valves to close. As described in Charter Items No. 10 and No. 11, the AVVs should have automatically operated to control SG pressures and remove decay heat following the isolation of the main steam lines. However, the crew had to take manual control of the AVVs to open them from the control room because they failed to automatically open to perform their decay heat removal function. This added an additional strain on the already limited resources of the crew, given they were dealing with other complications from the trip.

As discussed in the Updated Safety Analysis Report (USAR), Section 7.7.1.2.9, when either of the two MSIVs are less than 90% open (one out of two logic), the ICS is supposed to transfer SG pressure control from the turbine bypass valves to the AVVs. The AVVs failed to automatically open following the isolation of the main steam lines because the ICS logic for transferring pressure control to the AVVs was not completed. During troubleshooting activities, the licensee determined the two limit switches providing position indication for the MSIVs to the ICS had failed to perform their function. The actuating arm for Limit Switch ZS100B (associated with MSIV MS100) fell off the switch due to the loosening of its set screw, as described in CR-2021-05279, causing it to incorrectly indicate an open MSIV position. The other limit switch, ZS101B (associated with MSIV MS101) was found to be the incorrect part for the application, as described in CR-2021-05271, causing it to also indicate an incorrect open MSIV position. The incorrect part (86900806) used for Limit Switch ZS101B is a spring-return limit switch. Although it momentarily indicated a closed valve position, it then returned back (spring-returned) to indicate an open valve position as the MSIV continued going closed. The correct part (86901006) does not have a spring-return and would have continued indicating a closed valve position as the MSIV continued going closed.

Limit Switch ZS101B was installed in 2008 under safety-related WO 200205458, Main Steam Line 1 Isolation Valve Position. The WO replaced three of the five limit switches (ZS101A, ZS101B, and ZS101C) associated with MSIV MS101. Although the WO problem description identified the correct switch part number for ZS101B (86901006), the instructions provided as part of Operation 0040 of the WO, Replace switches ZS101A/B/C, did not specify which part numbers should be installed for each specific switch location. The WO had a detailed parts list, but it relied on the technicians performing the work to identify the correct replacement parts for each limit switch location by comparing them to the installed parts in the field.

The instructions simply stated, Install replacement switches in the proper location with original mounting hardware. Contact planning for additional parts. The work-in-progress (WIP) log of the WO, dated January 10, 2008, documents the confusion of the workers performing the replacement of the limit switches because it was not clear which switch part numbers were supposed to go in each of the switch positions. Although the technicians stopped the work to resolve the confusion, as documented in the WIP log, the licensee did not add additional information to the WO to specify which switch part numbers were supposed to go in each of the switch positions. The Team concluded the instructions provided in safety-related WO 200205458 were not appropriate to the circumstances because they led to the installation of an incorrect part for Limit Switch ZS101B in 2008.

Corrective Actions: The licensee entered the issue into their corrective action program and has plans to install the correct part for Limit Switch ZS101B during the next scheduled refueling outage. Since the licensee has repaired the ZS100B switch (the other failed limit switch for MSIV MS100), the licensee expects the ICS logic to appropriately transfer SG pressure control from the turbine bypass valves to the AVVs on a future closure of the MSIVs (due to the 1 out of 2 logic required for this to occur).

Corrective Action References:

CR-2021-05271, Incorrect Switch Installed at ZS101B, 7/10/2021 CR-2021-05279, ZS100B Found Degraded, 7/10/2021 CR-2021-05729, Operational Impact for Not Replacing ZS101B, 7/28/2021

Performance Assessment:

Performance Deficiency: The failure to have a procedure of a type appropriate to the circumstances for the replacement of MSIV MS101 Limit Switch ZS101B was contrary to 10 CFR 50 Appendix B, Criterion V, and was a performance deficiency. Specifically, the licensee failed to include appropriate instructions in safety-related WO 200205458 to ensure Limit Switch ZS101B was replaced with the correct part. As a result, the wrong part was used when replacing Limit Switch ZS101B in 2008.

Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Procedure Quality attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, the failure to provide appropriate instructions when replacing Limit Switch ZS101B to ensure it was replaced with the correct part did not ensure the capability of the AVVs to automatically remove decay heat when responding to initiating events to prevent undesirable consequences. During the July 8, 2021, reactor trip and the subsequent isolation of the main steam lines, additional control room resources had to be utilized to manually open the AVVs from the control room to provide a decay heat removal path. The Team also compared the finding with the examples listed in IMC 0612, Power Reactor Inspection Reports, Appendix E, Examples of Minor Issues. Example 4.m was found to be similar in that a failure to include necessary information in installation procedures adversely impacted the cornerstone objective and ultimately resulted in the AVVs being unable to automatically remove decay heat.

Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Using Exhibit 2, Mitigating Systems Screening Questions, the Team was able to answer no to all questions and determined the finding to be of very low safety significance (Green). Specifically, the degraded condition of the non-Technical Specification mitigating system did not represent a loss of a PRA system and/or function as defined in the Plant Risk Information e-Book (PRIB)or the licensees PRA.

Cross-Cutting Aspect: None The Team determined that the finding, having stemmed from a WO that was executed in 2008, was not indicative of current licensee performance.

Enforcement:

Violation: 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, requires, in part, that activities affecting quality be prescribed by documented procedures of a type appropriate to the circumstances and be accomplished in accordance with these procedures. The licensee established safety-related WO 200205458, Main Steam Line 1 Isolation Valve Position, as the implementing instructions for replacing the MSIV MS101 Limit Switch ZS101B, an activity affecting quality.

Contrary to the above, from January 10, 2008, through January 29, 2008, the licensee failed to have a procedure of a type appropriate to the circumstances for the replacement of Limit Switch ZS101B. Specifically, the licensee failed to include appropriate instructions in safety-related WO 200205458 to ensure Limit Switch ZS101B was replaced with the correct part.

Enforcement Action: This violation is being treated as a non-cited violation, consistent with Section 2.3.2 of the Enforcement Policy.

Failure to Inspect Emergency Diesel Generator Selector Switch Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Pending None 93812 Systems AV 05000346/2021050-04 Open On May 27, 2021, a self-revealed finding with its safety significance as yet to be determined (TBD) and an associated Apparent Violation (AV) of TS 5.4.1.a were identified for the licensees apparent failure to develop a preventive maintenance schedule for the inspection of emergency diesel generator (EDG) field flash selector switches (FFSSs).

Description:

On May 27, 2021, the Division 1 EDG failed to reach required voltage and frequency during surveillance testing. Troubleshooting efforts by the licensee found that manual switch contacts in the field flash circuit path had fouling present that prohibited the current flow necessary to flash the generator field during the EDG starting sequence. The safety-related EDGs use current from the stations 125 Vdc system to supply the generator field during the starting sequence. Once the current supplied by the generator is adequate to maintain the EDG field, a separate contact opens and stops the current flow from the 125 Vdc system.

The licensees investigation found that the FFSS contacts were visible without disassembling the switch and the contacts appeared fouled. The licensee noted that the vendor manual for the General Electric Type SBM control switch specifically called out performing a periodic cleaning of the contacts. The licensees assessment of the FFSS found that no preventive maintenance activities were being performed, and a review of industry operating experience identified three examples that described similar fouling issues. The solution described in the operating experience examples was to clean the switch contacts per the vendor recommendations.

The Team reviewed the Maintenance Servicing Section for contact cleaning in the GE Type SBM vendor instructions, and found that it stated, At regular intervals, the switch contacts should be inspected for wear and burning. An opening at the bottom of the switch has been provided for this. (See Figure 3) If contacts are slightly pitted or coated with sulphide, they should be cleaned with a flexible burnishing tool similar to that included in the XRT relay tool kit.

The licensees extent-of-condition review was limited to similar EDG switches that would not give indication of contact position through equipment operation or indicating lights. The review only identified the FFSS on the Division 2 EDG as an additional candidate for preventive maintenance. The licensee initiated actions to replace the Division EDG switch.

The Team concluded that the licensees extent-of-condition appeared narrowly scoped since it only covered switches associated with the EDGs. The Team recalled that the 2006 modification, which had installed the GE Type SBM switch for the FFSS, also installed another GE Type SBM switch in the stations 4160 Volts alternating current (Vac) switchgear.

Specifically, the licensee added a test switch labeled DST, which is used to test a time delay relay associated with the starting of the stations makeup pumps after a loss of offsite power (LOOP) coincident with a safety features actuation system (SFAS) signal. The DST test switch disconnects the time delay relay from its normal circuit and aligns it to test jacks for testing. The switch is restored after testing to ensure the makeup pump will start when needed. The failure of the contacts on the DST would prevent the proper operation of a makeup pump after a LOOP coincident with an SFAS signal. The makeup and purification system supply water to the reactor coolant system, circulates seal water for the reactor coolant pumps, and accommodates temporary changes in the required reactor coolant inventory. Following inquiries by the Team as to whether or not this switch was operated under similar conditions as the FFSS, the licensee concluded that it was likely vulnerable to the same failures as the FFSS. The was entered into the stations corrective action program (CAP) and the licensee intends to inspect the DST switch during the next refueling outage.

Corrective Actions: The licensee replaced the FFSS on both the Division 1 and Division 2 EDGs, and added steps in the stations 31-day EDG surveillance test to check the continuity of the FFSS contacts after restoring the switch to the 400 rpm position following the completion of testing.

Corrective Action References:

CR 2021-04282, EDG No. 1 Failed to Reach Required Voltage and Frequency During the 184-Day Test CR 2021-06146, 2021 NRC SIT: Switch DST For Makeup Pump Time Delay Relay 2Y1 Isolation Potentially Exposed to Similar Degradation Mechanisms as EDG FFSS

Performance Assessment:

Performance Deficiency: The Team concluded that the failure to develop a preventive maintenance schedule for the inspection of the FFSS, was contrary to Technical Specification 5.4.1 and Section 9.b of Regulatory Guide 1.33, Revision 2, February 1978, and constituted a performance deficiency.

Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Equipment Performance attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. consequences (i.e., core damage). Specifically, the Team concluded that not having a preventive maintenance schedule to inspect the FFSS contacts adversely affected the capability and reliability of systems that respond to initiating events to prevent undesirable consequences. The Team also compared the finding with the examples listed in IMC 0612, Power Reactor Inspection Reports, Appendix E, Examples of Minor Issues. Example 4.k was found to be similar in that a failure to properly test components adversely affected cornerstone objectives because the component was out of specification and adversely affected the equipments availability, reliability, and/or capability.

Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At-Power. The Team concluded that because the finding could not be easily screened to be of very low safety significance, a detailed risk evaluation was required. The FFSS 400 rpm contacts are only tested every 184 days and were last tested successfully in November 2020; therefore, the Team answered yes to Mitigating Systems Screening Question A.3, Does the degraded condition represent a loss of the PRA function of one train of a multi-train TS system for greater than its TS allowed outage time?

Cross-Cutting Aspect: None Because the finding stemmed from the work done by the licensee during a modification that was accomplished in 2006, the Team concluded that the assignment of a cross-cutting aspect to the finding was not appropriate, as discussed in NRC IMC 0612, Appendix B, Issue Screening.

Enforcement:

Apparent Violation: Davis-Besse Technical Specification 5.4.1(a) requires the licensee to establish, implement, and maintain applicable written procedures for the safety-related systems and activities recommended in Regulatory Guide (RG) 1.33, Revision 2, Appendix A, February 1978.

Section 3 of Regulatory Guide 1.33, Appendix A, Revision 2, February 1978, states, in part, that instructions for energizing, filling, venting, draining, startup, shutdown, and changing modes of operation should be prepared, as appropriate, for the following systems: s.2.1)

Emergency Power Sources (e.g., diesel generator, batteries).

Section 9.b of Regulatory Guide 1.33, Appendix A, Revision 2, February 1978, states, in part that preventive maintenance schedules should be developed to specify lubrication schedules, inspections of equipment, replacement of such items as filters and strainers, and inspection or replacement of parts that have a specific lifetime such as wear rings.

From April 2006 until May 27, 2021, the licensee apparently failed to establish and implement written procedures covering the preventive maintenance schedules for the inspection of equipment. Specifically, the licensee apparently failed to provide a preventive maintenance schedule for inspection of the field flash selector switch contacts for the Division 1 and Division 2 safety-related EDGs.

Enforcement Action: This issue is being treated as an apparent violation pending a final significance (enforcement) determination.

Failure to Install Emergency Diesel Generator Parts That Were Suitable to the Application Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Pending [H.6] - Design 93812 Systems AV 05000346/2021050-05 Margins Open On September 4, 2020, a self-revealed finding with its safety significance as yet TBD and an associated AV of 10 CFR Part 50, Appendix B, Criterion III, Design Control, and TS 3.8.1, Conditions B.1 and B.3, were identified for the licensees apparent failure to select an EDG speed switch replacement that was suitable to the application. This resulted in the unavailability and inoperability of the Division 2 EDG when it was being relied upon for plant safety.

Description:

On September 4, 2020, with the Davis-Besse Nuclear Power Station operating at approximately 100 percent power, EDG No. 2 failed to start during a scheduled monthly surveillance test. A failed speed switch was identified and replaced, restoring the EDG to operable status on September 7, 2020.

The licensees troubleshooting efforts found that the speed switch, which was a new design, had thermal damage to multiple internal components and had failed after only ten months in service. The licensee concluded, based on external vendor analysis and a review of applicable plant data, that an individual component on the speed switch, either a capacitor or metal oxide varistor (MOV), prematurely failed creating a positive ground on the stations DC system. The licensees review also found that a neutral ground existed on the High-Pressure Injection (HPI) Pump No. 2 DC oil pump motor, which was powered from the same DC bus, during a timeframe that overlapped with the positive speed switch ground.

Further, the licensee found that the positive ground was logged on DC Motor Control Center (MCC) No. 2 on August 25, 2020, and that the neutral bus DC ground was repaired on September 1, 2020. The combination of the two grounds yielded the explanation for why the speed switch was damaged. Specifically, in an ungrounded DC system a single ground will not cause component damage. The DC system is designed so two grounds are required before equipment damage will occur.

The licensee observed that all hard grounds on DC MCC No. 2 were clear on September 1, 2020, after the HPI Pump No. 2 DC oil pump breaker was opened to commence repairs. The licensee concluded that the speed switch failure must have occurred on or before September 1, 2020, and, without firm evidence pointing to a specific failure date, the licensee selected September 1, 2020, as the date when EDG No. 2 was considered inoperable.

On July 26, 2021, the licensee determined that the speed switch design was not compatible with the stations 125/250 Vdc system and the component failure was induced by this design vulnerability. Specifically, the stations DC system is a combination 125/250 Vdc system and, while the voltage potential between the positive and neutral buses is approximately 125 Vdc, the voltage potential between the ground and the positive bus was 201 Vdc. The switch design contained an MOV that was only rated for 170 Vdc, which was connected between the positive bus and ground. The MOV failed from long-term exposure to a voltage potential in excess of its voltage rating.

The Teams review of the operations ground readings found that a hard positive ground came in on August 25, 2020, but shifted between log readings on August 28th at 8:00 p.m. and August 29th at 8:00 a.m., to a hard neutral ground or a reduced positive ground. The Team concluded that the second hard ground (neutral bus) came in between log readings and allowed excessive current to flow through the speed switch and cause the thermal damage, which cleared the hard positive ground. This left the hard neutral ground, which was cleared on September 1, 2020, when the HPI No. 2 DC oil pump breaker was opened to facilitate repairs.

The Team reviewed licensee procedure NOP-CC-2007, Part/Component Equivalent Replacement Packages, Revision 3, and the licensees equivalency evaluation PERP 1226, Replacement of the EDG Speed Transmitters, ST6221 (EDG No. 1) and ST6231 (EDG No. 2), Revisions 4, 5 and 6. Licensee procedure NOP-CC-2007, Revision 3, had been in effect since March 28, 2011, and was the current revision during the inspection.

Equivalency evaluation PERP 1226 discussed two primary reasons for replacing the EDG speed switches. One reason was speed switch obsolescence, while the second was that operating experience had shown that the previous speed switches were susceptible to electromagnetic interference (EMI) and/or radio frequency interference (RFI). The new speed switches were designed to be protected from EMI and RFI, and were qualified as electromagnetically compatible (EMC) in accordance with EPRI TR-102323, Guidelines for Electromagnetic Compatibility Testing of Power Plant Equipment, Revision 4.

Licensee equivalency evaluation PERP 1226, was completed using licensee Form NOP-CC-2007-01, Part/Component Equivalent Replacement Packages. The Team found that NOP-CC-2007-01,Section IV. Evaluation, Item B, Form, stated:

Does the replacement component/part have any critical characteristics (e.g., material, weight, power requirements, brake horsepower, voltage ranges of operation, etc.) that are different from the original? Consider the replacement items conformance to the original material(s) of construction, structure, internal configuration, specifications, and standards of the original item. Consider the interface to plant design. State the basis and, if applicable, document the differences.

The Team found that the licensee listed input voltage as an applicable characteristic under this area of the evaluation form, but considered the old switch voltage requirements and the replacement switch voltage requirements to be identical. The Team did not find any discussion on the new speed switch using the licensees DC system ground as part of the EMC circuit design.

The Team reviewed the supplier deviation notification, which was included in equivalency evaluation, PERP 1226, Revision 5. The notification specifically stated that the speed switch was tested and qualified for EMC compliance per EPRI TR-102323, Revision 4, and NRC Regulatory Guide 1.180. Generally known electrical engineering design practices associated with designing a component to be EMC compatible include shielding, filtering, and grounding.

The Team concluded that the licensees equivalency evaluation process should have identified that the DC system ground was utilized in the speed switch EMC circuit design and was not compatible with the stations DC distribution system. The relevant information should have been identified through vendor drawing reviews, discussions with the vendor, or by providing the vendor with plant design information associated with the stations hybrid three-wire DC distribution system, which powered the replacement speed switch.

Corrective Actions: The licensee replaced the speed switches on both EDGs, and commenced efforts to complete and install a modification for a new speed switch design that is compatible with the stations DC distribution system. Additionally, as an interim measure, the licensee modified their DC ground hunting procedure to prioritize checking the EDGs when DC grounds are detected.

Corrective Action References:

CR-2020-06947, Lockout of EDG No. 2 During Monthly Testing CR-2021-00920, EDG No. 2 Speed Switch Failure from 9/4/2020 Past Operability CR-2021-05657, 2021 EDG Reliability Evaluation: Speed Switch Design Concern for Application in DC System

Performance Assessment:

Performance Deficiency: The licensee failed to select a speed switch replacement that was suitable to the application. While completing the EDG speed switch equivalency evaluation, PERP 1226, in accordance with licensee procedure NOP-CC-2007, Part/Component Equivalent Replacement Packages, the licensee failed to evaluate how the EMC portions of the speed switch circuit interfaced with the existing plant design. Specifically, EMC portions of the circuit interfaced with the stations DC system ground, which has a DC voltage potential of 201 Vdc between the DC ground and the positive DC bus. A metal oxide varistor within the speed switch, which was connected between the positive DC bus and DC ground, was only rated for 170 Vdc. The MOV prematurely failed after approximately ten months of being in a continuous overvoltage condition. The MOV failure resulted in a short between the positive DC bus and the DC ground.

Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Equipment Performance attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. (i.e., core damage). Specifically, the Team concluded that the failure to ensure that components installed on the speed switch were of appropriate quality and rated for the system voltage resulted in the failure of the speed switch and the inoperability of EDG No. 2. The finding was self-revealed during TS surveillance testing when EDG No. 2 failed to start. The Team also compared the finding with the examples listed in IMC 0612, Power Reactor Inspection Reports, Appendix E, Example of Minor Issues.

Example 5.c was found to be similar in that components that did not meet their safety-related procurement specifications, which were later found to be installed in the operating plant, adversely impacted the associated cornerstone objectives of ensuring the reliability, capability, or availability of a system, structure, or component (SSC) important to safety.

Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At-Power. The Team concluded that because the finding could not be easily screened to be of very low safety significance, a detailed risk evaluation was required. Specifically, the positive ground, which was the speed switch, was identified on August 25, 2020, and EDG No. 2 operability was restored on September 7, 2020. The TS allowed outage time for a single EDG is 7 days and the period of time EDG No. 2 was non-functional was at least 9 days; consequently, the Team was required to answer yes to Mitigating Systems Screening Question A.3, Does the degraded condition represent a loss of the PRA function of one train of a multi-train TS system for greater than its TS allowed outage time? Specifically, the Team considered the time frame between the August 28, 2020, 8:00 p.m. operator log readings and the August 29, 2020, 8:00 a.m. operator log readings when the DC ground detector reading transitioned from approximately -1.05 mA to approximately 0 mA as the time when the speed switch failed. Two separate grounds on the same DC bus would have allowed excessive current to flow and caused the thermal damage that was identified on the speed switch.

Cross-Cutting Aspect: H.6 - Design Margins: The organization operates and maintains equipment within design margins. Margins are carefully guarded and changed only through a systematic and rigorous process. Special attention is placed on maintaining fission product barriers, defense-in-depth, and safety-related equipment. Specifically, the licensee approved a design and installed the safety-related speed switches, which had no design margin between the maximum volage rating of the MOV and the applied voltage of the stations DC distribution system.

Enforcement:

Apparent Violation: Title 10 of the Code of Federal Regulations (CFR) Part 50, Appendix B, Criterion III, Design Control, requires, in part, that measures shall be established for the selection and review for suitability of application of materials, parts, equipment, and processes that are essential to the safety-related functions of the structures, systems and components.

Technical Specification 3.8.1, AC Sources-Operating, Condition B, Required Action B.1, requires, in part, that SR 3.8.1.1 is performed for the offsite circuit(s) within 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months and once per 8 hours9.259259e-5 days 0.00222 hours 1.322751e-5 weeks 3.044e-6 months thereafter.

Technical Specification 3.8.1, AC Sources-Operating, Condition B, Required Action B.3, requires, in part, that it is determined that the operable EDG is not inoperable due to common cause failure within 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months or SR 3.8.1.2 is performed for the operable EDG within 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months .

From October 4, 2019, and July 26, 2021, the licensee apparently failed to establish measures for the selection and review for suitability of application of materials, parts, equipment, and processes that are essential to the safety-related functions of the structures, systems, and components. Specifically, the licensee apparently failed to select a speed switch replacement that was suitable for the application. The licensee apparently failed to establish and provide the speed switch vendor with the required voltage potential specifications for the positive DC bus to ground and the neutral DC bus to ground. Providing the vendor with the necessary design information was required for assuring the selection of suitable parts.

Additionally, because the licensee was not aware of the Division 2 EDGs inoperability between September 1 and September 4, 2020, the licensee apparently failed to meet Technical Specification 3.8.1 limiting condition for operation of one DG operable and the required actions in Technical Specification 3.8.1, B.1 and B.3 were not followed.

Enforcement Action: This issue is being treated as an apparent violation pending a final significance (enforcement) determination.

Unresolved Item Potential Adverse Changes During 2006 Emergency Diesel 93812 (Open) Generator Modifications URI 05000346/2021050-06

Description:

On May 27, 2021, during TS surveillance testing, the Division 1 EDG failed to reach the required voltage and frequency. Troubleshooting efforts by the licensee found that manual switch contacts, in the field flash circuit path, had fouling present that prohibited the current flow necessary to flash the generator field during the EDG starting sequence. The safety-related EDGs use current from the stations 125 Vdc system to supply the generator field during the starting sequence. Once the current supplied by the generator is adequate to maintain the EDG field, a separate contact opens and stops the current flow from the 125 Vdc system.

The licensee modified the EDG field flash circuit when it implemented modification ECP 05-0095-00, EDG Loading Improvements, in early 2006. Originally, the field flash circuit had a single circuit path that was used to flash the field for the 31-day slow start surveillance test, the 184-day fast start surveillance test, and automatic starts from standby conditions. A series of modifications made multiple EDG changes, which included installing new voltage regulators in the EDGs and modifying the field flash circuits to have two parallel paths in the circuit. The desired path is selected using the field flash selector switch FFSS, which is a General Electric Type SBM control switch. The first path, which is selected for the 31-day surveillance test, flashes the generator field when EDG speed reaches 800 rpm. The second path, which is selected for automatic starts from standby conditions and for the 184-day fast start surveillance test, flashes the generator field when EDG speed reaches 400 rpm.

The Team performed a review of modification ECP 05-0095-00 and raised a concern that some of the changes may have either required prior NRC approval, and/or were not implemented in accordance 10 CFR 50, Appendix B, Criterion III, Design Control. Testing of the circuit path that was required to flash the generator field during EDG automatic start changed from 31 days to 184 days. The test frequency change potentially allowed degraded FFSS contacts to go undetected for an additional five months. Additionally, the Team found that 31-day testing with continuity checks on the FFSS contacts, which was discussed in the modification for General Electric SMB switches, was never implemented. Specifically, the Team found that Section D.2, Operating History of Equipment Being Installed by this Engineering Change, discussed the General Electric SBM switch and stated, The 31-day tests, including continuity through the contacts will monitor these switches for any concerns.

The Team found that the 31-day test with continuity checks was partially implemented for the similar makeup pump breaker DST test switch, which was also installed in modification ECP 05-0095-00. However, the 31-day continuity check of the DST test switch, which is also a General Electric SMB switch, only tested some of the switch contacts.

Planned Closure Actions: Further review by inspectors, along with possible collaboration with the NRC Office of Nuclear Reactor Regulation (NRR), is necessary to determine if the issues of concern associated with the implementation of modification ECP 05-0095-00 constitute performance deficiencies or violations of NRC requirements.

Licensee Actions: No additional licensee actions are necessary at this time.

Corrective Action References:

CR 2021-04282, EDG No. 1 Failed to Reach Required Voltage and Frequency During the 184 Day Test Unresolved Item Potential High Emergency Diesel Generator Field Currents 93812 (Open) When Adding Manual Loads URI 05000346/2021050-07

Description:

On June 24, 2021, while testing the Division 1 EDG, the licensee observed that the EDG output voltage was almost 10 percent above the nominal value. The licensee found that the internal circuit of the RA-70 microprocessor-based reference unit had failed. The RA-70 provides a resistance reference value to the automatic voltage regulator, and it had failed to its high limit setpoint. The licensee repaired the EDG, returned it to service, and performed several assessments to determine if the high voltage condition would have prevented the EDG from performing its safety function.

The Team reviewed the licensees maintenance rule functional failure assessment, which concluded that the high voltage condition would not have prevented the EDG from performing its design function and that it did not constitute a maintenance rule functional failure. The licensee reviewed the design change documents associated with the installation of the updated EDG excitation system and found that the specific failure mode of the RA-70 failing to its maximum resistance setting had been assessed in a failure modes and effects analysis.

The analysis concluded that the EDG would be able to start and load, and while the EDG voltage would be higher than normal, it would be within an acceptable range for long-term operation. The licensee performed a separate evaluation to validate the conclusions of the failure modes and effects analysis. Specifically, the evaluation assessed the effects of the high voltage condition on the operation of loads connected to the EDG.

The Team independently reviewed the evaluation and found that the conclusions associated with the downstream EDG loads were reasonable. However, the Team also found that the evaluation did not completely address the effects of adding manual loads to the EDG after automatic load sequencing, and the potential increase in generator exciter current levels.

The Teams concern was that the increased generator exciter currents could cause an exciter component failure and prevent proper operation of the EDG. The licensee revised the evaluation to address the concern associated with high generator exciter currents; however, the evaluation was not completed before the conclusion of the inspection. As a result, the Team was required to document this issue as an Unresolved Item in this report.

Planned Closure Actions: Inspectors will review the updated Division 1 EDG high voltage assessment to confirm that the issue of concern is not a performance deficiency or violation of NRC requirements. Specifically, NRC inspectors will ensure that the assessment provides a reasonable basis for concluding that the EDG was capable of performing its safety function.

Licensee Actions: The licensee provided the updated high voltage assessment to the Team on October 11, 2021. Inspectors will review the updated assessment and determine if any additional questions are necessary.

Corrective Action References:

CR 2021-04913, EDG Overvoltage During Monthly Run

EXIT MEETINGS AND DEBRIEFS

The inspectors confirmed that proprietary information was controlled to protect from public disclosure.

On October 7, 2021, the inspectors presented the special inspection results to Mr. T. Brown, Site Vice President, and other members of the licensee staff.

DOCUMENTS REVIEWED

Inspection Type Designation Description or Title Revision or

Procedure Date

93812 Calculations C-EE-002.01-013 125/250 VDC Distribution System Ground Detection 0

C-EE-002.01-013 125/250 VDC Distribution System Ground Detection; 0

Addendum 1

C-EE-002.01-017 125/250 Volt System Failure Analysis 0

C-EE-024-01-010 Emergency Diesel Generator Room Electrical Equipment 0

Temperature Evaluation

C-EE-024.01-010 Addendums 1-12; Emergency Diesel Generator Room 0

Electrical Equipment Temperature Evaluation

C-ICE-025.01- Emergency Diesel Generator Indication Uncertainty; 0

007 Addendum A01

C-ME-24.03-02 SBO Building Ventilation 2

C-ME-24.03-02 Addendum 1; SBO Building Ventilation 2

Corrective Action CR-2003-06394 Potentially Inaccurate Statement in CR 02-07393 08/08/2003

Documents CR-2016-12653 GS2338 Will Not Close 10/22/2016

CR-2017-10465 Recent Hope Creek EDG OE: Failure of New Dynalco 10/30/2017

Speed Switch

CR-2017-10873 10 CFR Part 21 Notice from Engine Systems Inc. for EDG 10/30/2017

Speed Switch

CR-2019-04187 External Rust on EDG#1 Exhaust 05/07/2019

CR-2019-04973 EDG 2 184-Day Test, Speed and Voltage Variations 06/06/2019

CR-2019-05371 #1 Emergency Diesel Generator Monthly Water Jacket 06/21/2019

Copper and Ammonia Analysis are Trending Higher than

Typical Values

CR-2019-05707 C30205 SBODG Ground Meter Low mAMPs 07/03/2019

CR-2020-01501 Unexpected SBODG Trip During Electronic Governor 02/27/2020

Adjustment

CR-2020-01540 SBODG Fuse #2 (DC Fuel Pump) Blew During Testing 02/28/2020

CR-2020-01622 1CS11A Failed to Operate from Control Room During Plant 03/01/2020

Shutdown

CR-2020-01959 Degraded Conax Connections for ZSICS11B 03/08/2020

CR-2020-02203 Emergency Diesel Generator 1 Emergency Shutdown 03/13/2020

CR-2020-02220 1SC11A (Main Steam Line 2 Atmospheric Vent Valve) Not 03/14/2020

Inspection Type Designation Description or Title Revision or

Procedure Date

Stroking Smooth

CR-2020-02248 ICS11A Did Not Operate Smoothly 03/15/2020

CR-2020-02399 ICS11A (Main Steam Line Atmospheric Vent Valve) 03/19/2020

Positioner Is Venting

CR-2020-02462 ICS11A (Main Steam Line Atmospheric Vent Valve) Position 03/20/2020

Left out of Tolerance

CR-2020-02730 Steam Packing Unloader Valves Not Controlling Properly 03/29/2020

CR-2020-02902 System Monitoring EDG 1 Slow Start Time 04/03/2020

CR-2020-04930 Missed MSPI Failure Report in the First Quarter 2020 06/09/2020

CR-2020-05188 Missed Opportunity During MRB Review of CR-2020-02203 06/22/2020

CR-2020-05622 Air Leak on 1CS11A Air Regulator 07/11/2020

CR-2020-05924 System Monitoring: EDG 1 High Exhaust Temperature 07/24/2020

CR-2020-06691 Bus 2 DC SYS GRND I097 in Alarm 08/25/2020

CR-2020-06856 Abnormal DC MCC 2 Ground Indication 09/01/2020

CR-2020-06947 Lockout of EDG #2 During Monthly Test 09/04/2020

CR-2020-07529 DEHC Inverter Battery Bank is Degrading 09/28/2020

CR-2021-00920 EDG 2 Speed Switch Failure from 09/04/2020 Past 02/12/2021

Operability

CR-2021-00920 EDG 2 Speed Switch Failure from 09/04/2020 Past 02/12/2021

Operability (Proprietary)

CR-2021-04282 EDG #1 Failed to Reach Required Voltage and Frequency 05/27/2021

During the 184 Day Test

CR-2021-04520 Negative Trend in EDG Reliability During Cycle 22 06/08/2021

CR-2021-04582 SBODG Voltage Regulator 1 Green Light Flashing 06/10/2021

CR-2021-04601 SBODG Voltage Regulators #1 and #2 Malfunctions 06/11/2021

CR-2021-04913 EDG 1 Overvoltage During Monthly Run 06/24/2021

CR-2021-05220 DEHC D3201A UPS Failed to Power DEHC on Loss of E32A 07/09/2021

[7/8/21 Reactor Trip]

CR-2021-05222 AVV 1 Failed to Control in Auto Post Trip [7/8/21 Reactor 07/09/2021

Trip]

CR-2021-05225 AVV 2 Failed to Control from the Control Room Post Trip 07/09/2021

[7/8/21 Reactor Trip]

CR-2021-05228 SG1 Startup Feedwater Valve Failed to Control in Automatic 07/08/2021

Inspection Type Designation Description or Title Revision or

Procedure Date

[7/8/2021 Reactor Trip]

CR-2021-05230 SG2 Startup Feedwater Valve Failed to Control in Automatic 07/09/2021

[7/8/2021 Reactor Trip]

CR-2021-05235 MUXA:Z673 Computer Point Not Tracking Main Valve 07/09/2021

Position for SP6B [7/8/2021 Reactor Trip]

CR-2021-05242 BE306 Requires Replacement [7/8/2021 Reactor Trip] 07/09/2021

CR-2021-05250 Turbine/Reactor Trip on 7/8/2021 07/09/2021

CR-2021-05262 Loss of DEHC Operator Workstation During 7/8/2021 07/09/2021

Reactor Trip

CR-2021-05270 Failure of the Interlock Between TBV's and AVV's 07/09/2021

CR-2021-05271 Incorrect Switch Installed at ZS101B 07/10/2021

CR-2021-05279 ZS100B Found Degraded 07/10/2021

CR-2021-05283 Assessment of Operations Performance During Reactor Trip 07/10/2021

and Recovery on July 8, 2021

CR-2021-05285 Debris Found on Auxiliary Building Roof after Plant Trip 07/20/2021

CR-2021-05294 Missing Locknut Identified on ICS11A (AVV 2) 07/10/2021

CR-2021-05319 Defective Battery Cell Identified at D3201B During 07/12/2021

Troubleshooting of DEHC UPS [7/8/21 Reactor Trip]

CR-2021-05330 GS2338 Not Responding. Stuck OPEN @ 60% [Reactor Trip 07/12/2021

7/8/2021]

CR-2021-05494 Gland Steam Relief Lift During Transferring of Gland Steam 07/19/2021

from Main Steam to Aux Steam [Reactor Trip 7/8/2021]

CR-2021-05657 2021 EDG Reliability Evaluation: Speed Switch Design 07/26/2021

Concern for Application in DC System

CR-2021-05732 Aux Boiler Issues Experience During Plant Trip Due to Long 07/28/2021

Standing Issues

CR-2021-05916 Component Classifications and Single Point Vulnerability 08/05/2021

Reviews Are Not Being Performed/Completed for New

Equipment

CR-2021-05921 Missing Locknut Identified During Extent-of-Condition on 07/10/2021

ICS11B (AVV 1)

CR-2021-06134 NRC SIT: Incorrect AP-913 Classification for C3621 08/13/2021

CR-2021-06146 2021 NRC SIT: Switch DST for MUP Time Delay Relay 2Y1 08/13/2021

Isolation Potentially Exposed to Similar Degradation

Inspection Type Designation Description or Title Revision or

Procedure Date

Mechanisms as EDG FFSS

CR-2021-06212 2021 NRC SIT: Incorrect Value Recorded in Operator 08/17/2021

Rounds for August 2020 DC MCC 2 Ground Indication

CR-2021-06582 2021 NRC SIT: Calculation Contains Incorrect Value in Table 08/31/2021

CR-G201-2006- GS2338 Unable to Keep Gland Seal Steam at Desired 11/20/2006

10309 2.5-3.5 psig For Trending Purposes

CR-G201-2010- 120# Relief, GS1930 is Leaking Steam at 2.0 psig 04/08/2010

75049

CR-G201-2010- GS1938 and GS1939 Are Open When Should Be Closed 08/15/2010

246

CR-G201-2011- Unplanned Release During the Transfer of Gland Steam to 01/28/2011

87940 Auxiliary Steam

CR-G201-3020- Steam Seal Feed from Main Steam is Opening During Main 07/10/2010

79531 Turbine Operation

OE-2019-0238-1 IN 19-02, Emergency Diesel Generator Excitation System 07/18/2019

Diode Failures

Corrective Action CR-2021-05173 NRC Identified Concern Regarding Potentially Inadequate 07/07/2021

Documents Part 21s Leading to EDG Failures

Resulting from CR-2021-05729 Operational Impact for Not Replacing ZS101B 07/28/2021

Inspection CR-2021-05919 Maintenance Rule Evaluation for CR-2021-05262 Required 08/05/2021

CR-2021-06024 NRC SIT: Maintenance Rule Failure Review for 08/09/2021

CR-2021-05319 Defective Battery Cell Identified at D3201B

During Troubleshooting of DEHC UPS

CR-2021-06488 Error Found in MAAP Files Supporting PRA Model at 08/27/2021

Davis-Besse

CR-2021-07235 2021 NRC SIT: EDG Loading in DB-OP-02000 Titled, "RPS, 09/27/2021

SFAS, SFRCS TRIP, or SG Tube Rupture"

Drawings E-0049B; Elementary Wiring Diagrams Treated Water MY PMPS 22

Sheet 1B (Charging)

E-0064B, Elementary Wiring Diagrams EMER DSL GENS Field Flash 6

Sheet 13 and Shorting CRKTS

E-0064B; Elementary Wiring Diagrams EMER DSL GENS Field Flash 5

Sheet 13 and Shorting CRKTS

M-022A Main Turbine Gland Steam Seal and Drain System 19

Inspection Type Designation Description or Title Revision or

Procedure Date

M-180-00010; Schematic GOV & Voltage Control Emergency Diesel 20

Sheets 1 & 2 Generator 1-1 & 1-2

M-180-00031 Schematic Diagram CVT Static Exciter Regulator 312 19

M-180-00031; EDG Excitation System Interconnection Diagram 23

Sheets 1-3

M-180Q-0013; Schematic Diagram Engine Control for Emergency Diesel 25

Sheets 1-2 Generator 1-1 & 1-2

M-180Q-0013; Schematic Diagram Engine Control for Emergency Diesel 19

Sheets 1-2 Generator 1-1 & 1-2

M-180Q-0014 Schematic Diagram Engine Control 14

OS-008; Sheet 3 Operational Schematic Main Steam and Reheat System 32

OS-009 Operational Schematic HP & LP Turbine Drains and Seal 22

System

SF-003B; SFRCS Internal Schematic Diagram MN STM LIne-1 WU 3

Sheet 11 ISO VLV MS-101-1

Engineering 02-0737-00 Emergency Diesel Generator Excitation System 11

Changes Replacement

2-0737-01 Emergency Diesel Generator Excitation System 2

Replacement, Supplement 1

03-0080-00 Equivalent Replacement of the Field Flash Contactor (FFC) 07/08/2003

for Emergency Diesel Generator (s) 1-1, and 1-2

05-0140 Replacement of Electrohydraulic Control (EHC) and Turbine 6

Supervisory Instrumentation (TSI) Systems

2-0134 Replace Voltage Regulations System 7

20-0159-001 Replacement of OCS11B Valve Positioner 0

ECR 05-0095-00 EDG Loading Improvements 5

MPR-2838 Evaluation of Reliability of Replacement EDG Excitation 08/24/2005

System for Davis-Besse Nuclear Power Plant

PERP 001226 Replacement of the EDG Speed Transmitters, ST6221 4-6

(EDG 1-1) and ST6231 (EDG 1-2)

Miscellaneous Operator Rounds; DCMCC2 Ground Indication 08/23/2020-

09/02/2020

Product Data Sheet; Dynalco SST-2000 Speed Switches / 03/01/2009

Transmitters

Inspection Type Designation Description or Title Revision or

Procedure Date

General Electric Hitachi GEH Nuclear Parts e-Newsletter Fall 2015

DBNPS Surveillance Test Intervals List 1

Davis-Besse Nuclear Power Station Quality Assurance 28

Program Manual

Davis-Besse Engineering Department Simple Diagram; Undated

Normal System Voltage for Ungrounded DC System

Davis-Besse Licensee Event Report 2021-001-00; 04/12/2021

Emergency Diesel Generator Speed Switch Failure Due to

Direct Current System Ground

Littlefuse UltraMOV Varistor Series Datasheet 10/10/2019

Dynalco SST2000 Series Universal Speed Switch & Speed 10/20/2016

Transmitter; Installation and Operation Manual

Unit Log Entry; Speed Switch Design Issue Compensatory 07/26/2021

Measures

Chart; Emergency Diesel Generator 2 Speed Switch 08/05/2021

Operational Voltage Measurements

EDG Degraded Performance Presentation (Proprietary) 07/26/2021

Control Room Required Reading; Operator Rounds Ground 08/18/2021

Indication

Outside the Control Room Licensed and Non-Licensed 08/18/2021

Required Reading; Operator Rounds Ground Indication

Equipment Operator Required Reading; Operator Rounds 08/18/2021

Ground Indication

Maintenance Rule Program Manual 39

7/8/21 Reactor Trip Operations Performance PowerPoint 07/27/2021

Presentation

7/8/21 Reactor Trip SUFW Control Valve Position Curves 07/27/2021

7/8/21 Reactor Trip Steam Generator Train 1 and 2 Trend 07/29/2021

Curves

7/8/21 Reactor Trip Primary System Trend Curves 07/29/2021

Routine Rapid Feedwater Response Trend Curves 07/29/2021

Maintenance Rule Program Manual 39

Purchase Order UG-1278 04/05/2006

Purchase Order 45189942 Closeout Checklist 09/10/2020

Inspection Type Designation Description or Title Revision or

Procedure Date

Certificate of Conformance for Purchase Order 45517680 12/17/2018

EDG Degraded Performance Presentation 07/26/2021

Purchase Order 45652187 09/14/2020

Purchase Order 45652047 09/06/2020

Purchase Order 45517680 07/27/2017

Purchase Order 45500676 09/08/2016

Purchase Order 45564999 02/05/2020

String Data Package for 92B-ISP2340 0

Instrument Package for PV-2338 1

Instrument Package for PCV-1939 3

Instrument Package for PCV-1938 3

Davis-Besse Unit Logs 07/03/2019 -

07/09/2021

Various EDG System Health Reports 07/31/2019 -

09/22/2021

Selected List of EDG OE 07/2021

200-0213-RPT- Evaluation of High EDG1-1 Voltage at Davis-Besse 3

001

200-0213-RPT- Evaluation of High EDG1-1 Voltage at Davis-Besse 2

001

88516 Vishay General Semiconductor; 1N5408 General Purpose 08/01/2013

Plastic Rectifier

AN9767.1 Littlefuse Varistors Basic Properties, Teminology and 07/1999

Theory

AN9771.1 LIttlefuse Application Note; Selecting a Littlefuse Varistor 07/1999

E000044 Material Engineering Evaluation; Commercial Grade 4

Dedication; General Electric Type SBM Rotary Selectro

Switches

EPRI TR-100185 Motor and Generator Insulation Life Estimation, Volume 1 01/1992

ERPI-NP-3095 Generic Qualification of Rotary Hand Switches 05/1983

GEH-2038 Control and Transfer Switch Type SBM A and E

GEH-2038 Control and Transfer Switch Type SBM C

GET-6169E Selection and Application Guide for SB Control and Transfer 07/1992

Inspection Type Designation Description or Title Revision or

Procedure Date

Switch

M-003B-00001 DEHC Equipment Manual 09/16/2014

M-324-00147-03 Instruction Manual for 195 & 196 Series Meter Relay 04/07/2005

Controllers

Operations DC Grounds and EDG Speed Switches 07/29/2021

Standing Order

21-002

Operations Gland Steam Local Operator During GS2385 Transfers 08/06/2021

Standing Order

21-003

Vendor Manual Ametek Solidstate Controls Instruction / Technical Manual 4

E-855Q-00001 (Proprietary)

Vendor Manual 06/16/2021

M-180-00093;

2600 KW

Emergency

Diesel Generator

Book II

Maintenance

Manual

Continued

(Proprietary)

Vendor Manual; Basler Electric; Instruction Manual for Generator Excitation 2

E-287Q-00001 System

NDE Reports 10CFR21-0118 Part 21 Report; Engine Systems, Inc; Speed Switch PNs 10/26/2017

ESI50267C, ESI50267E, ESI50267H, and ESI50267K

Procedures DB-ME-03047 Make Up Pump #1 Time Delay Monthly Functional Test 4

DB-ME-03047 Make Up Pump #1 Time Delay Monthly Functional Test 0

DB-ME-03048 Make Up Pump #2 Time Delay Monthly Functional Test 0

DB-OP--6910 Trip Recovery 37

DB-OP-02000 RPS, SFAS, SFRCS Trip, or SG Tube Rupture 31

DB-OP-06205 Turbine Generator and Main Feedwater Pump Turbine Gland 16, 17, 24

Steam and Turbine Drains

DB-OP-06205 Turbine Generator and Main Feedwater Pump Turbine Gland 25

Inspection Type Designation Description or Title Revision or

Procedure Date

Steam and Turbine Drains

DB-OP-06224 Main Feed Pump and Turbine 44

DB-OP-06225 MDFP Operating Procedure 26

DB-OP-06233 Auxiliary Feedwater System 45

DB-OP-06316 Diesel Generator Operating Procedure 65

DB-OP-06322 Locating Grounds on the Station 250/125 VDC System 8

DB-OP-06334 Station Blackout Diesel Generator Operating Procedure 32

DB-SC-03070 Emergency Diesel Generator 1 Monthly Test 12

DB-SC-03070 Emergency Diesel Generator 1 Monthly Test 43

DB-SC-03071 Emergency Diesel Generator 2 Monthly Test 40

DB-SC-03071 Emergency Diesel Generator 2 Monthly Test 11

DBBP-DCU-0010 Engineering Change Closeout Processing 0 - 10

NG-EN-00333 Vendor Manual Control 8

NOBP-CC-2003 Configuration Management Database Control 4

NOBP-CC-2003 Configuration Management Database Control 3

NOBP-CC-7001 Procurement Packages 9

NOBP-ER-3005 FENOC Equipment Vulnerability Review Process 3

NOBP-ER-3900 Equipment Reliability Common Definitions and Structure 3 & 12

NOBP-ER-3900 Equipment Reliability Common Definitions and Structure 0 and 1

NOBP-ER-3901 Component Classification ER Workbench Module 1 5 & 10

NOBP-ER-3903 Component Template Implementation 0-2

NOBP-ER-3903 Component Template Implementation ER Workbench 9

Module

NOBP-LP-2011 Cause Analysis 28

NOBP-LP-2100 Operating Experience Process 22

NOBP-LP-4003A FENOC 10 CFR 50.59 User Guidelines 2

NOBP-LP-4003B 10 CFR 50.59 Mentoring Review Committee 1

NOP-CC-1003 Vendor Technical Information 4

NOP-CC-2001 Design Verification 2-4

NOP-CC-2002 Design Input 1 and 2

NOP-CC-2003 Engineering Changes 9

Inspection Type Designation Description or Title Revision or

Procedure Date

NOP-CC-2003 Engineering Changes 19

NOP-CC-2004 Design Interface Reviews and Evaluations 10

NOP-CC-2004 Design Interface Reviews and Evaluations 2-5

NOP-CC-2007 Part/Component Equivalent Replacement Packages 3

NOP-CC-7002 Procurement Engineering 6

NOP-ER-1001 Continuous Equipment Performance Improvement 6

NOP-ER-1001 Continuous Equipment Performance Improvement 3

NOP-ER-3001 Problem Solving and Decision Making 8

NOP-ER-3004 Maintenance Rule Program 6

NOP-LP-2001 Corrective Action Program 48

NOP-LP-4003 Evaluation of Changes, Tests and Experiments 11/14/2005

NOP-OP-1002 Conduct of Operations 16

NOP-WM-3001 Work Management PM Process 20

NOP-WM-3001 Work Management PM Process 3 and 4

NORM-LP-2003 Analytical Methods Guidebook 14

NORM-OP-1002 Conduct of Operations 9

Radiation Work NOBP-ER-3901 Component Classification 0 and 1

Permits (RWPs)

Work Orders 200205458 Replace Switches ZS101A, B, C 01/29/2008

200495462 PM 7257 K5-1 *RPLC* Adjuster RA-70 EDG 1 08/12/2014

200589418 Perform UPS/Power Feed Testing per ECP 05-0140-004 ITR 04/07/2014

200629726 Replace ST6231 EDG 2 Speed Switch 08/19/2019

200629727 Replace ST6221 EDG 1 Speed Switch 10/02/2020

200631761 Replace the Two Existing C30203 Synchrostart Speed 05/17/2017

Switches With a Dynalco Equivalent

200687928 PM 1804 Calibrate ICS Modules 04/02/2018

200688424 PM 11213 Calibrate ICS Modules 04/02/2018

200688512 PM 11928 MS101 Limit Switches Functional Check 03/26/2018

200732752 Monitor SBODG Speed Switch Voltages 02/28/2018

200756127 Integrated SFAS Act Ch1 FA Norm RFL 03/11/2020

200762235 EDG 1 184 Day Test FA Norm 06/24/2020

200766425 PM 7736 K5-3 *Rplc* Electronic Governor 08/21/2019

200768462 PM 11212 Calibrate ICS Modules 04/06/2020

Inspection Type Designation Description or Title Revision or

Procedure Date

200781326 EDG 1 184 Day Test FA Norm 11/08/2020

200786857 SC3076-001 04.000 K5-1 EDG1 184 Day DA 30 05/21/2021

200789245 Troubleshooting Emergency Diesel Generator 1-2 Engine 07/09/2019

200794277 SBODG Ground Meter at 0 Milliamps 07/10/2019

200795495 PM 11798 Clean and Inspect D3201A 06/10/2021

200812888 EDG 1 184 Day Test FA Norm 03/11/2020

200816842 Troubleshooting Activities to Identify Cause of SBODG 02/27/2020

Issues

200817105 PM 7257 K5-1 *RPLC* Adjuster RA-70 EDG 1 06/25/2021

200817835 Emergency Shutdown EDG #1 03/13/2020

200819167 DCMCC2; (D2P & DBP) Ground Hunting with Cart 12/03/2020

200831322 Diesel Generator 1-2 Speed Transmitter 09/05/2020

200832897 Replace Leaking Battery 04/15/2021

200856289 EDG 1 Failed to Reach a Steady State Voltage and 05/27/2021

Frequency

200857743 Troubleshoot Voltage Regulator Due to Failing High 06/24/2021

200858488 Troubleshoot and Repair EHC Inverter/UPS 07/14/2021

200858490 Troubleshoot/Calibrate/Repair/Replace ICS11A Control 07/11/2021

200858508 PM 7519 BE306 Breaker Swap 07/16/2021

200858529 Replace Faulty Battery 07/14/2021

200858874 Troubleshoot/Repair/Replace Components/Calibrate GS2338 07/22/2021

Valve Actuator PV2338

200859743 Replace SP7A/B Controller Modules 09/02/2021

601266808 GS1938 and GS1939 Troubleshooting 01/27/2021

601302699 PIC2338 Not Controlling Properly and Needs Repair 01/27/2021

601324629 Investigation and Repair of Limit Switch ZS100B 07/10/2021

601325155 Calibration Check of PT-2340 07/15/2021

July 27, 2021

MEMORANDUM TO: Daniel

E. Kimble, Senior Resident Inspector, Team Lead

Projects Branch 3

Division of Reactor Projects

FROM: Michelle

W. Hayes, Acting Deputy Division Director

Division of Reactor Safety

SUBJECT: SPECIAL INSPECTION CHARTER FOR DAVIS-BESSE NUCLEAR

POWER STATION EMERGENCY DIESEL GENERATOR

DEGRADED PERFORMANCE AND THE JULY 8, 2021,

COMPLICATED REACTOR TRIP

The purpose of the special inspection is to review the circumstances associated with

Davis-Besse Nuclear Power Station emergency diesel generator degraded performance and the

July 8, 2021, complicated trip.

Diesel Generator Failures

There have been four failures of the safety-related emergency diesel generator (EDG) start

circuits in the past sixteen months at Davis-Besse. There was also a similar failure of the

station blackout (SBO) diesel within the past 24 months. Davis-Besse has two EDGs and one

SBO (all three diesels are similar models made by EMD).

On May 27, 2021, plant operators started EDG 1 for the scheduled 184-day surveillance test

which included a fast start. The start was unsuccessful. The EDG started and reached rated

speed but did not produce any electrical power. The licensee stood up a problem-solving team

that determined the fault to be a field flash selector switch making intermittent contact. The

switch (GE SBM type) has two positions and had no continuity through the 400 rpm contacts

roughly 1 out of 5 times when switched from the 800 rpm to 400 rpm position. The safety

start/fast start setting is 400 rpm, and the EDG is inoperable if the field doesnt flash at that

speed. The 800 rpm setting is used only for the monthly slow start surveillance so that the EDG

can be started at no-load idle, which is ~450 rpm. In its degraded condition, the EDG would not

have been capable of producing power without operator action to flash the generator field. The

licensee replaced the switch and the EDG tested satisfactorily. Plant operators ran the previous

184-day surveillance test (fast start) on November 12, 2020. The previous monthly test (idle

start) was run on April 29, 2021. A similar failure occurred during integrated safety features

actuation system testing on March 13, 2020 (CR 2020-02203). The licensee inspected the

switch visually and noted corrosion of the switch contacts, especially on the 400 rpm contacts.

CONTACT: Richard Skokowski

630-829-9833

D. Kimble -2-

On June 24, 2021, during the next monthly idle start test of EDG 1, the voltage (4553.5 V)

exceeded the voltage acceptance criteria (4088-4400 V). Attempts to manually adjust voltage

were unsuccessful. EDG 1 was shut down and secured. A problem-solving team determined

that a failed microprocessor-based reference adjuster (RA70) was the cause. This

microprocessor-based reference adjuster was installed as a modification in ~2005 to replace the

obsolete motor-controlled potentiometer. EDG 2 was idle started to rule out common cause

failure and satisfy LCO 3.8.1 action B.3.2 ensuring an operable EDG. The licensee replaced

the EDG 1 reference adjuster on June 25, 2021, with a spare the licensee had onsite. Following

this repair, the EDG was successfully idle started and declared operable.

On September 4, 2020, EDG 2 failed to start and experienced a lockout during a scheduled

monthly surveillance test due to a failed speed switch. The failed speed switch had been in

service for ten months before the failure and had been installed as a preventative maintenance

component replacement in October 2019. This speed switch was a Dynalco model subject to a

2017 Part 21 (2017-50) that identified a possible failure mode related to voltages spikes at Hope

Creek:

Investigation revealed that although the nominal system voltage is 120 VDC, the

customer's external relay coils that are connected to the speed switch output contacts do

not have transient suppression, such as with a flyback diode. In these cases, the

voltage transient produced when de-energizing the relay's coil is large enough to

damage the capacitors. Hope Creek recorded a voltage spike exceeding 1467 VDC.

For customers that have flyback diodes or a similar transient suppression device across

the external relay coil, there is no issue. However, for customers without such

protection, the magnitude of the voltage transient may be sufficient to damage and

degrade the capacitor to the point of a low resistance path to ground.

On July 3, 2019, the licensee detected an electrical ground on the SBO control circuit.

Troubleshooting determined that the electrical ground was associated with the speed sensing

circuit. The licensee found the speed switch failed and replaced it. The SBO is not

safety-related but is risk significant. This speed switch failure is similar to the speed switch

failure in EDG 2 and similar to issues described in the Part 21. In addition, Davis-Besse does

not have any transient suppression installed on any of its diesel start circuits.

In response to Part 21 2017-50, the licensee sent the two Dynalco speed switches in

warehouse stock back to the vendor (ESI) to address the identified defect. Following

refurbishment, the licensee placed the switches back into stock and used them to replace the

speed switches in the SBO and EDG 2 as described above. A new Dynalco speed switch was

installed in EDG 1 in 2020.

Based on the deterministic criteria provided in Management Directive (MD) 8.3, NRC Incident

Investigation Program, Emergency Diesel Generators Degraded Performance met MD 8.3,

Criterion e, Involved possible adverse generic implications. A Region III Senior Reactor

Analyst completed a modified Davis-Besse model event assessment to provide risk insights.

The assessment resulted in a preliminary Incremental Conditional Core Damage Probability

(ICCDP) value of 7E-7 and 5E-6 for specific events, which places the risk in the overlap region

between a Special Inspection Team and No Additional Inspection. Accordingly, based on the

deterministic and risk criteria in the MD 8.3, and as provided in Regional Procedure 8.31,

Special Inspections at Licensed Facility, the Region determined that a Special Inspection was

warranted.

D. Kimble -3-

Complicated Trip

On July 8, 2021, at 2154, the reactor tripped due to loss of power to the main generator

automatic voltage regulator and digital electrohydraulic control system (DEHC) while performing

a post maintenance functional check of the alternate feed breaker (BF306) to non-safety motor

control centers (MCC) E32A&

B. When returning to normal breaker alignment, the normal

feeder BE306, faulted and smoke was observed causing MCC E32A&B to lose power. This

combined with a bad battery cell on the uninterruptible power supply to the main generator

automatic voltage regulator, resulted in a turbine trip and reactor trip. Moisture separator

reheater 1 second stage steam source valve (MS-199) failed to close due to the loss of its

power supply (deenergized MCC E32A) and acted as a significant steam demand on steam

generator 1 until it was closed via handwheel later in the event.

Immediately following the trip and with MS-199 continuing to draw excess steam from steam

generator 1, reactor coolant system pressure (RCS) dropped due to overcooling. Pressurizer

(PZR) level dropped to 16 inches and uncovered the PZR heaters. At 2158 operators entered

the overcooling section of DB-OP-02000 and lined up low pressure injection (LPI)/high pressure

injection (HPI) pumps in piggyback mode. RCS pressure dropped below the nominal 1800 psi

capacity of the HPI pump in piggyback mode, probably allowing injection into the RCS. The

injected volume is difficult to know for sure because the makeup pump (higher pressure but

lower capacity than the HPI pump) was injecting through the same lines.

Following this, steam generator 1 level dropped because startup feedwater valve 1 (SP7B)

failed to open fast enough to offset the steam loss due to MS-199 remaining open. Operators

took manual control of SP7B at 35 steam generator level and opened the valve but were not

able to arrest the level decrease quickly enough. Steam generator 1 level continued to drop

until it hit the 23.5 steam feed rupture control system (SFRCS) actuation setpoint at 2201. This

initiated both auxiliary feedwater (AFW) pumps. The plant stabilized with AFW feeding the

steam generators and turbine bypass valve operating to remove decay heat.

At 2358 low steam generator 2 pressure caused an SFRCS actuation which closed the main

steam isolation valves (MSIVs) leading to a loss of the condenser as a heat sink. This transient

began at 2347 when operators were transferring gland steam from main steam to auxiliary

steam around the same general time (the exact timeline is not currently known by inspectors)

that they were realigning feedwater from AFW to main feedwater (MFW).

Following the MSIV isolation the crew attempted to establish control of steam generator

pressures and RCS heat-up using the Atmospheric Vent Valves (AVVs). They determined that

neither AVV would control in auto and placed both in manual (hand) control. Operators

observed AVV 2 cycling 5-20% open. Attempts to close AVV 2 from the control room were

unsuccessful. The crew isolated instrument air to AVV 2 and established manual control locally

using the handwheel.

Based on the deterministic criteria provided in Management Directive (MD) 8.3, NRC Incident

Investigation Program, July 8, 2021, Complicated Reactor Trip met the following criteria:

d. Led to the loss of a safety function or multiple failures in systems used to mitigate an

actual event;

f. Involved significant unexpected system interactions; and

h. Involved questions or concerns pertaining to licensee operational performance.

D. Kimble -4-

A Region III Senior Reactor Analyst completed an event assessment using the Davis-Besse

SPAR (Standardized Plant Analysis Risk) model. The assessment resulted in a preliminary

mean value of conditional core damage probability (CCDP) for this event of 1.2E-6, with 5% and

95% values at 8.4E-8 and 4.3E-6, respectively. This CCDP value, especially when considering

the qualitative factors, places this event in the overlap region of Special Inspection and No

Additional Inspection. Accordingly, based on the deterministic and risk criteria in the MD 8.3,

and as provided in Regional Procedure 8.31, Special Inspections at Licensed Facility, the

Region determined that a Special Inspection was warranted.

Conclusion

Based on the results of the two MD 8.3 reviews completed, a special inspection team will

commence an inspection on July 26, 2021, at Davis-Besse to review circumstances associated

with Davis-Besse Nuclear Power Station EDG degraded performance and the July 8, 2021,

complicated trip. The special inspection team will be led by you and will include Kevin Barclay,

David Reeser, and Lionel Rodriguez from Region III, Julie Winslow from the Office of Nuclear

Reactor Regulation, and Kenn Miller from the Office of Nuclear Research.

The special inspection will determine the sequence of events and will evaluate the facts,

circumstances, and the licensees actions surrounding these events. The specific charter for

the team is enclosed.

Docket No. 50-346

License No. NPF-3

Enclosure:

Davis-Besse Nuclear Power Plant Special Inspection Charter

DAVIS-BESSE NUCLEAR POWER PLANT SPECIAL INSPECTION CHARTER

This special inspection team is chartered to assess the circumstances surrounding the five

diesel generator failures that occurred at Davis-Besse within the last two years and the complicated

trip on July 8, 2021. The special inspection will be conducted in accordance with Inspection

Procedure 93812, Special Inspection. The Inspection team will conduct an entrance and exit

meeting and document the inspection findings and conclusions in a Special Inspection Team

final report within 45 days of inspection completion.

The inspection activities will include, but are not limited to, the items listed below:

Diesel Generator Failures

1. Identify a timeline for all diesel failures within the last two years (including the station

blackout generators). Include relevant and major plant conditions, system lineups,

and operator actions in response to the failures, and any applicable maintenance or

component replacements prior to each failure.

2. Review licensees troubleshooting efforts and follow-up evaluations for each of these

failures to confirm the adequacy of the licensees assessments including extent of

condition.

3. Review identified failed components for acceptability of design, including

environmental conditions and appropriate surge protection, where applicable.

4. Assess the licensees operability, functionally and/or availability assessment

associated with the Division 1 EDG June 24, 2021, failure, focusing on the resulting

high voltage condition and its impact on the availability of the diesel-powered

components.

5. Considering the cause information from the March 13, 2020, failure of the Division 1

EDG to start during the integrated safety features actuation system (SFAS) test,

evaluate the licensees past operability and functionality assessment of the Division 1

Diesel Generator following the semiannual fast start test failure on May 27, 2021.

This assessment should focus on the start circuitry for potential common issues that

could have caused these failures.

6. Assess the licensees evaluation and response to all applicable Part 21 reports.

7. Access the licensees monitoring of the diesel generator systems performance

information including the licensees system health reports, corrective action program

for possible trends and overall timeliness and effectiveness of evaluating associated

failures, degradations, and deficiencies.

8. Perform an independent search of a sample of industry information associated with the sites

diesel generators and validate that the licensee reviewed the issues and that their reviews

were adequate.

9. Promptly identify and convey potential generic safety concerns to regional

management and NRR/DRO/IOEB when appropriate, who will initiate appropriate

follow-up actions.

Complicated Trip

10. Identify a timeline for the significant events associated with the July 8, 2021,

complicated reactor trip, including the initial plant conditions, the relevant activities

on-going prior to the trip, and the significant alarms and indication, operator actions and

equipment major performance until the plant was placed in a stable condition.

11. Review and evaluate equipment performance during the event with particular

emphasis in the following areas:

a. Atmospheric Vent Valve Operation after MSIV closure;

b. Start Up Feedwater Valves SP7A and SP7B; and

c. Uninterruptable Power Supply failure.

2. Review and evaluate operator response during the event with particular emphasis in

the following areas:

a. Conditions that resulted in SFRCS actuation and MSIV isolation; and

b. Response to initial overcooling event.

Charter Approval

Signed by Skokowski, Richard

on 07/27/21

Richard

A. Skokowski, Chief,

Engineering Branch 3, Division of Reactor Safety

Signed by Hayes, Michelle

on 07/27/21

Michelle

W. Hayes, Acting Deputy Division Director,

Division of Reactor Safety

2

v t e Inspection Report - Davis Besse - 2021050
80 \| 81 \| 82 \| 83 \| 84 \| 85 \| 86 \| 87 \| 88 \| 89 \| 90 \| 91 \| 92 \| 93 \| 94 \| 95 \| 96 \| 97 \| 98 \| 99 00 \| 01 \| 02 \| 03 \| 04 \| 05 \| 06 \| 07 \| 08 \| 09 \| 10 \| 11 \| 12 \| 13 \| 14 \| 15 \| 16 \| 17 \| 18 \| 19 \| 20 \| 21 \| 22 \| 23 \| 24 \|
2021 Quarterly Integrated 1Q (0) 2Q (0) 3Q (0) 4Q (0) Assessments Mid Cycle End of Cycle 10(0) 11(0) 12(0) 40(0) 50(0) 90(0) 91(0) 92(0) Security 401(0) 402(0) 403(0) 404(0) Operator Exams 301(0) Emergency Planning 501(0) Other

IR 05000346/2021050

Text

SUBJECT:

Dear Mr. Brown:

Enclosure:

Inspection Report

SUMMARY

List of Findings and Violations

Additional Tracking Items

INSPECTION SCOPES

OTHER ACTIVITIES

1. Identify a timeline for all diesel failures within the last two years (including the station

2. Review licensees troubleshooting efforts and follow-up evaluations for each of these

3. Review identified failed components for acceptability of design, including environmental

4. Assess the licensees operability, functionally and/or availability assessment associated with

5. Considering the cause information from the March 13, 2020, failure of the Division 1 EDG to

6. Assess the licensees evaluation and response to all applicable Part 21 reports.

7. Assess the licensee's monitoring of the diesel generator systems performance information

8. Perform an independent search of a sample of industry information associated with the

9. Promptly identify and convey potential generic safety concerns to regional management and

INSPECTION RESULTS

Description:

Performance Assessment:

Enforcement:

Description:

Performance Assessment:

Enforcement:

Description:

Performance Assessment:

Enforcement:

Description:

Performance Assessment:

Enforcement:

Description:

Performance Assessment:

Enforcement:

Description:

Description:

EXIT MEETINGS AND DEBRIEFS

DOCUMENTS REVIEWED

Navigation menu

Search