IR 05000440/2019003

Integrated Inspection Report 05000440/2019003

ML19318F401
Person / Time
Site:	Perry
Issue date:	11/14/2019
From:	Billy Dickson Reactor Projects Region 3 Branch 4
To:	Payne F FirstEnergy Nuclear Operating Co
References
IR 2019003
Download: ML19318F401 (25)

Text

ber 14, 2019

SUBJECT:

PERRY NUCLEAR POWER PLANT - INTEGRATED INSPECTION REPORT 05000440/2019003

Dear Mr. Payne:

On September 30, 2019, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at Perry Nuclear Power Plant and discussed the results of this inspection with you and other members of your staff. The results of this inspection are documented in the enclosed report.

Two findings of very low safety significance (Green) are documented in this report. One of these findings involved a violation of NRC requirements. We are treating this violation as a non-cited violation (NCV) consistent with Section 2.3.2 of the Enforcement Policy.

If you contest the violation or significance or severity of the violation documented in this inspection report, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN:

Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region III; the Director, Office of Enforcement; and the NRC Resident Inspector at Perry Nuclear Power Plant.

If you disagree with a cross-cutting aspect assignment or a finding not associated with a regulatory requirement in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region III; and the NRC Resident Inspector at Perry Nuclear Power Plant. This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with 10 CFR 2.390, Public Inspections, Exemptions, Requests for Withholding.

Sincerely,

/RA/

Billy C. Dickson, Jr., Chief Branch 2 Division of Reactor Projects Docket No. 05000440 License No. NPF-58

Enclosure:

As stated

Inspection Report

Docket Number: 05000440 License Number: NPF-58 Report Number: 05000440/2019003 Enterprise Identifier: I-2019-003-0056 Licensee: FirstEnergy Nuclear Operating Company Facility: Perry Nuclear Power Plant Location: Perry OH Inspection Dates: July 01, 2019 to September 30, 2019 Inspectors: S. Bell, Health Physicist J. Cassidy, Senior Health Physicist J. Nance, Senior Resident Inspector J. Rutkowski, Project Engineer J. Steffes, Senior Resident Inspector Approved By: Billy C. Dickson, Jr., Chief Branch 2 Division of Reactor Projects Enclosure

SUMMARY

The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting an integrated inspection at Perry Nuclear Power Plant in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.

List of Findings and Violations Failure to Properly Seat Essential Service Water "A" Pump Control Power Fuses results in Loss of Safety Function Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green [H.9] - Training 71152 Systems NCV 05000440/2019003-01 Open A self-revealed finding of very low safety significance (Green) and associated non-cited violation of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, for the licensee's failure to accomplish work activities in accordance with the instructions, procedures, or drawings. Specifically, the licensee failed to properly install the A essential service water control power fuses in accordance with SOI-R22, Metal Clad Switchgear 5-15 kV, paragraph 2.3 such that the fuses were able to back out and cause an unplanned inoperability of several safety related systems during routine breaker operation.

LER 2019-001-00 for Perry Nuclear Power Plant, Loss of Generator Excitation Causes Turbine Trip and Reactor Protection System Actuation.

Cornerstone Significance Cross-Cutting Report Aspect Section Initiating Events Green None (NPP) 71153 FIN 05000440/2019003-02 Open A self-revealed finding of very low safety significance (Green) was identified for the licensee's failure to perform adequate preventive maintenance on the main generator exciter brushes, which led to brush failures, loss of generator field excitation, generator lockout, and reactor scram. No violation of regulatory requirements was identified.

Additional Tracking Items None.

PLANT STATUS

The plant began the inspection period at rated thermal power. On July 27, 2019, at 7:29 p.m.,

the unit scrammed in response to a turbine trip caused by closure of the turbine stop valves.

The turbine stop valve closure occurred during turbine mechanical overspeed testing due to a fault in the trip test circuitry. The licensee replaced the mechanical shutoff valve in the overspeed test circuitry and commenced a reactor startup on July 30, 2019, achieved criticality at 1:08 a.m. on July 31, 2019 and synched to the grid later that day. The unit was returned to rated thermal power at 12:40 a.m. on August 9, 2019. On September 14, 2019, the licensee commenced power reduction to 18 percent reactor power in order to address a hydraulic leak associated with number 3 turbine control valve. Following attempted repairs to the number three turbine control valve, the licensee commenced power ascension and returned the plant to rated thermal power on September 16, 2019. The reactor remained at approximately 100 percent power for the remainder of the inspection period except for rod sequence exchanges and required testing.

On April 25, 2018, FirstEnergy Solutions (FES) / FirstEnergy Nuclear Operating Company (FENOC) notified the U.S. Nuclear Regulatory Commission (NRC) that they intend to shut down all four of their operating nuclear power plants (ADAMS Accession Number ML18115A007). On March 21, 2018, FES, FirstEnergy Nuclear Generation (FENGEN), and FENOC filed for bankruptcy. On July 26, 2019, FES/FENOC submitted a letter to the NRC withdrawing the April 25, 2018, certification of permanent cessation of power operations for Davis-Besse Nuclear Power Station and Perry Nuclear Power Plant (ADAMS Accession Number ML19207A097). The NRC continues to maintain focus on public health and safety and the protection of the environment. This will include a continuous evaluation by inspectors to determine whether the licensee's financial condition is impacting safe operation of the plant.

INSPECTION SCOPES

Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors performed plant status activities described in IMC 2515 Appendix D, Plant Status and conducted routine reviews using IP 71152, Problem Identification and Resolution. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.

71111.04Q - Equipment Alignment Partial Walkdown Sample (IP Section 03.01)

(1) B train of motor control center and switchgear ventilation and battery exhaust systems on September 4 to 5, 2019

(2) Division 1 diesel generator and associated subsystems while the division 2 diesel generator was inoperable for a surveillance test on September 30, 2019

71111.05Q - Fire Protection Quarterly Inspection (IP Section 03.01)

(1) Fuel handling building 620 foot elevation on September 10, 2019

(2) Auxiliary building 599 foot elevation fire zone 1AB-2 on September 27, 2019

(3) Intermediate building 620 foot elevation fire zone 0IB-3 on September 27, 2019

(4) Service building 604 foot and 620 foot elevations on September 27, 2019

71111.06 - Flood Protection Measures Inspection Activities - Internal Flooding (IP Section 02.02a.)

(1) Liquid radwaste sumps system following EOP-3, Secondary Containment Control, procedure entry due to back leakage into the B residual heat removal pump room sump on July 9 and 10, 2019

71111.11Q - Licensed Operator Requalification Program and Licensed Operator Performance Licensed Operator Performance in the Actual Plant/Main Control Room (IP Section 03.01)

(1) The inspectors observed and evaluated licensed operator performance in the Control Room during plant cooldown following a turbine trip and reactor scram due to a turbine overspeed testing malfunction on July 27, 2019. Additionally the inspectors observed and evaluated licensed operator performance in the Control Room for the subsequent reactor startup on July 31, 2019.

Licensed Operator Requalification Training/Examinations (IP Section 03.02) (1 Sample)

(1) The inspectors observed and evaluated an operations crew-evaluated scenario in the plant training simulator on July 22, 2019.

71111.12 - Maintenance Effectiveness Routine Maintenance Effectiveness Inspection (IP Section 02.01)

(1) Leak detection system following drywell identified leakage increase on August 8 to 21, 2019

(2) Review of generator excitor brush inspections following turbine trip and reactor scram on August 8 to 19, 2019

71111.13 - Maintenance Risk Assessments and Emergent Work Control Risk Assessment and Management Sample (IP Section 03.01)

(1) Plant risk evaluation with the plant shutdown and cooled down during troubleshooting and corrective efforts associated with the turbine trip and reactor scram that occurred during turbine mechanical overspeed testing on July 29 to 30, 2019

(2) Risk management review for extended low power operations with the main generator offline for circuitry repairs on August 7 to 19, 2019

(3) Risk management review for parameters indicating a small leak in the drywell on August 21 to 26, 2019

(4) Risk assessment and management for 5A feedwater heater extraction steam supply line pin hole steam leak on inside edge of elbow with housekeeping clamp installed on August 26 to 28, 2019

71111.15 - Operability Determinations and Functionality Assessments Operability Determination or Functionality Assessment (IP Section 02.02)

(1) Roll-up fire barrier between diesel work shop and diesel generator hallway functionality assessment on July 11 to 12, 2019

(2) Diesel driven fire pump functionality assessment on July 15 to 16, 2019

(3) Flex lake water pump 2 functionality assessment on August 19 to 23, 2019

(4) Division 1 emergency diesel generator circulating lube oil pump and lube oil heater functionality assessment August 19 to 23, 2019

(5) Nuclear closed cooling drywell inboard isolation valve failed fuse on August 30 to September 5, 2019

(6) Turbine control valve number 3 reactor protection system lack of indications during surveillance testing and unplanned Limiting Condition for Operation entry on September 15 to 16, 2019

71111.19 - Post-Maintenance Testing Post-Maintenance Test Sample (IP Section 03.01)

(1) Rod control and information system circuit card replacement testing on July 15, 2019

(2) A reactor recirculation loop flow control valve controller testing following replacement of Modicon I/O card 'GND'/CR on July 23, 2019

(3) Diesel driven fire pump test following replacement of coolant relief valve and alternator on July 24, 2019

(4) D main steam line tunnel high temperature functional following agastat time delay relay replacement on August 16, 2019

(5) B flex lake water pump testing following breaker linkage repair on August 29, 2019

(6) B reactor feedwater booster pump test following seal replacement on August 30 to September 3, 2019

71111.20 - Refueling and Other Outage Activities Refueling/Other Outage Sample (IP Section 03.01)

(1) The inspectors evaluated forced outage activities following turbine trip and subsequent reactor scram due to the turbine overspeed testing event on July 27 to 30, 2019.

71111.22 - Surveillance Testing Surveillance Tests (other) (IP Section 03.01)

(1) SVI-E31-T5311-B, Turbine Building Main Steam Line Temperature High Channel Functional for 1E31-N361B and 1E31-N361C on July 17 to 23, 2019

(2) SVI-R43-T1317, Diesel Generator Start and Load Division 1 on August 19 to 23, 2019

(3) PTI-N32-P0001, Turbine Overspeed Protection Devices Trip and Turbine Lube Oil Pump Starts Weekly Test on August 26 to 29, 2019

Inservice Testing (IP Section 03.01) (1 Sample)

(1) SVI-E22-T2001, High Pressure Core Spray Pump and Valve operability test on August 8 to 13, 2019

FLEX Testing (IP Section 03.02) (1 Sample)

(1) Flex lake water pump A and B annual functional test on August 26 to 29, 2019

71114.06 - Drill Evaluation Drill/Training Evolution Observation (IP Section 03.02)

(1) a simulator-based licensed operator requalification training on July 22,

71124.06 - Radioactive Gaseous and Liquid Effluent Treatment Walk Downs and Observations (IP Section 02.01)

The inspectors walked down the following gaseous and liquid radioactive effluent monitoring and filtered ventilation systems to assess the material condition and verify proper alignment according to plant design:

(1)

• Unit 1 Plant Vent

• Unit 2 Plant Vent

• Off-Gas Vent

• Turbine Building/Heater Bay Vent Calibration and Testing Program (Process & Effluent Monitors) (IP Section 02.02) (1 Sample)

The inspectors reviewed the following gaseous and liquid effluent monitor instrument calibrations and tests:

(1)

• Unit 1 Vent Noble Gas Monitor 1D17-K786

• Off-Gas Vent Noble Gas Monitor 1D17-K836

• LRW to ESW Radiation Monitor D17-K606

• Liquid Radwaste High Flow G50-N445

Sampling and Analysis (IP Section 02.03) (1 Sample)

(1) Radioactive Effluent Sampling and Analysis Activities

• Unit 1 Plant Vent, July 25, 2019

• Unit 2 Plant Vent, July 25, 2019

• Off-Gas Vent, July 25, 2019

• Turbine Building/Heater Bay Vent, July 25, 2019 Effluent Discharges

• None were available for review during this inspection

Instrumentation and Equipment (IP Section 02.04) (1 Sample)

The inspectors reviewed the following radioactive effluent discharge system surveillance test results:

(1) Work Order 200767889, Annulus Exhaust Gas Treatment System Train A Flow and Filter Operability Test

Dose Calculations (IP Section 02.05) (1 Sample)

(1) Liquid and Gaseous Discharge Permits

• SVI-D175268 Effluent Sampling/Analysis & Dose Assessment, March 2018

• SVI-D175268 Effluent Sampling/Analysis & Dose Assessment, July 2018

• SVI-D175268 Effluent Sampling/Analysis & Dose Assessment, February 2019 The inspectors also reviewed the following land use census

• August 2018 Abnormal Gaseous or Liquid Tank Discharges

• None were available for review during this inspection

71124.07 - Radiological Environmental Monitoring Program Site Inspection (IP Section 02.01)

(1) The inspectors reviewed the following:

Walkdowns, Calibrations, and Maintenance Record Review

• Environmental air sampling station; location number 1

• Environmental air sampling station; location number 3

• Environmental air sampling station; location number 6

• Environmental air sampling station; location number 7

• Environmental dosimeter location number 1

• Environmental dosimeter location number 3

• Environmental dosimeter location number 6

• Environmental dosimeter location number 10 Environmental Sample Collections and Preparation Observation

• Environmental air sampling station; location number 6

• Environmental air sampling station; location number 7

• Surface water sample; location number 59

• Municipal water sample; location number 39 Licensee Actions in Response to Missed Sample, Inoperable Sampler, Lost TLD or Anomalous Measurement

• Environmental air sampling station location number 35; loss of power; 07/17/2019

• Broadleaf vegetation sampling location number 70; missed sample; 07/2019 Sampling Program for the Potential of Licensed Material Entering Groundwater

• [List 3-5 structures]

• [List 3-5 structures]

Groundwater Protection Initiative (GPI) Implementation (IP Section 02.02) (1 Sample)

(1) The inspectors evaluated the licensees voluntary groundwater protection initiative.

OTHER ACTIVITIES - BASELINE

71151 - Performance Indicator Verification MS08: Heat Removal Systems (IP Section 02.07)

(1) Unit 1 (July 1, 2018 - June 30, 2019)

MS09: Residual Heat Removal Systems (IP Section 02.08) (1 Sample)

(1) Unit 1 (July 1, 2018 - June 30, 2019)

MS10: Cooling Water Support Systems (IP Section 02.09) (1 Sample)

(1) Unit 1 (July 1, 2018 - June 30, 2019)

PR01: Radiological Effluent Technical Specifications/Offsite Dose Calculation Manual Radiological Effluent Occurrences (RETS/ODCM) Radiological Effluent Occurrences Sample (IP Section 02.16) (1 Sample)

(1) October 2018 - June 2019

71152 - Problem Identification and Resolution Annual Follow-up of Selected Issues (IP Section 02.03)

(1) CR 2019-01671; Failure to perform adequate preventive maintenance on the main generator exciter brushes, led to brush failures, loss of generator field excitation, generator lockout, and reactor scram

(2) CR 2019-04667; Emergency Service Water A inoperability due to control power fuses being improperly installed

71153 - Follow-up of Events and Notices of Enforcement Discretion Event Report (IP Section 03.02)

(1) LER 05000440/2019-001-00, Loss of Generator Excitation Causes Turbine Trip and Reactor Protection System Actuation (ADAMS accession: ML19114A443). Details regarding the disposition of this LER are documented in the Results section.

Personnel Performance (IP Section 03.03) (1 Sample)

(1) On July 27, 2019, at 7:29 p.m., a turbine trip and subsequent reactor scram occurred during the performance of turbine mechanical overspeed testing. The inspectors evaluated the unplanned event and associated licensee performance.

INSPECTION RESULTS

Failure to Properly Seat Essential Service Water A Pump Control Power Fuses results in Loss of Safety Function Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green [H.9] - Training 71152 Systems NCV 05000440/2019003-01 Open A self-revealed finding of very low safety significance (Green) and associated non-cited violation of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, for the licensee's failure to accomplish work activities in accordance with the instructions, procedures, or drawings. Specifically, the licensee failed to properly install the A essential service water control power fuses in accordance with SOI-R22, Metal Clad Switchgear 5-15 kV, paragraph 2.3 such that the fuses were able to back out and cause an unplanned inoperability of several safety related systems during routine breaker operation.

Description:

On May 24, 2019, after the start of the emergency service water (ESW) pump A for pump packing adjustment, two unexpected alarms annunciated in the control room indicating residual heat removal (RHR) A and Division 1 diesel generator were out of service. The operators walked down panels and noted that panel 1H13P0601 indications were illuminated for ESW Pump A Breaker Out / Power Loss / Overload. Upon investigation, the licensee discovered that the closing control power fuses (AX fuses) for the ESW A breaker were not seated properly. This resulted in the inoperability of the ESW A system and the following systems: low pressure core spray (LPCS) room cooler and LPCS, A RHR, A emergency closed cooling (ECC), and division 1 emergency diesel generator. The inspectors reviewed plant narrative logs and determined that the licensee last manipulated the AX fuses on April 25, 2019. The licensee made an eight-hour non-emergency notification to the NRC as required by 10 CFR 50.72(b)(3)(v)(D) Accident Mitigation for the inoperability of a single train system, LPCS, in Modes 1, 2, and 3.

Upon further review, the inspectors determined that the licensee had failed to do an adequate past operability review. Specifically, the licensee had determined that a non-licensed operator had not properly installed the AX fuses on April 25, 2019, but did not consider seismic implications while the fuses were improperly installed. From the time the licensee installed fuses on April 25, 2019, until the alarms annunciated in the control room on May 25, 2019, plant operators started the ESW A pump four times. It is reasonable to assume that the fuses were not completely installed and backed out because of the vibrations caused by breaker operations, both closing and opening. At the time of this inspection report issuance, the licensee continued to assess the seismic concerns raised by the inspectors.

Corrective Actions: Fuse block AX, for the ESW pump A breaker, was re-seated by the initial responder. Further investigation by the licensee determined that when properly and fully seated, the AX fuse block had no issues relative to staying in place with adequate tightness. The licensee is also pursuing a change to the Perry Switchgear and Electrical Operator Safety and Administration lesson plans to include a maintenance fundamental discussion of seating fuses and fuse blocks with physical and visual cues. The corrective actions included, but were not limited to, the importance of fully seated fuses within their receptors, ways to identify fuses and fuse block receptors which may be loose, the need to identify the issue in the work management / corrective action processes, and non-licensed operator training for seating fuses and fuse blocks, to include physical or visual cues, during a non-licensed operator continuing training cycle.

Corrective Action References: CR 2019-04667

Performance Assessment:

Performance Deficiency: The licensee's failure to appropriately install fuses such that they maintained their seismic qualification and would not back out as a result of routine breaker operation or during a seismic event was a performance deficiency.

Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Equipment Performance attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, the failure to ensure that the ESW pump A control power fuses were properly installed negatively impacted the licensee's ability to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences (i.e., core damage).

Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Under Exhibit 2 -

Mitigating Systems Screening Questions, the inspectors answered Yes to Question #2, Does the finding represent a loss of system and/or function? The inspectors continued the significance determination with a detailed risk evaluation.

A Senior Reactor Analyst performed a detailed risk evaluation using the Standardized Plant Analysis Risk Model (SPAR) version 8.60 for Perry. The ESW pump control power fuses were last manipulated approximately 30 days prior to the alarm indications. During the 30-day period, the pump ran successfully several times. The SRA assumed that only seismic events would challenge the operation of the pump in the degraded condition. Therefore, only seismic initiating events were evaluated for a change in risk. The ESW pump A was modeled as a failure to start for an exposure period of 30 days.

The result was a change in core damage frequency less than 1E-7/yr., which is a finding of very low safety significance. The dominant core damage sequence was a seismic-induced loss of offsite power, failure of both ESW pumps, and the failure to recover power.

Cross-Cutting Aspect: H.9 - Training: The organization provides training and ensures knowledge transfer to maintain a knowledgeable, technically competent workforce and instill nuclear safety values. Specifically, the licensee failed to provide appropriate training to non-licensed operators, instead trusting in skill of the craft, to ensure fuses had been installed correctly and identify conditions where they were not on April 25, 2019.

Enforcement:

Violation: Title 10 of the Code of Federal Regulations, Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, requires, in part, activities affecting quality shall be prescribed by documented instructions, procedures, or drawings, of a type appropriate to the circumstances and shall be accomplished in accordance with these instructions, procedures, or drawings.

Contrary to the above on April 25, 2019, the licensee failed to accomplish activities affecting quality in accordance with instructions, procedures, and drawings. Specifically, the licensee failed to install the A ESW pump breaker AX control power fuses in accordance with instruction SOI-R22, Metal Clad Switchgear 5-15 kV, paragraph 2.3, such that the fuses backed out and caused an unplanned inoperability of several safety-related systems on May 24, 2019.

Enforcement Action: This violation is being treated as an non-cited violation, consistent with Section 2.3.2 of the Enforcement Policy.

Observation: Failure to Perform Adequate Preventive Maintenance on the Main 71152 Generator Exciter Brushes Results in Generator Lockout and Reactor Scram.

The inspectors performed a detailed review of CR 2019-01671, Main Generator/Main Turbine Trip Resulting in a Reactor Scram. The licensee's casual evaluation for this issue determined that the apparent cause of the generator lockout and reactor scram was that the preventive maintenance on the generator field excitation brushes was less than adequate.

Specifically, the licensee stated that the criteria for brush inspection and replacement should have added margin to ensure brushes could not wear to the point where brush spring tension could not be maintained between inspection intervals. The inspectors determined that the vendor manual recommended that the exciter brush inspection be performed on a weekly basis and that acceptance criteria for that inspection for exciter brush length be at least one-eighth of one inch. The licensee's maintenance plan called for a monthly inspection of the the exciter brushes with at least one-eighth of one inch acceptance criteria. The inspectors determined that the longer inspection frequency in conjunction with the acceptance criteria resulted in the brushes degrading between inspection intervals such that spring tension was no longer maintained and allowed the brushes to chatter which ultimately led to the flashover event. The licensee's corrective actions altered the acceptance criteria and provided extra margin to ensure that the monthly inspection periodicity would not result in the brush spring pressure reduction such that the brushes would not chatter.

The inspectors also reviewed past operating experience and determined that there have been numerous flashover events in the industry over the past thirty-five years, with most caused by exciter brushes wearing down to the point that brush spring tension was not maintained.

Four of these events were very similar to the licensee's event in February 2019. In 1985 a nuclear operating plant experienced a flashover event and determined that the root cause was the loss of field excitation due to excessive brush wear. The brushes had been inspected seventeen days before the failure and reported as acceptable. As a result of that event exciter brush inspection was changed from once per month to every fourteen days. In 2004 another nuclear operating plant had a flashover event. The root causes of the event were the Generator/Exciter vendor recommended inspections were not implemented when the GE generator was installed in 1986 and the lessons learned from a similar Alterrex brush failure in 1993 were not applied. The fourth event occurred in November 2005 at another nuclear power plant, the cause being the exciter brushes being worn down below the point where exciter brush tension could not be maintained. These four events were opportunities for the licensee to evaluate its preventive maintenance plans for the main generator exciter brushes.

Following the event on February 25, 2019, the licensee polled the industry and found that the four plants that responded to its inquiry all inspect their main generator exciter brushes either bi-weekly or weekly. Also, only one plant used the same or similar brush replacement criteria. The other three plants reported that they had a more conservative recommendation for replacing brushes. According to the licensee, the results of the industry information shows that the past replacement criteria was less than adequate and needed to be changed.

Observation: Failure to Properly Seat Essential Service Water A Pump Control 71152 Power Fuses results in Loss of Safety Function The inspectors performed a detailed review of CR 2019-04667, Indications received for ESW Pump A Breaker Power Loss. The event caused the inoperability of a train of safety-related equipment, the loss of safety function for the Low Pressure Core Spray System, a condition prohibited by the plant's technical specifications, and caused the licensee to submit an eight-hour non-emergency Reactor Plant Event Notification Report in accordance with 10 CFR 50.72(b)(3)(v). The licensee identified that the cause of the loss of control power was due to a non-licensed operator (NLO) not recognizing that the control power fuses for emergency service water pump breaker A were not fully seated following installation.

Corrective actions implemented by the licensee included, but were not limited to, changes to switchgear and electrical operator safety training lesson plans and a discussion of the event during non-licensed operator training. The inspectors noted that the licensee's procedures used by the NLO, specifically SOI-R22, Metal Clad Switchgear 5-15 KV and SOI-P45/P49, Emergency Service Water and Screen Wash Systems, did not appear to have acceptance criteria to ensure fuses are fully seated. The acceptance criteria in the procedures was a continuity test to ensure fuse continuity. The inspectors determined there did not exist criteria to ensure proper seating of the fuse holder/clip. Through discussion with the licensee, the inspectors determined that the NLOs received initial training in their switchgear and electrical operator safety training classes on how to properly install and remove fuse clips/holders but relied upon skill of the craft thereafter. The inspectors further determined, through conversation, that NLOs are not retrained on that skill of the craft activity. The inspectors determined that the current acceptance criteria of proper fuse installation through fuse continuity checks did not reveal, and likely would not reveal, incorrect fuse installation such as that occurred on April 25, 2019, which resulted in an unplanned inoperability of several safety-related systems.

LER 2019-001-00 for Perry Nuclear Power Plant, Loss of Generator Excitation Causes Turbine Trip and Reactor Protection System Actuation.

Cornerstone Significance Cross-Cutting Report Aspect Section Initiating Events Green None (NPP) 71153 FIN 05000440/2019003-02 Open A self-revealed finding of very low safety significance (Green) was identified for the licensee's failure to perform adequate preventive maintenance on the main generator exciter brushes, which led to brush failures, loss of generator field excitation, generator lockout, and reactor scram. No violation of regulatory requirements was identified.

Description:

On February 25, 2019, at 12:24 a.m., the Main Generator received a lockout signal from the Loss of Excitation relay following its associated time delay relay actuation. In the six seconds prior to the lockout signal, there was a decrease in grid voltage from approximately 346kV to approximately 311kV. During that same six second time period Main Generator Mega Volt Amps Reactive (MVARs) increased significantly. The equipment responded as expected for a valid loss of excitation event because once the field current stops, the generator acts as an induction motor instead of a synchronous generator and draws MVARs from the grid to support its magnetic field. The flow of MVARs in turn pulled down local grid voltage until the generator tripped, then voltage recovered. There were numerous annunciators and equipment indications following the scram due to the voltage perturbation and expected bus transfers.

Investigation into the cause of the generator trip found evidence of collector ring flashover for the inboard (positive polarity) exciter collector ring. This was confirmed as the cause of the generator loss of excitation event following visual inspection of the generator exciter. Vendor manual documentation and industry operating experience support that conclusion and mention collector ring flashover as the most prevalent cause of loss of excitation events within the industry. The generator brushes and collector rings are a vital part of the ability of the generator to work as a synchronous machine to convert mechanical, rotating energy into electrical energy. The collector rings, or slip rings, are metal rings heat shrunk onto the end of the generator rotor shaft that are maintained electrically isolated from the shaft and each other. They provide a contact surface for the generator brushes to ride upon as the shaft rotates to deliver the field current developed by the generator voltage regulator.

The slip rings are highly polished steel forgings with spiral grooves machined in the outer surface. The grooves allow the field current to even out across a few of the ring surfaces and allow a space for air flow and cooling. Each exciter brush is offset by approximately 1/4 inch.

Therefore, not all brushes ride on the same ring surface and all the ring surfaces can wear more evenly. The progression of this event is that one or more of the exciter inboard brushes reached its wear limit. The brush spring(s) was out of range to ensure positive, adequate downward pressure to the brush(es) to assure proper electrical connection was maintained.

When the brush spring(s) pressure was limited, the brush(es) began to bounce/chatter. Each time a brush fails to make adequate connection, the current it is carrying is broken. As the radial distance between the brush and collector ring surface initiates the current is strong enough to cross over the minimal air gap; this creates an arc.

An arc can impose physical damage and create surface defects in the form of pits (peaks and valleys) that act as increased friction to increase the wear rate of other brushes that may ride on the same ring surface. The longer this bouncing and subsequent arcing continues conditions for impaired electrical connection degrade in an exponential manner. Arcing also creates an environment for increased heating of the collector ring and brushes as high current discharges repeatedly occur. Once the radial distance between the brush and collector ring surface increases enough, the air gap becomes great enough that the arc does not connect the brush and collector ring, but finds the path of least resistance, some external source to ground. This is what is referred to as a flashover event.

Licensee Event Report (LER) 05000440/2019-001-00, Loss of Generator Excitation Causes Turbine Trip and Reactor Protection System Actuation, was submitted by the licensee for this event.

Corrective Actions: The licensee revised the preventive maintenance plan to modify the brush replacement criteria to add margin to account for the monthly inspection frequency.

The licensee also added detail to the brush vibration preventive maintenance task to ensure all exciter brushes are monitored during routine task performances.

Corrective Action References: CR 2019-01671

Performance Assessment:

Performance Deficiency: The inspectors determined that the inadequate preventative maintenance on the main generator exiter brushes, which resulted in a reactor scram, was a performance deficiency which was within the licnesee's ability to foresee and prevent.

Specifically, licensee procedure NOP-WM-1001, Order Planning Process, paragraph 4.3.4.6.c, requires the licensee to include appropriate acceptance criteria. However, on February 4, 2019, the monthly brush inspection failed to detect and replace the main generator's exciter brushes prior to having the brushes degrade below the minimum acceptable length, resulting in the loss of brush spring pressure, exciter arching, loss of the generator and reactor scram.

Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Equipment Performance attribute of the Initiating Events cornerstone and adversely affected the cornerstone objective to limit the likelihood of events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the inadequate preventative maintenance of the generator exciter brushes impacted the Initiating Events cornerstone and adversely affected the cornerstone objective to limit the likelihood of events that upset plant stability and challenge critical safety functions during shutdown as well as power operations.

Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At-Power. The inspectors determined that this finding was a licensee performance deficiency of very low safety significance (Green) because although the finding did cause a reactor trip (scram), it did not cause the loss of mitigation equipment relied upon to transition the plant from the onset of the trip to a stable shutdown condition.

Cross-Cutting Aspect: Not Present Performance. No cross cutting aspect was assigned to this finding because the inspectors determined the finding did not reflect present licensee performance.

Enforcement:

Inspectors did not identify a violation of regulatory requirements associated with this finding.

EXIT MEETINGS AND DEBRIEFS

The inspectors verified no proprietary information was retained or documented in this report.

• On September 30, 2019, the inspectors presented the integrated inspection results to Mr. F. Payne, Site Vice President, and other members of the licensee staff.

• On July 26, 2019, the inspectors presented the Radiation Protection Baseline Inspection inspection results to Mr. F. Payne; Site Vice President, and other members of the licensee staff.

• On October 9, 2019, the inspectors presented the Integrated Baseline Inspection inspection results to Mr. F. Payne, Site Vice President, and other members of the licensee staff.

DOCUMENTS REVIEWED

Inspection Type Designation Description or Title Revision or

Procedure Date

71111.04Q Drawings 302-0351-00000 Standby Diesel Generator Starting Air GG

2-0352-00000 Standby Diesel Generator Fuel Oil System LL

2-0353-00000 Standby Diesel Generator Lube Oil T

2-0354-00000 Standby Diesel Generator Jacket Water V

Procedures VLI-M23/24 MCC, Switchgear and Miscellaneous Electrical Equipment 8

Area HVAC System

VLI-R44 Division 1 and 2 Diesel Generator Starting Air System 6

VLI-R45 Division 1 and 2 Diesel Generator Fuel Oil System (Unit 1) 5

VLI-R46 Division 1 and 2 Diesel Generator Jacket Water Systems 4

(Unit 1)

VLI-R47 Division 1 and 2 Diesel Generator Lube Oil 7

71111.05Q Procedures FPI-0FH Fuel Handling Building 6

FPI-0IB Intermediate Building 10

FPI-1AB Auxiliary Building Unit 1 4

FPI-SB Service Building 3

71111.06 Corrective Action CR 2019-05701 Unplanned Entry into EOP-03 Secondary Containment 07/03/2019

Documents Control

Drawings 302-0740-00000 Liquid Radwaste Sumps System FF

71111.11Q Miscellaneous OTLC- OTLC-3058201903_PY-SGC1 - Evaluated Simulator 0

3058201903_PY- Scenario

SGC1

Procedures Evolution Specific Cycle 18 - Startup 130 - Part 1 0

Reactivity Plan

Evolution Specific Cycle 18 - Startup 130 - Part 2 0

Reactivity Plan

IOI-8 Shutdown by Manual Reactor Scram 8

ONI-C71-1 Reactor Scram 23

71111.12 Miscellaneous ATL-2019-0605- Perry System Health Report 2018

ATA-030

Work Orders 200556421 Replace Agastat E7000 TD Relay 08/15/2019

200753231 Generator Brush Monthly Inspection Aug 8 - 19,

2019

Inspection Type Designation Description or Title Revision or

Procedure Date

71111.13 Corrective Action CR 2019-06321 Turbine Trip during Weekly Turbine Overspeed Test 07/27/2019

Documents CR 2019-06720 Drywell Identified Leakage Rose 0.1 GPM 08/10/2019

CR 2019-06983 Parameters Indicate a Small Leak in Drywell 08/20/2019

Miscellaneous 601236050 PY-1E31 Leak Detection System

Procedures NOP-OP-1005 Shutdown Defense In Depth 17

NOP-OP-1007 Risk Management 30

PAP-1924 Risk-Informed Safety Assessment and Risk Management 9

PAP-1925 Shutdown Defense In Depth Assessment and Management 23

71111.15 Corrective Action CR 2019-05162 FLEX Equipment Remains Unavailable for Maintenance and 06/12/2019

Documents Allowed Out of Service Time, per FLEX Specifications 2,1,2,

2.1.4, and 2.1.8, will be Exceeded

CR 2019-05928 Coolant Leak from Diesel Fire Pump when Starting Engine - 07/12/2019

Unplanned Impairment

CR 2019-05934 Diesel Driven Fire Pump Belt Loose and Coolant Leak 07/13/2019

CR 2019-06928 FLEX Lake Water Pump #2 Main Panel Breaker Failed to 08/08/2019

Operate

CR 2019-06981 Failure of Div 1 DG Lube Oil Control Power Fuse and 42 08/20/2019

Contact Following Monthly SVI Run

CR 2019-07260 Blown Mainline Fuse for Nuclear Closed Cooling Drywell 08/30/2019

Isolation Valve

CR 2019-07639 Pressure Switch 1C71N0005A Failed to Actuate During the 09/15/2019

Performance of SVI-N31-T1151 Step 5.1.4.44 in

Accordance with Order Number 200625793

Drawings 4549-43-036-9 Turbine Control Diagram 9A

B-208-151 Turbine Control (EHC) Control Valve Logic A

Miscellaneous eSOMS Plant Plant Narrative Logs 08/20/2019

Narrative Logs

eSOMS Plant Day and Evening Shift July 10 and

Narrative Logs 12, 2019

Procedures FPI-A-T03 Champion Fire Pump Operator Instructions 6

FSG-40.5 FLEX Generator Loading and Plant System Operations 5

NOP-LP-7300 FLEX Program for the Perry Nuclear Power Plant 2

NOP-OP-1009 Operability Determinations and Functionality Assessments 8

Inspection Type Designation Description or Title Revision or

Procedure Date

PAP-1910 Fire Protection Program 40

PDB-R0003 Flex Specifications 4

PTI-P54-P0041 Semiannual Fire Door Inspection 13

SVI-N31-T1151 Main Turbine Valve Exercise Test 09/16/2019

Work Orders 200625793 PY-SVI-N31T1151 09/15/2019

WO 200800884 Pressure Switch 1C71N0005A Failed to Actuate During the 0

Performance of SVI-N31-T1151

71111.19 Procedures SVI-E31-T0078-D MSL Tunnel Temperature High Channel D Functional for 08/27/2010

1E-31-N604D and 1E31-N605D

Work Orders 200417053 Replacement of Scan Control Card, Vector Receiver Card, 07/15/2019

and Several Addition Circuit Cards

200556421 Replace Agastat E7000 TD Relay 05/13/2019

200790224 Replace Modicon I/O card 'GND'/CR 07/23/2019

200795274 Diesel Fire Service Pump, Replace Jacket Water Heat 07/18/2019

Exchanger Relief Valve and Alternator

200795656 Replace Mechanical Seal if Required Based on Current 09/12/2019

Troubleshooting being Performed on Feedwater Booster

Pump B

200798951 Reactor Feedwater Booster Pump B Motor Determ and Seal 09/22/2019

Replacement

200799218 Flex Lake Water Pump 08/19/2019

71111.20 Corrective Action CR 2019-06321 Turbine Trip during Weekly Turbine Overspeed Test 07/27/2019

Documents

Procedures IOI-0004 Shutdown 26

IOI-0005 Maintaining Hot Shutdown 22

ONI-C71-1 Reactor Scram 23

71111.22 Corrective Action 2019-06321 Prompt Functionality Assessment Form 08/14/2019

Documents

Miscellaneous DI-236 Calculation - Perry Low Pressure Turbine Missile Probability 05/02/2013

Analysis

DI-236 Calculation Addendum - Perry Low Pressure Turbine Missile 1

Probability Analysis

Procedures 200798399 Turbine Overspeed Protection Devices Trip and Turbine 08/06/2019

Lube Oil Pump Starts Test

Inspection Type Designation Description or Title Revision or

Procedure Date

SOI-R43 Division 1 and 2 Diesel Generator System 48

SVI-R43-T1317 Diesel Generator Start and Load Division 1 24

Work Orders 200711841 Flex Water Pump Functional Testing 08/28/2019

200729023 SVI-E22-T2001, High Pressure Core Spray Pump and Valve Aug 8 - 13,

Operability Test 2019

200729376 SVI-E31-T5311-B, (92D) Turbine Building Main Steam Line 07/17/2019

Temperature High Channel Functional for 1E31-N361B and

1E31-N361C

71114.06 Miscellaneous OTLC- OTLC-3058201903_PY-SGC1 - Evaluated Simulator 0

3058201903_PY- Scenario

SGC1

PNPP10481 Emergency Notification Cover Sheet Rev

01/15/2018

PNPP7794 Initial Notification Form Rev

01/09/2018

PNPP9100 Alert Notification Message Rev

04/24/2019

71124.06 Corrective Action SA-BN-2019- 2019 RETS/REMP Self Assessment in Preparation for the 06/04/2019

Documents 1476 NRC Audit

Corrective Action 2019-06218 2019 NRC Effluent Inspection Noble Gas Observation 07/24/2019

Documents ATA-2019-10132 2019 Effluents NRC Followup Action for D17 Flowrate and 08/09/2019

Resulting from Total Volume

Inspection

Miscellaneous Offsite Dose Calculation Manual 22

2018 Annual Environmental and Effluent Release Report 04/29/2019

2017 Annual Environmental and Effluent Release Report 05/01/2018

Results of Radiochemistry Cross Check Program 3rd Quarter

2017 through

1st Quarter

2019

Procedures ICI-C-D17-20 System Calibration Instruction 9

ODCM Offsite Dose Calculation Manual 22

SOI-D17 Airborne Radiation Monitoring System (Effluents) 15

SOI-D17A Process Radiation Monitoring System 21

Inspection Type Designation Description or Title Revision or

Procedure Date

SVI-D17-T5268 Effluent Sampling/Analysis & Dose Assessment 11

Self-Assessments SA-BN-2019- 2019 RETS/REMP self assessment for the NRC audit 06/04/2019

1476

Work Orders 2001703018 PY-SVI-D17T5268 Effluent Sampling/Analysis & Dose 03/07/2018

Assessment

2001753748 PY-SVI-D17T5268 Effluent Sampling/Analysis & Dose 04/08/2019

Assessment

20017753746 PY-SVI-D17T5268 Effluent Sampling/Analysis & Dose 02/04/2019

Assessment

200648169 Unit 1 Vent Noble Gas Radiation Monitor Calibration for 05/18/2017

1D17-K786

200671740 LRW to ESW Radiation Monitor Channel Calibration for 02/14/2018

D17-K606

200693314 Liquid Radwaste High Flow Calibration for G50-N445 08/29/2018

200693360 Off-Gas Vent Noble Gas Radiation Monitor Calibration for 09/18/2018

1D17-K836

200695479 SVI-M15-T2140-A Annulus Exhaust Gas Treatment System 03/30/2018

Train A Flow and Filter Operability Test

200703022 PY-SVI-D17T5268 Effluent Sampling/Analysis & Dose 07/05/2018

Assessment

200717059 Unit 1 Vent Noble Gas Radiation Monitor Calibration for 05/15/2019

1D17-K786

200767889 SVI-M15-T2140-A Annulus Exhaust Gas Treatment System 09/16/2018

Train A Flow and Filter Operability Test

2007769871 Unit 1 Vent Noble Gas Radiation Monitor Calibration for 10/18/2018

1D17-K786

71124.07 Calibration Environmental Sample Pump 7004 Calibration 04/15/2019

Records Environmental Sample Pump 09 Calibration 04/15/2019

Environmental Sample Pump 7004 Calibration 05/02/2018

SVI-D51-T0307-A Meteorological Monitoring System A, 60 Meter Wind Speed 11/30/2018

Calibration

SVI-D51-T0308-A Meteorological Monitoring System A 60 Meter Wind 05/07/2019

Direction Calibration

SVI-D51-T0308-B Meteorological Monitoring B Channel 60 Meter Wind 05/19/2019

Inspection Type Designation Description or Title Revision or

Procedure Date

Direction Calibration

Corrective Action 2018-11953 New XOQ/DOX Evaluation 05/22/2018

Documents

Corrective Action 2019-06219 2019 NRC Effluent Inspection ODCM Comment 07/24/2019

Documents 2019-06256 Delays Performing, Analyzing and Incorporating the Effluent 07/25/2019

Resulting from Dispersion Factors (XOQ/DOQ)

Inspection

Engineering 2018 Annual Report of the Meteorological Program 02/05/2019

Evaluations Comparison of 10-Year (2006-2016) Versus Historical 7 Site 01/01/2018

Year X/Qs and D/Qs from the Perry Nuclear Power Station

Five-Year Update of Groundwater Flow Characteristics 05/04/2016

Report

Procedures REMP-0006 Fish Sampling Instruction 10

REMP-0009 Surface and Drinking Water Sampling 9

REMP-0012 Food Product Sampling 8

REMP-0013 Sampling Locations 12

71151 Miscellaneous NOBP-LP-4012- Mitigating Systems Performance Index (MSPI) Unavailability 07/01/2018 -

Index (UAI) & Unreliability Index (URI) for Heat Removal 06/30/2019

System (RCIC)

NOBP-LP-4012- Mitigating Systems Performance Index (MSPI) Unavailability 07/01/2018 -

Index (UAI) & Unreliability Index (URI) for Residual Heat 06/30/2019

Removal (RHR)

NOBP-LP-4012- RETS/ODCM Radiological Effluent Occurrence 4th Quarter

2018 through

2nd Quarter

2019

NOBP-LP-4012- Mitigating Systems Performance Index (MSPI) Unavailability 07/01/2018 -

Index (UAI) & Unreliability Index (URI) for Emergency 06/30/2019

Service Water (ESW)

71152 Corrective Action CR 2019-01671 Main Generator/Main Turbine Trip Resulting in a Reactor 02/25/2019

Documents Scram

CR 2019-04667 Indications Received for ESW Pump A Breaker Power Loss 05/24/2019

Procedures NOBP-ER-3009 Plant Health Reporting 16

NOP-SS-8001 FENOC Activity Tracking 04

Inspection Type Designation Description or Title Revision or

Procedure Date

NOP-WM-1001 Order Planning Process 30

NOP-WM-3001 Work Management PM Process 19

SOI-P45/P49 Emergency Service Water and Screen Wash Systems 33

SOI-R22 Metal Clad Switchgear 5-15 KV 28

Work Orders 200753223 Inspect Main Generator & Alterex Brush 12/12/2018

200753225 Inspect Main Generator & Alterex Brush 02/04/2019

71153 Corrective Action CR 2019-01671 Main Generator/Main Turbine Trip Resulting in a Reactor 02/25/2019

Documents Scram

Miscellaneous Scram No. 1-19- Post Scram Restart Report Perry Nuclear Power Plant 07/30/2019

Procedures PYBP-SITE-0019 Post Reactor Scram Evaluation 4

2