IR 05000220/2021004
ML22026A350 | |
Person / Time | |
---|---|
Site: | Nine Mile Point |
Issue date: | 01/26/2022 |
From: | Erin Carfang NRC/RGN-I/DORS |
To: | Rhoades D Exelon Generation Co LLC, Exelon Nuclear |
Carfang E | |
References | |
IR 2021004 | |
Download: ML22026A350 (22) | |
Text
January 26, 2022
SUBJECT:
NINE MILE POINT NUCLEAR STATION, UNITS 1 AND 2 - INTEGRATED INSPECTION REPORT 05000220/2021004 AND 05000410/2021004
Dear Mr. Rhoades:
On December 31, 2021, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at Nine Mile Point Nuclear Station, Units 1 and 2. On January 20, 2022, the NRC inspectors discussed the results of this inspection with Mr. Adam Schuerman, Plant Manager,
and other mem bers of your staff. The results of this inspection are documented in the enclos ed report.
One finding of very low safety significance (Green) is documented in this report. This finding involved a violation of NRC requirements. We are treating this violation as a non-cited violation (NCV) consistent with Section 2.3.2 of the Enforcement Policy.
If you contest the violation or the significance or severity of the violation documented in this inspection report, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN:
Docum ent Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region I; the Director, Office of Enforcement ; and the NRC R esident Inspector at Nine Mile Point Nuclear Station, Units 1 and 2. This letter, its enclosure, and your resp onse (if any) will be m ade available for public inspection and copying at http://www.nrc.gov/reading-rm /a dam s.h tm l and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding.
Sincerely, Erin E. Carfang, Chief Projects Branch 1 Division of Operating Reactor Safety
Docket Nos. 05000220 and 05000410 License Nos. DPR-63 and NPF-69
Enclosure:
As stated
Inspection Report
Docket Numbers: 05000220 and 05000410
License Numbers: DPR-63 and NPF -69
Report Numbers: 05000220/2021004 and 05000410/2021004
Enterprise Identifier: I-2021- 004-0115
Licensee: Exelon Generation Com pany, LLC
Facility: Nine Mile Point Nuclear Station
Location: Oswego, NY
Inspection Dates: October 1, 2021 to December 31, 2021
Inspectors: G. Stock, Senior Resident Inspector C. Kline, Resident Inspector B. Sienel, Resident Inspector J. Demarshall, Senior Operations Engineer T. Fish, Senior Operations Engineer J. Hawkins, Senior Project Engineer P. Ott, Operations Engineer
Approved By: Erin E. Carfang, Chief Projects Branch 1 Division of Operating Reactor Safety
Enclosure
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licens ees performance by conducting an integrated inspection at Nine Mile Point Nuclear Station, Units 1 and 2, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
Failure to Correct a Condition Adverse to Quality with an Uninterruptible Power Supply Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green None (NPP) 71152 Systems NCV 05000220,05000410/2021004-01 Open/Closed The inspectors documented a self -revealed Green non-cited violation (NCV) of Title 10 of the Code of Federal Regulations (10 CFR) Part 50, Appendix B, Criterion XVI, Corrective Action, when Exelon did not ensure that a condition adverse to quality (CAQ) associated with an identified deviation from the normal operating frequency of an uninterruptible power supply (UPS) was promptly identified and corrected. Specifically, on August 1, 2016, after performing maintenance on and placing the UPS -162A in service, Exelon documented an off -normal condition related to the UPS inverter frequency being lower than normal and oscillating.
Exelon closed this issue report (IR) to trend without properly evaluating and correcting the condition, resulting in the failure of the reactor protection system (RPS) UPS-162A and causing the plant to experience a half -scram condition, an isolation of both emergency condensers, and an isolation of the reactor water cleanup (RWCU) system on September 25, 2021.
Additional Tracking Items
Type Issue Num ber Title Report Section Status LER 05000220/2021001-00 LER 2021-001-00, Average 71153 Closed Power Range Monitors Declared Inoperable Due to Trip of Reactor Recirculation Pum p 13 LER 05000220/2021002 -00 LER 2021-002-00, Isolation 71153 Closed of Both Emergency Condensers Due to L oss of UPS-162A
PLANT STATUS
Unit 1 began the inspection period at rated thermal power.
On October 2, 2021, reactor power was reduced to approximately 75 percent for a control rod pattern adjustment, control rod scram time testing, and turbine valve testing and returned to rated thermal power the same day.
On Novem ber 2, 2021, reactor power was reduced to approximately 98 percent to maintain inlet to discharge differential temperature within state limits due to condenser fouling.
On Novem ber 13, 2021, reactor power was reduced to approximately 51 percent to correct the condenser fouling. On November 13, 2021, Unit 1 was returned to rated thermal power.
Unit 1 remained at or near rated thermal power for the remainder of the inspection period.
Unit 2 began the inspection period at rated thermal power. On October 9, November 21, and December 10, 2021, reactor power was reduced to approximately 75 percent to perform control rod channel interference testing, control rod scram time testing, and control rod pattern adjustments. In each instance, Unit 2 was returned to rated thermal power on the same day or the following day and remained at or near rated thermal power for the remainder of the inspection period.
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public websit e a t http://www.nrc.gov/reading -
rm /doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirement s most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light -Water Reactor Inspection Program - Operations Phase.
The inspectors performed activities described in IMC 2515, Appendix D, Plant Status, conducted routine reviews using IP 71152, Problem Identification and Resolution, observed risk significant activities, and completed on-site portions of IPs.
The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards
REACTOR SAFETY
71111.01 - Adverse Weather Protection
Seasonal Extreme Weather Sample (IP Section 03.01) (1 Sample)
- (1) The inspectors evaluated readiness for seasonal extreme weather conditions prior to the onset of seasonal cold temperatures for the Unit 1 emergency diesel generators and the Unit 2 Division I and II diesel generators.
Impending Severe Weather Sample (IP Section 03.02) (1 Sample)
- (1) The inspectors evaluated the adequacy of the overall preparations to protect risk -
significant systems from impending severe weather for high winds on Decem ber 6, 2021.
71111.04 - Equipm ent Alignment
Partial Walkdown Sample (IP Section 03.01) (4 Samples)
The inspectors evaluated system configurations during partial walkdowns of the following systems/trains:
- (1) Unit 2 control rod drive on November 23, 2021
- (2) Unit 2 Division II residual heat removal on November 23, 2021
- (3) Unit 2 Division II diesel generator on November 29, 2021
- (4) Unit 2 reactor building closed loop cooling water on November 29, 2021
Complete Walkdown Sample (IP Section 03.02) (1 Sample)
- (1) The inspectors evaluated system configurations during a complete walkdown of the Unit 2 standby gas treatment system on November 4, 2021.
71111.05 - Fire Protection
Fire Area Walkdown and Inspection Sample (IP Section 03.01) (8 Sam ples)
The inspectors evaluated the implementation of the fire protection program by conducting a walkdown and performing a review to verify program compliance, equipment functionality, material condition, and operational readiness of the following fire areas:
- (1) Unit 2 reactor building 175' and 240', reactor core isolation cooling pump room, fire area 2, on October 5, 2021
- (2) Unit 2 reactor building 261', standby gas treatment rooms A and B, fire areas 4, 73, and 74, on November 4, 2021
- (3) Unit 2 Division I switchgear room, fire area 17, on November 22, 2021
- (4) Unit 2 Division II diesel generator control room, fire area 29, on November 23, 2021
- (5) Unit 2 reactor building 328, fire areas 34 and 35, on November 30, 2021
- (6) Unit 2 Division III diesel generator control room 261/274', fire area 30, on Decem ber 13, 2021
- (7) Unit 2 Division I diesel generator control room 261/274', fire area 28, on Decem ber 13, 2021
- (8) Unit 2 control building 288' relay room, fire area 24, on December 14, 2021
71111.06 - Flood Protection Measures
Inspection Activities - Internal Flooding (IP Section 03.01) (1 Sample)
The inspectors evaluated internal flooding mitigation protections in the:
- (1) Unit 2 Division I and II emergency switchgear rooms, on November 22, 2021
71111.11A - Licensed Operator Requalification Program and Licensed Operator Performance
Requalification Examination Results (IP Section 03.03) (2 Samples)
- (1) The inspectors reviewed and evaluated the licensed o perator examination failure rates for the requalification annual operating exam administered October - Novem ber 2021.
- (2) The inspectors reviewed and evaluated the Unit 1 licensed operator annual requalification results for the annual operating exam on December 6, 2021.
71111.11B - Licensed Operator Requalification Program and Licensed Operator Performance
Licensed Operator Requalification Program (IP Section 03.04) (1 Sample)
- (1) Biennial Requalification Written Examinations
The inspectors evaluated the quality of the licensed operator biennial requalification written examination administered October - Novem ber 2021.
Annual Requalification Operating Tests
The inspectors evaluated the adequacy of the facility licensees annual re qualification operating test.
Administration of an Annual Requalification Operating Test
The inspectors evaluated the effectiveness of the facility licensee in administering requalification operating tests required by 10 CFR 55.59(a)(2) and that the facility licensee is effectively evaluating their licensed operators for mastery of training objectives.
Requalification Examination Security
The inspectors evaluated the ability of the facility licensee to safeguard examination material, such that the examination is not compromised.
Remedial Training and Re-examinations
The inspectors evaluated the effectiveness of remedial training conducted by the licensee, and reviewed the adequacy of re-examinations for licensed operators who did not pass a required requalification examination.
Operator License Conditions
The inspectors evaluated the licensees program for ensuring that licensed operators meet the conditions of their licenses.
Control Room Simulator
The inspectors evaluated the adequacy of the facility licensees control room simulator in modeling the actual plant, and for meeting the requirements contained in 10 CFR 55.46.
Problem Identification and Resolution
The inspectors evaluated the licensees ability to identify and resolve problems associated with licensed operator performance.
71111.11Q - Licensed Operator Requalification Program and Licensed Operator Performance
Licensed Operator Performance in the Actual Plant/Main Control Room (IP Section 03.01) (2 Samples)
- (1) The inspectors observed Unit 1 operations personnel during a planned reactor downpower to approximately 80 percent for a rod pattern adjustment and control rod scram time testing on October 1, 2021.
- (2) The inspectors observed Unit 2 operations personnel during a pla nned reactor downpower to approximately 97 percent and control rod friction testing on October 8, 2021.
Licensed Operator Requalification Training/Examinations (IP Section 03.02) (2 Samples)
- (1) The inspectors observed a Unit 2 simulator evaluation that included the loss of an emergency switchgear bus, a trip of a feedwater heater drain pump, a failure to SCRAM, and a loss of coolant accident inside containment on October 12, 2021.
- (2) The inspectors observed a Unit 1 annual examination t hat included an unplanned opening of an electro-matic relief valve, the 12 emergency condenser tube leak, an instrument air leak, and a steam leak inside containment on October 26, 2021.
71111.12 - Maintenance Effectiveness
Maintenance Effectiveness (IP Section 03.01) (3 S am p les)
The inspectors evaluated the effectiveness of maintenance to ensure the following structures, systems, and components (SSCs) remain capable of performing their intended function:
- (1) Unit 1 safety-related ventilation control room chiller 11B oil leak
- (2) Unit 2 automatic depressurization system makeup valve stuck open
- (3) Unit 2 service water discharge valve failure to open
71111.13 - Maintenance Risk Assessments and Emergent Work Control
Risk Assessment and Management Sample (IP Section 03.01) (4 Sam ples)
The inspectors evaluated the accuracy and completeness of risk assessments for the following planned and emergent work activities to ensure configuration changes and appropriate work controls were addressed:
- (1) Unit 2 elevated risk during scheduled high pressure core spray maintenance on October 26, 2021
- (2) Unit 1 elevated risk during scheduled 12 feedwater pump maintenance on Novem ber 3, 2021
- (3) Unit 2 elevated risk during unplanned rea ctor core isolation cooling maintenance on Novem ber 9, 2021
- (4) Unit 2 elevated risk during scheduled Division I emergency diesel generator 6 -year preventive maintenance on November 17, 2021
71111.15 - Operability Determinations and Functionality Assessments
Operability Determination or Functionality Assessment (IP Section 03.01) (5 Samples)
The inspectors evaluated the licensee's justifications and actions associated with the following operability determinations and functionality assessments:
- (1) Unit 1 emergency diesel generator 102 due to inaccurate day tank level indication on October 5, 2021
- (2) Unit 1 11 and 12 emergency cooling systems following isolation during the loss of reactor protection system 11 on October 15, 2021
- (3) Unit 2 Division III diesel generator due to potential fuel oil lubricity out of specification on October 18, 2021
- (4) Unit 2 Division II diesel generator following jacket water leak on November 3, 2021
- (5) Unit 2 reactor core isolation cooling due to flow controller failure on Novem ber 8, 2021
71111.18 - Plant Modifications
Temporary Modifications and/or Permanent Modifications (IP Section 03.01 and/or 03.02) (1 Sample)
The inspectors evaluated the following temporary or permanent modifications:
- (1) Perm anent Modification: ECP-21-000437, Unit 2 Division II Emergency Diesel Generator Governor Booster Modification
71111.19 - Post-Maintenance Testing
Post-Maintenance Test Sample (IP Section 03.01) (4 Sam ples)
The inspectors evaluated the following post -maintenance test activities to verify system operability and functionality:
- (1) Unit 2 Division II diesel generator governor replacement on November 6, 2021
- (2) Unit 2 Division I control building chiller maintenance on Decem ber 2, 2021
- (3) Unit 1 121 core spray pump and booster pump breaker replacements on Decem ber 2, 2021
- (4) Unit 2 reactor core isolation cooling following flow controller maintenance on November 10, 2021
71111.22 - Surveillance Testing
The inspectors evaluated the following surveillance tests:
Surveillance Tests (other) (IP Section 03.01)
- (1) N1-ST-Q8A, Liquid Poison 11 and Check Valve Operability Test, on October 18, 2021
- (2) N2-ISP-CSL-Q001, Low Pressure Core Spray Pump Discharge Flow - Low (Bypass)
Instrument Channel Functional Test, on December 21, 2021
RCS Leakage Detection Testing (IP Section 03.01) (1 Sample)
- (1) N2-OSP-LOG-S001, Shift Checks - Mode 1, on November 29, 2021
71114.06 - Drill Evaluation
Select Emergency Preparedness Drills and/or Training for Observation (IP Section 03.01) (1 Sample)
- (1) The inspectors evaluated a Unit 2 simulator evaluation which included a failure to SCRAM, a loss of coolant accident, and the declaration of an Alert on October 12,
OTHER ACTIVITIES - BASELINE
===71151 - Performance Indicator Verification
The inspectors verified licensee performance indicators submittals listed below:
MS08: Heat Removal Systems (IP Section 02.07)===
- (1) Unit 1 - October 1, 2020 through September 30, 2021
- (2) Unit 2 - October 1, 2020 through September 30, 2021
MS09: Residual Heat Removal Systems (IP Section 02.08) (2 S am p les)
- (1) Unit 1 - October 1, 2020 through September 30, 2021
- (2) Unit 2 - October 1, 2020 through September 30, 2021
MS10: Cooling Water Support Systems (IP Section 02.09) (2 Samples)
- (1) Unit 2 - October 1, 2020 through September 30, 2021
- (2) Unit 1 - October 1, 2020 through September 30, 2021
71152 - Problem Identification and Resolution (PI&R)
Semiannual Trend Review (IP Section 02.02) (1 Sample)
- (1) The inspectors reviewed the licensees corrective action program for potential adverse trends that might be indicative of a more significant safety issue.
Annual Follow -up of Selected Issues (IP Section 02.03) (2 Samples)
The inspectors reviewed the licensees implementation of its corrective action program related to the following issues:
- (1) IR 04448532 - Review of Uninterruptible Power Supply Fa ilure
- (2) IR 04372127 - Review of Trend in Surveillance Test Interval Implementation
71153 - Follow Up of Events and Notices of Enforcement Discretion
Event Report (IP Section 03.02) (2 Sam ples)
The inspectors evaluated the following licensee event reports (LERs):
- (1) LER 05000220/2021-001-00, Average Power Range Monitors Declared Inoperable Due to Trip of Reactor Recirculation Pump 13.
(ADAMS Accession No. ML21144A050). The inspectors determined that it was not reasonable to foresee or correct the cause discussed in the LER, therefore no performance deficiency was identified. The inspectors did not identify a violation of NRC requirements.
This LER is closed.
- (2) LER 05000220/2021-002-00, Isolation of Both Emergency Condensers Due to Loss of UPS-162A. (ADAMS Accession No. ML21327A024).
The inspection conclusions associated with this LER are documented in this report under Inspection Results.
Th is LER is closed.
INSPECTION RESULTS
Failure to Correct a Condition Adverse to Quality with an Uninterruptible Power Supply Cornerstone Significance Cross-Cutting Report Aspect Section Mitigating Green None (NPP) 71152 Systems NCV 05000220,05000410/2021004 - 01 Open/Closed The inspectors documented a self -revealed Green non -cited violation (NCV) of Title 10 of the Code of Federal Regulations (10 CFR) Part 50, Appendix B, Criterion XVI, Corrective Action, when Exelon did not ensure that a condition adverse to quality (CAQ) associated with an identified deviation from the normal operating frequency of an uninterruptible power supply (UPS) was promptly identified and corrected.
Specifically, on August 1, 2016, after performing maintenance on and placing the UPS -162A in service, Exelon documented an off -normal condition related to the UPS inverter frequency being lower than normal and oscillating.
Exelon closed this issue report (IR) to trend without properly evaluating and correcting the condition, resulting in the failure of the reactor protection system (RPS) UPS -162A and causing the plant to experience a half scram condition, an isolation of both emergency condensers, and an isolation of the reactor water cleanup (RWCU) system on September 25, 2021.
Description:
In 1993, motor generator sets at Unit 1 were replaced with four safety -related UPSs (UPS-162A and - B, UPS-172A and - B) to supply conditioned 120 VAC power to the RPS and instrument buses.
Preventive maintenance is performed on the Unit 1 internal UPS circuit cards by replacing them every 10 years.
On September 25, 2021, UPS-162A experienced voltage and frequency perturbations
(-5 VAC and 0.2 Hz from the normal values) which resulted in actuation of the RPS bus 11 under frequency protective relaying.
This actuation c aused a loss and subsequent restoration of RPS bus 11 on UPS -162A which resulted in a half scram, an isolation of both emergency condensers, and RWCU isolation.
After troubleshooting the equipment and consulting with the vendor, Exelon determined a capacitor, as well as an oscillator and sync circuit card, required replacement.
Exelon replaced these components, sent them for failure analysis, and restored the UPS to service.
On October 28, 2021, Exelon completed a corrective action program evaluation (CAP E)under IR 04448532.
This evaluation ruled out the capacitor as a cause of the UPS failure.
Exelon determined that the cause of the failure was a premature failure of the UPS -162A oscillator and sync circuit card.
The evaluation found that the circuit card issue had potentially existed since the UPS circuit cards were last replaced and placed in service in July 2016.
Exelons CAPE documented that two days after placing the UPS in-service, IR 02699078 was written for observed UPS -162A frequency oscillations of 0.2 Hz and a lower than expected voltage. This IR was closed with no additional actions taken to troubleshoot or correct the abnormal condition.
Exelons CAPE determined that the organization failed to adequately address the documented frequency oscillations and generate a work order to investigate and repair the condition.
Exelon also determined that procedure N1-EPM-UPS-003, UPS Maintenance PM, was inadequate to address frequency oscillations and that system internal and external operating experienc e (OPEX) reviews were inadequate.
Exelon received the vendor failure analysis report (FAR 21- 005) on November 15, 2021.
This report concluded, in part, that the failed oscillator and sync boards caused the inverter to phase drift and the UPS to lose sync within 15 minutes when temperatures exceeded 105 degrees Fahrenheit, as opposed to the normal temperature rating of 120 degrees Fahrenheit.
The inspectors reviewed Exelons CAPE, failure analysis report, and procedures.
Exelons IR 02699078, which documented the observed frequency oscillations in 2016, also documented Recommended Actions to determine if oscillations can lead to a fault in the UPS (could lead to a loss of RPS 11).
The inspectors found no documented answer to the recommended action above. The Functional Basis section of the IR states, The fluctuation (0.25 Hz) is within the acceptance criteria for N1-EPM-UPS-003 (59.7-60.3 Hz). A walkdown was performed by EMD [electrical maintenance] and they did not identify an immediate concern with functionality.
UP S-162A remains functional.
However, Exelons CAPE concluded that procedure N1-EPM-UPS-003 was inappropriately used to determine the functionality of UPS-162A, specifically that frequency oscillations, whether in the acceptance criteria band or not, are an abnormal condition and should not be accepted under any circumstance, as this is a sign of degradation and potential premature failure.
The inspectors also noted that Exelon procedure PI -AA-120, "Issue Identification and Screening Process," Revision 11, Section 4.4.6, states, in part, to determine if the as -found operability and functionality of any system, structure, or component is affected by the condition and document the bas is of the determination.
Section 4.6.6 states, in part, to determine if any additional actions are necessary and route the issue to the responsible department to create actions.
Based on the information above, the inspectors determined that Exelon did not ensure that a CAQ associated with an identified deviation from the normal operating frequency of UPS-162A was promptly identified and corrected.
Specifically, after performing maintenance and placing the UPS in service, Exelon documented an off -normal condition related to the UPS inverter frequency being lower than normal and oscillating yet closed this IR without properly evaluating and correcting the condition.
Corrective Actions: Exelon replaced the failed UPS oscillator and sync circuit cards, revis ed UPS maintenance procedures, added frequency oscillation observation to the system walkdown checklist, sent the failed circuit cards out for failure analysis, and generated a crew learning for timely OPEX reviews.
Corrective Action Reference s: IR 04446321
Performance Assessment:
Performance Deficiency: Exelon's failure to ensure that a CAQ associated with an identified deviation from the normal operating frequency of UPS -162A was promptly identified and corrected is a performance deficiency that was within Exelons ability to foresee and correct, and which should have been prevented.
Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Design Control attribute of the Mitigating Sy st em s cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, Exelon not ensuring that a CAQ associ ated with a safety -related UPS was promptly identified and corrected resulted in reduced UPS reliability, a half -scram condition, an isolation of both emergency condensers, and an isolation of the RWCU system.
Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At -Power.
Using Exhibit 2 of Appendix A, the inspectors determined this finding was of very low safety significance (Green) because Yes was not answered to any of the screening questions.
Cross-Cutting Aspect: Not Present Performance. No cross -cutting aspect was assigned to this finding because the inspectors determined the finding did not reflect present licensee performance.
Enforcem ent:
Violation: 10 CFR Pa rt 50, Appendix B, Criterion XVI, Corrective Action, requires, in part, that measures shall be established to ensure that CAQs, such as deviations and nonconformances are promptly identified and corrected. Exelons procedure PI -AA-120, "Issue Identification and Screening Process," states, in part, for CAQ to determine if the as -
found operability and functionality of any system, structure, or component is affected by the condition and document the basis of the determination.
The procedure also states to determine if any additional actions are necessary and route the issue to the responsible department to create actions.
Contrary to the above, between August 1, 2016, and September 25, 2021, Exelon did not ensure that a CAQ associated with an identified deviation from the normal operating frequency of UPS -162A was promptly identified and corrected.
Specifically, on August 1, 2016, after performing maintenance and placing the UPS in service, Exelon docum ented an off-normal condition related to the UPS inverter frequency being lower than normal and oscillating but closed this IR without properly evaluating and correcting the condition resulting in the failure of the RPS UPS -162A and caused the plant to experience a half -scram condition, an isolation of both emergency condensers, and an isolation of the RWCU system.
Enforcement Action: This violation is being treated as a n NCV, consistent with Section 2.3.2 of the Enforcement Policy.
Minor Performance Deficiency 71152 Minor Performance Deficiency: The inspectors reviewed a CAPE completed by Exelon staff, and their identified corrective actions, following a series of issues in the implementation of the surveillance frequency control program, including a missed surveillance as a result of an inadequat e evaluation.
The missed surveillance was for the calibration of Unit 1 remote shutdown panel reactor vessel water level indication in accordance with the surveillance frequency control program.
Exelons CAPE included an extent of condition review of all completed surveillance frequency changes to ensure no other technical flaws existed. Corrective actions included ensuring future surveillance frequency changes are routed through regulatory assurance to verify the procedures align with the appropriate technical specification surveillance requirem ents. Exelon also established a checklist to be performed during the development of future surveillance frequency changes that includes the steps required to avoid missed surveillances.
The inspectors reviewed the evaluation, extent of condition review, and relevant IR s and interviewed the surveillance frequency control program owner.
Based on the documents reviewed and discussions with personnel, the inspectors determined Exelons evaluation of the issue was adequate and provided for corrective action commensurate with the safety significance of the problem.
Exelon's failure to perform the missed surveillance discussed above in the required timeframe is a performance deficiency.
Screening: The inspectors determined the performance deficiency was minor.
The inspectors evaluated the missed surveillance for significance in accordance with the guidance in IMC 0612, Appendix B, "Issue Screening," and Appendix E, "Examples of Minor Issues." The performance deficiency w as minor because it did not adversely affect a cornerstone objective and following identification an appropriate risk assessment was developed and the surveillance successfully completed in accordance with T echnical Specification Surveillance Requirement 4.0.3.
Significance: The inspectors assessed the significance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At -Power.
This performance deficiency was of minor significance and, therefore, are not subject to enforcement action in accordance with the NRC's Enforcement Policy.
Minor Performance Deficiency 71152 Minor Performance Deficiency: The inspectors performed a review of a recent UPS failure that occurred at Unit 1 on September 25, 2021.
Exelons CAPE, under IR 04448532, documented that UPS -162A experienced voltage and frequency perturbations ( -5 VAC and 0.2 Hz from the normal values) which resulted in actuation of the RPS bus 11 underfrequency protective relaying. This actuation caused a loss and subsequent restoration of RPS bus 11 on UPS -162A which resulted in a half SCRAM, an isolation of both emergency condensers, and RWCU isolation. The inspectors reviewed this CAPE to assess Exelons ability to adequately identify problems, evaluate the causes of the equipment issues, and resolve the issues in a timely manner.
During the review, the inspectors noted one more than minor perform ance deficiency (PD)which is documented separately as an NCV in this report, and one minor PD, which is documented below.
Minor PD: Exelon procedure PI-AA-115, "Operating Experience Program," Revision 5, Section 4 for the receipt, screening, and processing of OPEX documents states, in part, to perform an initial screening to determine the appropriate OPEX evaluation significance, initiate an IR as applicable in Attachment 1, or if there is potentially new information that challenges the conservatism of current external event design assumptions, and retain relevant OPEX information in the ENGAGE Health notebooks in accordance with procedure ER-AA-2030, "Conduct of Equipment Reliability Manual." Procedure ER-AA-2030, Section 4.3.4 for OPEX and peer lessons learned states, in part, that engineers should read daily industry OPEX reports, review similar station IRs, and perform periodic OPEX searches on industry websites. In addition, the procedure discusses that the ENGAGE Health notebooks is where relevant OPEX should be retained and at a minimum updated semiannually.
Attachm ent 2 of this procedure goes on to state that the engineer is responsible for initial OPEX reviews and the generation of IRs for required Level 3 OPEX reviews.
Contrary to these procedures, the inspectors found that Exelons CAPE determined that, The OPEX reviews documented in Engage Health for this system appear to be inadequate.
More robust OPEX reviews may have allowed the frequency oscillations to be identified as a potential failure mechanism earlier.
This contributed to the failure in this event.
The inspectors noted that the CAPE generated an action to generate a crew learning for timely OPEX reviews. Based on the external OPEX and the lack reviews and retention of the OPEX for the UPS system, the inspectors questioned Exelon about whether a "crew learning" was enough to correct this deficiency.
The inspectors determined that the "crew learning" was an action that could correct the OPEX review and retention issues going forward but that the potential OPEX gap could remain unless system engineers conducted a "look back" to ensure previous OPEX was accurately captured.
Exelon documented this in IR 04448532.
Significance: The inspectors assessed the si gnificance of the finding using Appendix A, The Significance Determination Process (SDP) for Findings At -Power.
Observation: Sem i-Annual Trend 71152 The inspectors evaluated a sample of issues and events that occurred from July 2021 through December 2021 to determine whether issues were appropriately considered as emerging or adverse trends. The inspectors verified issues were appropriately evaluated by Exelon staff for potential trends and addressed within the scope of the corrective action program or through department review.
The inspectors did not identify any new trends that could indicate a more significant safety issue. Exelon documented a trend in critical component failures during this evaluation period. This included the failure of the Unit 1 12 high pressure coolant injection controller, loss of RPS 11 following the failure of UP S -162A, the Unit 2 'C' service water pump discharge valve failure to open, and Division II emergency diesel generator jacket water leak.
These issues were captured in trend IR 04459468 which resulted in Exelon personnel performing a work group evaluation for commonalities between the failures.
Exelon also identified several instances of rework associated with supplemental workforce over a short period of tim e and documented a potential trend in IR 04439810. None of these cases resulted in inoperability or unavailability of safety systems. The inspectors also noted other minor potential adverse trends identified by Exelon staff in the areas of equipment challenges in both units' instrument air systems and failures of fire detectors at both units requiring additional walkdowns and patrols.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
- On January 20, 2022, the inspector s presented the integrated inspection results to Mr. Adam Schuerman, Plant Manager, and other members of the licensee staff.
THIRD PARTY REVIEWS
Inspectors reviewed the World Association of Nuclear Operators report that was issued during the inspection period.
DOCUMENTS REVIEWED
Inspection Type Designation Description or Title Revision or
Procedure Date
71111.01 Corrective Action 04198397
Docum ents 04445508
04461612
Procedures N1-OP-64 Meteorological Monitoring 02100
N2-OP-102 Meteorological Monitoring 02800
OP-AA-108-111 Severe Weather and Natural Disaster Guidelines 023
Work Orders C93760254
C93762635
C93805503
71111.04 Drawings PID-11D Piping and Instrum entation Diagram Service Water System 12
PID-13A Piping and Instrumentation Diagram Reactor Building 19
Closed Loop Cooling Water
PID-30A Piping and Instrumentation Diagram Control Rod Drive 29
Hydraulic System
PID-30B Piping and Instrumentation Diagram Control Rod Drive 14
Hydraulic System
PID-30C Piping and Instrumentation Diagram Control Rod Drive 15
Hydraulic System
PID-31A Piping and Instrum entation Diagram Residual Heat 27
Removal System
PID-43G Piping and Instrum entation Diagram Fire Protection - 28
Water
PID-61B Piping and Instrum entation Diagram Primary Containm ent 23
Purge and Standby Gas Treatment
PID-61C Piping and Instrum entation Diagram Primary Containm ent 8
Purge and Standby Gas Treatment
Miscellaneous Green NCV Failure to Ensure Proper Control of the Standby Gas
05000410/2018002-Treatment System Damper Valve, 2GTS -V2000, Within
Procedures, Materials, and Design Controls
Procedures N2-OP-13 LINEUPS, Reactor Building Closed Loop Cooling System 004
N2-OP-30 Control Rod Drive 26
N2-OP-31 Residual Heat Removal System 036T1
Inspection Type Designation Description or Title Revision or
Procedure Date
N2-OP-61B Standby Gas Treatment 17
N2-OP-61B-Standby Gas Treatment - LINEUPS 0100
LINEUPS
PID-31B Piping and Instrum entation Diagram Residual Heat 23
Removal System
PID-31C Piping and Instrum entation Diagram Residual Heat 17
Removal System
PID-31E Piping and Instrum entation Diagram Residual Heat 21
Removal System
71111.05 Drawings PID - 43G Charcoal Bed Deluge System 28
Fire Plans N2-FPI-PFP-0201 Station Fire Protection Instruction 06
N2-PFP-CB261 Control Building, El 261/274 0
N2-PFP-CB288 Control Building, El 288 0
N2-PFP-RX-328 Reactor Building, El 328 0
71111.06 Procedures N2-PRA-012 Nine Mile Point Unit 2 Probabilistic Risk Assessment 3
Internal Flooding Analysis Notebook
71111.11Q Miscellaneous Reactivity October 2021 Quarterly Load Drop 0
Maneuver Plan
NM1C25-3
Reactivity October 2021 Sequence Exchange, Scram Time Testing, 0
Maneuver Plan and Friction Testing
NM2C18-9
Procedures N1-ST-R1 Control Rod Scram Insertion Time Test 02600
N2-OSP-RDS-Channel Interference Testing 00600
@002
71111.12 Corrective Action 01986288
Docum ents 02005727
2468122
04389154
04412983
04413060
04446429
Corrective Action 04464682
Docum ents
Inspection Type Designation Description or Title Revision or
Procedure Date
Resulting from
Inspection
Miscellaneous (a)(1) Determination Approved
for Unit 2 Service 12/3/21
Water
1-CRAC-F02 Maintenance Rule System Basis Document
Unit 2 Maintenance Automatic Depressurization
Rule System Basis
Docum ent
Unit 2 Maintenance Containment Inerting/N2
Rule System Basis
Docum ent
Unit 2 Maintenance Instrument Air
Rule System Basis
Docum ent
Unit 2 Maintenance Service Water
Rule System Basis
Docum ent
Procedures N1-MMP-210-001 Control Room Chiller Maintenance
Work Orders C91883376
C93774298
C93785657
C93785742
C93786175
71111.13 Procedures OP-AA-108-117 Protected System Verification IAW 7
OP-NM-108-117 Protected Equipm ent Program at Nine Mile Point 00500
WC-AA-101-1004 On-Line Maintenance For Limiting Condition For Operation 009
of System s of Com ponents
71111.15 Corrective Action 02715754
Docum ents 04448532
04449792
04449971
04449971
04453026
Inspection Type Designation Description or Title Revision or
Procedure Date
04458203
04459510
Drawings C-18026-C Emergency Diesel Generator #102 Starting Air, Cooling 28
Water Lube Oil and Fuel
C-19410-C Fold Out 10/10A - Elementary Wiring Diagram Diesel 47
Generators #102 and #103 Control Ckts
C-19589-C Elementary Wiring Diagram Reactor Protection System 51
(Emergency Cooling Channel #11)
C-19845-C Elem entary Wiring Diagram DC Valve Board 12 26
Procedures N1-ST-M4A Emergency Diesel Generator 102 and PB 102 Operability 03000
Test
N2-OSP-I CS-RCIC Pump and Valve Operability Test and System 01600
Q@002 Integrity Test and ASME XI Functional Test and Analysis
Work Orders C93808212
71111.18 Drawings 0001040209048 Control Diagram Shutdown System 13
PID-104A Piping and Instrum entation Diagram Standby Diesel Gen 34
System
Engineering ECP-21-000437 EDG Governor Booster 0000
Changes
Work Orders C93809772
71111.19 Corrective Action 04380928
Docum ents 044595101
04461978
Procedures MA-AA-716-012 Post Maintenance Testing 26
N1-EPM-GEN-150 4.16kV Breaker Inspection PM 01800
N1-ST-Q1B CS 121 Pump, Valve and SDC Water Seal Check Valve 02300
Operability Test
N2-MSP-HVK-Control Building Chiller Performance Test 00000
2Y001
N2-OSP-EGS-Diesel Generator and Diesel Air Start Valve Operability 024T1
M@001 Test - Division I and II
N2-OSP-I CS-RCIC Pump and Valve Operability Test and System 01600
Q@002 Integrity Test and ASME XI Functional Test and Analysis
Work Orders C93658634
Inspection Type Designation Description or Title Revision or
Procedure Date
C93658635
C93811429
C93812090
C93812875
71111.22 Drawings F-63019-C Reactor Liquid Poison System ASME Section XI Boundary 17
Diagram
Procedures N1-ST-Q4A Liquid Poison Pump 11 and Check Valve Operability Test 01800
N2-ISP-CSL-Q001 LPCS Pump Discharge Flow - Low (Bypass) Instrument 00700
Channel Functional Test
N2-OSP-LOG-S001 Shift Checks - Mode 1 03800
Work Orders C93771647
71151 Corrective Action 04444544
Docum ents
Miscellaneous N1-MSPI-001 Nine Mile Point 1 MSPI Basis Document 11
N2-MSPI-001 Nine Mile Point 2 MSPI Basis Document 16
NEI 99-02 Regulatory Assessment Performance Indicator Guideline 7
71152 Corrective Action 04289904
Docum ents 04368005
04371766
04372127
04433486
04439810
04441194
04442220
04446429
04448532
04458203
04459468
4372131
Procedures ER-AA-425 Implementation of Technical Specification Surveillance 004
Frequency Control Program
N1-ISP-036-003 HI/LO Reactor Water Level Instrument Trip Channel 01400
Test/Calibration
19