05000440/LER-2009-003, Regarding Completion of Technical Specifications Required Shutdown Due to Division 2 Emergency Service Water (ESW) Inoperability
| ML093580023 | |
| Person / Time | |
|---|---|
| Site: | Perry  |
| Issue date: | 12/14/2009 |
| From: | Bezilla M FirstEnergy Nuclear Operating Co |
| To: | Document Control Desk, Office of Nuclear Reactor Regulation |
| References | |
| FOIA/PA-2010-0209, L-09-331 LER 09-003-00 | |
| Download: ML093580023 (10) | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i) 10 CFR 50.73(a)(2)(vii), Common Cause Inoperability 10 CFR 50.73(a)(2)(ii) 10 CFR 50.73(a)(2)(viii)(A) 10 CFR 50.73(a)(2)(ii)(B), Unanalyzed Condition 10 CFR 50.73(a)(2)(viii)(B) 10 CFR 50.73(a)(2)(iii) 10 CFR 50.73(a)(2)(ix)(A) 10 CFR 50.73(a)(2)(iv)(A), System Actuation 10 CFR 50.73(a)(2)(x) 10 CFR 50.73(a)(2)(v)(A), Loss of Safety Function - Shutdown the Reactor 10 CFR 50.73(a)(2)(v)(B), Loss of Safety Function - Remove Residual Heat 10 CFR 50.73(a)(2)(i)(A), Completion of TS Shutdown 10 CFR 50.73(a)(2)(v), Loss of Safety Function 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 4402009003R00 - NRC Website | |
text
FENOC FirstEnergy Nuclear Operating Company Perry Nuclear Power Station 10 Center Road Perry, Ohio 44081 Mark B. Bezilla Vice President 440-280-5382 Fax-440-280-8029 December 14, 2009 L-09-331 10 CFR 50.73 ATTN: Document Control Desk U.S. Nuclear Regulatory Commission Washington, DC 20555-0001
SUBJECT:
Perry Nuclear Power Plant Docket No. 50-440, License No. NPF-58 Licensee Event Report Submittals Enclosed are Licensee Event Report (LER) 2009-003, "Completion of Technical Specifications Required Shutdown due to Division 2 Emergency Service Water (ESW)
Inoperability," and LER 2009-004, "Reactor Recirculation Pump Failure Results in Manual Reactor Protection System Actuation." There are no regulatory commitments contained in this submittal.
If there are any questions or if additional information is required, please contact Mr. Robert Coad, Manager - Regulatory Compliance, at (440) 280-5328.
Sincerely, Mark B. Bezilla
Enclosures:
LER 2009-003 LER 2009-004 cc:
NRR Project Manager NRC Resident Inspector NRC Region III
NRC FORM 366 U.S. NUCLEAR REGULATORY COMMISSION APPROVED BY OMB NO. 3150-0104 EXPIRES 8/31/2010 (9-2007)
, the NRC may (See reverse for required number of not conduct or sponsor, and a person is not required to respond to, the digits/characters for each block) information collection.
- 3. PAGE Perry Nuclear Power Plant 05000440 1 OF 5
- 4. TITLE Completion of Technical Specifications Required Shutdown due to Division 2 Emergency Service Water (ESW) Inoperability
- 5. EVENT DATE
- 6. LER NUMBER
- 7. REPORT DATE
- 8. OTHER FACILITIES INVOLVED SEQUENTIAL REV FACILITY NAME DOCKET NUMBER MONTH DAY YEAR YEAR NUMBER NO.
MONTH DAY YEAR 2009 003 00 12 14 200 FACILITY NAME DOCKET NUMBER
- 9. OPERATING MODE
- 11. THIS REPORT IS SUBMITTED PURSUANT TO THE REQUIREMENTS OF 10 CFR §: (Check all that apply)
El 20.2201(b)
E] '20.2203(a)(3)(i)
[E 50.73(a)(2)(i)(C)
E] 50.73(a)(2)(vii) 1
[] 20.2201(d)
E] 20.2203(a)(3)(ii)
El 50.73(a)(2)(ii)(a)
[] 50.73(a)(2)(viii)(A)
[E 20.2203(a)(1)
E] 20.2203(a)(4)
El 50.73(a)(2)(ii)(B)
El 50.73(a)(2)(viii)(B)
El 20.2203(a)(2)(i)
E] 50.36(c)(1)(i)(A)
E] 50.73(a)(2)(iii)
El 50.73(a)(2)(ix)(A)
- 10. POWER LEVEL
[] 20.2203(a)(2)(ii)
E] 50.36(c)(1)(ii)(A)
E] 50.73(a)(2)(iv)(A)
El 50.73(a)(2)(x)
E] 20.2203(a)(2)(iii)
E] 50.36(c)(2)
E] 50.73(a)(2)(v)(A)
El 73.71 (a)(4) 100 El 20.2203(a)(2)(iv)
El 50.46(a)(3)(ii)
El 50.73(a)(2)(v)(B)
El 73.71 (a)(5)
El 20.2203(a)(2)(v)
[] 50.73(a)(2)(i)(A)
El 50.73(a)(2)(v)(C)
El OTHER Specify in Abstract below
_ 20.2203(a)(2)(vi)
Z 50.73(a)(2)(i)(B)
El 50.73(a)(2)(v)(D) or in As a preventive measure, the ESW pumps were tested to verify that there was no consequential damage as a result of the wetting. On October 15, 2009, the Division 2 ESW pump B cabling and motor were tested to verify functionality beginning at approximately 0230 hours0.00266 days <br />0.0639 hours <br />3.80291e-4 weeks <br />8.7515e-5 months <br />. Following these tests, at 0601 hours0.00696 days <br />0.167 hours <br />9.937169e-4 weeks <br />2.286805e-4 months <br /> the Division 2 ESW pump B was started for a planned maintenance run. At 0718 hours0.00831 days <br />0.199 hours <br />0.00119 weeks <br />2.73199e-4 months <br />, the ESW pump B tripped for unknown reasons. A decision was made to commence a controlled plant shutdown due to the anticipated investigation and expected repair time exceeding the applicable TS Limiting Condition for Operation completion time. At approximately 1225 hours0.0142 days <br />0.34 hours <br />0.00203 weeks <br />4.661125e-4 months <br />, with the plant operating in Mode 1 at 100 percent RTP, a planned shutdown was commenced due to Division 2 ESW System inoperability.
At 2241 hours0.0259 days <br />0.623 hours <br />0.00371 weeks <br />8.527005e-4 months <br />, during the shutdown evolution, reactor recirculation pump A failed to transfer to slow speed during pump downshift and subsequently tripped. On October 16, 2009, at 0048 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br />, with the reactor operating at approximately 30 percent RTP, shutdown of the plant was completed by manual actuation of the RPS when the Mode switch was placed in Shutdown in accordance with plant procedures (reference LER 2009-004, "Reactor Recirculation Pump Failure Results in Manual Reactor Protection System Actuation").
The plant remained in Mode 3 until 0429 hours0.00497 days <br />0.119 hours <br />7.093254e-4 weeks <br />1.632345e-4 months <br /> on October 19, 2009, when the plant entered Mode 4.
The plant was brought into compliance with TS 3.4.10 Action A.1 at 0045 on October 21, 2009, when an alternate method of decay heat removal was established. Following repairs and testing, the Division 2 ESW pump B was restored to service at 0515 and declared operable at 1358 hours0.0157 days <br />0.377 hours <br />0.00225 weeks <br />5.16719e-4 months <br /> on October 28, 2009.
CAUSE OF EVENT
Based on cable failure analysis, the cause of the Division 2 ESW pump B trip was determined to be a preexisting manufacturing defect in the power supply cable. The location of the failure confirmed that the failure was not related to the wetting of plant equipment in the ESW pumphouse. The location of the cable failure was approximately 1100 feet from the ESW pumphouse.
The cable that failed was a 5kV rated safety-related Anaconda Uniblend with black Ethylene Propylene Rubber insulated shielded cable [CBL5]. A contributing cause to the cable failure is the effects of the predictive maintenance testing (Direct Current meggedPolarization Index) methodology.
Three tests were conducted within a 30 minute timeframe during troubleshooting. This, coupled with the AC current on motor start, accelerated the degraded cable failure.
Another potential root cause was identified as "water treeing". Water treeing is described as a series of tiny hollow channels that can develop in insulation exposed to water and electrical stress. This was determined based on known submergence of the cable either between manhole dewatering periods or from groundwater intrusion into the conduit. The failure analysis results do not indicate water intrusion affected overall cable insulation integrity. However, industry documents indicate that a wet condition can allow a defect to proceed to failure faster than if it had been dry. It is indeterminate whether this damage mechanism would have caused a failure by itself without a preexisting cable insulation defect.
EVENT ANALYSIS
The plant's ESW System is comprised of three independent subsystem loops A, B, and C. These open subsystem loops take suction from and return to Lake Erie. Each loop contains a separate pump, which is operated from a preferred power source or a standby diesel generator (DG) [EK].
The B loop supplies cooling water to Division 2 residual heat removal (RHR) [BO], standby DG, and emergency closed cooling water heat exchangers, and a fuel pool cooling heat exchanger. The ESW loops run intermittently for plant evolutions such as waste discharges, chemical treatments, ESW testing, and testing of supported systems. The ESW loops only normally run for extended periods to support plant operations such as shutdown cooling during outages and when necessary to support operation of control complex chillers and ventilation trains.
The following Systems were considered inoperable as a result of the failure of ESW pump B and consequently the Division 2 ESW System inoperability:
" Division 2 Emergency Diesel Generator
- Residual Heat Removal B in the low pressure coolant injection, suppression pool cooling, shutdown cooling and containment spray modes of operation
" Low Pressure Coolant Injection C
" Control Room HVAC Train B
- Hydrogen Analyzer B
" Emergency Closed Cooling B
" Combustible Gas Mixing System B.
Mode 4 shutdown cooling TS require two available methods of decay heat removal. With the loss of the Division 2 ESW capability, RHR B was unavailable for decay heat removal, leaving only one credited decay heat removal system, using RHR A in the shutdown cooling mode. As a result, the plant operators used the full timeframe authorized by TS prior to entering Mode 4 to utilize the decay heat removal capabilities of the Main Condenser for as long as practicable.
The plant operating crews were trained on this method of decay heat removal prior to the shutdown.
The plant remained in this condition until the Main Condenser was verified to be a valid alternative for decay heat removal capabilities while remaining in cold shutdown. Per the plant's defense-in-depth philosophy, the main method of decay heat removal capability was posted as protected equipment, including the associated RPS isolation function and all support systems. During this timeframe, the plant was maintained at a heightened awareness state with work activities strictly controlled. No work was performed in the switchyard, and maintenance that could challenge shutdown cooling was limited and direct oversight was provided. Prior to the subsequent reactor startup, extent of condition testing was performed on Division 1 and 3 ESW trains to verify that a common cause condition did not exist in those trains.
A bounding probabilistic risk assessment was performed for the loss of Division 2 ESW System using the average unavailability model. For this event, the probabilistic risk assessment calculated the change in Core Damage Frequency (CDF) to be 8.56E-07. The Large Early Release Frequency (LERF) is on the order of 1 E-08. Events with changes in CDF of less than 1.OE-06 and a LERF of less than 1.OE-07 are not considered to be significant risk events. Based on the probabilistic risk assessment results, this event is considered to be of low safety siqnificance.
CORRECTIVE ACTIONS
The failed cable section was replaced with a Unit 2 installed spare. The good sections of the faulted cable were spliced at two locations, thereby eliminating the underground faulted location. Corrective actions were initiated to replace the Division 1 and 2 ESW motor cables, revise cable testing procedures, implement a modification to install a drain path in elastomer seals installed in safety-related penetrations, generate a preventive maintenance task to replace similar cables at a 20 year frequency, and share lessons learned with engineering and maintenance personnel. Additionally, the lessons learned will be used in testing a Service Water motor cable or a Circulating Water motor cable to determine if water treeing exists, and based on these results, the population of cables will be expanded to test the remaining non safety 5kV and 13kV Anaconda shielded cables in 5kV or 13kV non safety applications.
PREVIOUS SIMILAR EVENTS
A review of Licensee Event Reports and the corrective action program database for the past 3 years did not identify any previous similar events.
COMMITMENTS
There are no regulatory commitments contained in this report. Actions described in this document represent intended or planned actions, are described for the NRC's information, and are not regulatory commitments.
Energy Industry Identification System Codes are identified in the text as [XX].
INTRODUCTION
On October 15, 2009, at approximately 1225 hours0.0142 days <br />0.34 hours <br />0.00203 weeks <br />4.661125e-4 months <br />, a planned shutdown was commenced due to Emergency Service Water (ESW) system [BI] pump [P] B inoperability (reference LER 2009-003, "Completion of Technical Specifications Required Shutdown due to Division 2 Emergency Service Water (ESW) Inoperability"). At 2241 hours0.0259 days <br />0.623 hours <br />0.00371 weeks <br />8.527005e-4 months <br />, reactor recirculation (RRC) [AD] pump [P] A failed to transfer to slow speed during pump downshift and subsequently tripped. On October 16, 2009, at 0048 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br />, with the reactor operating at approximately 30 percent rated thermal power (RTP),
shutdown of the plant was completed by a manual actuation of the Reactor Protection System (RPS)
[JC] when the Mode switch was placed in Shutdown in accordance with plant procedures. A four-hour non-emergency notification in accordance with 10 CFR 50.72(b)(2)(iv)(B) was made at 0148 hours0.00171 days <br />0.0411 hours <br />2.44709e-4 weeks <br />5.6314e-5 months <br /> for actuation of the RPS with the reactor critical and is documented in Event Notification 45440.
The safety significance of this event is considered to be low. This event is being reported in accordance with 10 CFR 50.73(a)(2)(iv)(A) for any event resulting in manual actuation of the RPS.
EVENT DESCRIPTION
On October 15, 2009, at approximately 1225 hours0.0142 days <br />0.34 hours <br />0.00203 weeks <br />4.661125e-4 months <br />, with the plant in Mode 1 (i.e., Power Operation) and the reactor operating at approximately 100 percent RTP, a planned shutdown was commenced with the intent of completing the shutdown by manual reactor scram in accordance with plant procedures and the Evolution Specific Reactivity Plan. The plant was performing a Technical Specifications required shutdown due to the anticipated time necessary to troubleshoot and repair the cause of the ESW system pump B inoperability.
At 2241 hours0.0259 days <br />0.623 hours <br />0.00371 weeks <br />8.527005e-4 months <br />, with the reactor operating at approximately 40 percent RTP, plant operators attempted to downshift RRC pumps A and B from fast to slow speed. Per the plant's system operating instructions, following the low-frequency motor-generators (LFMGs) [MG] start, the transfer sequence was initiated by simultaneously taking both RRC pump breaker controls 5A and 5B to transfer.
Breakers 5A and 5B opened and RRC pump B Breaker 2B closed as expected and pump B continued in slow speed. RRC pump A Breaker 2A never closed to re-energize the A pump in slow speed. After an approximate 43 second time delay, Breaker 1A tripped open to shutdown LFMG A.
No abnormal conditions were noted by visual inspection at the breaker cubicles or the LFMG Auxiliary Relay Panels immediately following the event.
At 2248 hours0.026 days <br />0.624 hours <br />0.00372 weeks <br />8.55364e-4 months <br />, Feedwater Heater 4 isolated which created some balance of plant perturbations. The Evolution Specific Reactivity Plan contained reevaluation criteria for the trip of RRC pump A.
However, due to the current plant conditions and the time required to revise the Evolution Specific Reactivity Plan, preparations were made for a manual reactor scram. On October 16, 2009 at 0048 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br />, a manual actuation of RPS occurred when the Mode switch was taken to Shutdown at approximately 30 percent RTP vice the planned 19.5 -22.5 percent RTP with the generator offline.
Following the RPS actuation, the turbine and generator tripped as expected. All control rods were fully inserted into the core. No automatic Emergency Core Cooling Systems or Reactor Core Isolation Cooling system response was required and level control was maintained using the feedwater system with the main condenser being used to remove decay heat. The plant remained in
Mode 3 until 0429 hours0.00497 days <br />0.119 hours <br />7.093254e-4 weeks <br />1.632345e-4 months <br /> on October 19, 2009, when the plant entered Mode 4. Overall level control systems responded as expected and operator response to the scram was consistent with the training that was conducted prior to the planned plant shutdown.
CAUSE OF EVENT
The cause of the RRC pump A failure to transfer to slow speed was due to the LFMG output breaker failing to close. This was caused by an intermittent connection of the retaining clip on the contactor [CNTR] due to not being properly seated. The retaining clip was not properly installed on the 1B33K0131A [GE CR305] contactor because a skill-based error occurred during a preventive maintenance task that inspected the contactor in April of 2007. As part of this maintenance task, the contactor coil is inspected, which would require the retainer clip to be removed from its normal position. No additional work activities were identified after April 2007.
Therefore, the root cause was determined to be less than adequate use of human performance tools in that self-checking was not effectively applied to ensure the intended actions were correct. This was determined to be a skill-based error that involved less than adequate self-checking, peer-checking, and pre-job briefs of important steps on critical equipment. The preventive maintenance task does not require verification of the retaining clip installation. Additionally, the post maintenance test and 32 additional starts of the LFMG A set had no anomalies.
A contributing cause was determined to be a too narrowly focused operating experience screen which missed a similar industry event that that occurred on February 22, 2006, related to an improper installation of a retaining clip on a similar General Electric Contactor (CR105 versus CR305) within the RPS.
EVENT ANALYSIS
The RRC system provides a forced coolant flow through the core to remove heat from the fuel to allow operation at significantly higher power than would otherwise be possible. The system consists of two recirculation pump loops external to the reactor vessel. A bounding probabilistic risk assessment was performed for the manual scram using the average unavailability model. For this event, the probabilistic risk assessment calculated the change in Core Damage Frequency (CDF) to be 1.69E-07. The Large Early Release Frequency is on the order of 1 E-08. Events with changes in CDF of less than 1.OE-06 and a LERF of less than 1.OE-07 are not considered to be significant risk events. Based on the probabilistic risk assessment results, this event is considered to be of low
safety significance
CORRECTIVE ACTIONS
The 1B33K131A LFMG contactor was replaced and proper assembly of the contactor was verified.
A separate root cause investigation was recently performed to determine the causes of undesirable
- individual and organizational behaviors related to human performance issues. The Human Performance error identified in this event is similar to the errors investigated in the root cause.
Corrective actions were focused on addressing the inconsistent use of human performance tools by individual workers and lack of reinforcement of tool usage by supervisors and management.
A review of these corrective actions was performed and it was concluded that those were adequate to address the human performance aspects of this event; therefore, no additional human performance corrective actions were initiated.
Corrective actions were initiated to revise the Maintenance Plans for these and similar contactors to add an operation to review/discuss the relevant operating experience as part of the pre-job brief and to add a step to perform peer verification that the retaining clip has been properly seated on the detent. Additionally, corrective actions were developed to address the screening weakness of the operating experience with the section operating experience coordinators and to revise the RRC system operating procedure to add a step to confirm the LFMG Voltage Regulator Output voltage after the supply breaker is closed. Maintenance training will be revised to incorporate lessons learned.
PREVIOUS SIMILAR EVENTS
A review of Licensee Event Reports and the Corrective Action Program database for the past three years was completed for conditions written for RRC pump failures to transfer to slow speed and improper installation of equipment.
LER 2007-007, "Reactor Recirculation Pump Failure Results in Manual Reactor Protection System Actuation," describes a condition where the RRC pump B failed to transfer to slow speed and subsequently tripped on June 22, 2007. The cause of the RRC pump B failure to transfer to slow speed was a malfunction of the low-frequency motor-generator set control and interlock circuit Agastat time-delay relay. This was due to the auxiliary contact failing to open as a result of less than adequate programmatic controls for relay checkout and calibration. A contributing cause was less than adequate implementation of testing controls for installing relays.
A review of corrective action program documents over the last three years found only condition reports associated with this event and the events reported under LER 2007-007. The corrective actions taken for this previous event could not reasonably be expected to prevent the occurrence of this event.
COMMITMENTS
There are no regulatory commitments contained in this report. Actions described in this document represent intended or planned actions, are described for the NRC's information, and are not regulatory commitments.
I NIRC FORM 366A (9-2007)
PRINTED ON RECYCLED PAPER