05000346/LER-2009-001, Regarding Containment Air Cooler Fans Inoperable Due to Misapplication of Potter and Brumfield Rotary Relays
| ML093500598 | |
| Person / Time | |
|---|---|
| Site: | Davis Besse  |
| Issue date: | 12/14/2009 |
| From: | Allen B FirstEnergy Nuclear Operating Co |
| To: | Document Control Desk, Office of Nuclear Reactor Regulation |
| References | |
| L-09-310 LER 09-001-00 | |
| Download: ML093500598 (6) | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(i) 10 CFR 50.73(a)(2)(vii), Common Cause Inoperability 10 CFR 50.73(a)(2)(ii)(A), Seriously Degraded 10 CFR 50.73(a)(2)(viii)(A) 10 CFR 50.73(a)(2)(ii)(B), Unanalyzed Condition 10 CFR 50.73(a)(2)(viii)(B) 10 CFR 50.73(a)(2)(iii) 10 CFR 50.73(a)(2)(ix)(A) 10 CFR 50.73(a)(2)(iv)(A), System Actuation 10 CFR 50.73(a)(2)(x) 10 CFR 50.73(a)(2)(v)(A), Loss of Safety Function - Shutdown the Reactor 10 CFR 50.73(a)(2)(v)(B), Loss of Safety Function - Remove Residual Heat 10 CFR 50.73(a)(2)(i)(A), Completion of TS Shutdown 10 CFR 50.73(a)(2)(v), Loss of Safety Function |
| 3462009001R00 - NRC Website | |
text
FENOC FE01O%
5501 North State Route 2 FirstEnergy Nuclear Operating Company Oak Harbor, Ohio 43449 Barry S. Allen 419-321-7676 Vice President - Nuclear Fax: 419-321-7582 December 14, 2009 L-09-31 0 10 CFR 50.73 ATTN: Document Control Desk United States Nuclear Regulatory Commission Washington, D.C. 20555-0001
SUBJECT:
Davis-Besse Nuclear Power Station Docket Number 50-346, License Number NPF-3 Licensee Event Report 2009-001 Enclosed is Licensee Event Report (LER) 2009-001, "Containment Air Cooler Fans Inoperable Due to Misapplication of Potter and Brumfield Rotary Relays." This LER is being reported in accordance with 10 CFR 50.73(a)(2)(i)(B) as an operation or condition prohibited by the Technical Specifications.
There are no regulatory commitments contained in this letter or its enclosure. The actions described represent intended or planned actions and are described for information only. If there are any questions or if additional information is required, please contact Mr. Dale R. Wuokko, Manager, Site Regulatory Compliance, at (419) 321-7120.
Sincerely, Barry S. Allen Enclosure: LER 2009-001-00 cc:
NRC Region III Administrator NRC Resident Inspector NRR Project Manager Utility Radiological Safety Board
NRC FORM 366 U.S. NUCLEAR REGULATORY COMMISSION APPROVED BY OMB NO. 3150-0104 EXPIRES 8/31/2010 (6-2004)
, the NRC may (See reverse for required number of not conduct or sponsor, and a person is not required to respond to, the digits/characters for each block) information collection.
- 3. PAGE Davis-Besse Nuclear Power Station 05000346 1 OF 5
- 4. TITLE Containment Air Cooler Fans Inoperable Due to Misapplication of Potter and Brumfield Rotary Relays
- 5. EVENT DATE
- 6. LER NUMBER i
- 7. REPORT DATE
- 8. OTHER FACILITIES INVOLVED I SEQUENTIAL REV FACILITY NAME DOCKET NUMBER MONTH DAY YEAR YEAR NUMBER NO MONTH DAY YEAR 05000 IFACILITY NAME DOCKET NUMBER 10 13 2009 2009 -
001 00 12 14 2009 05000
- 9. OPERATING MODE
- 11. THIS REPORT IS SUBMITTED PURSUANT TO THE REQUIREMENTS OF 10 CFR §: (Check all that apply) 20.2201i(b) 20.2203(a)(3)(i)
El 50.73(a)(2)(i)(C)
Li 50.73(a)(2)(vii) 1L 20.2201(d)
Li 20.2203(a)(3)(ii)
Li 50.73(a)(2)(ii)(A)
E]
50.73(a)(2)(viii)(A)
E]
20.2203(a)(1)
E]
20.2203(a)(4)
Li 50.73(a)(2)(ii)(B)
Li 50.73(a)(2)(viii)(B)
[]
20.2203(a)(2)(i)
E]
50.36(c)(1)(i)(A)
Li 50.73(a)(2)(iii)
EL 50.73(a)(2)(ix)(A)
- 10. POWER LEVEL Li 20.2203(a)(2)(ii)
E]
50.36(c)(1)(ii)(A)
E]
50.73(a)(2)(iv)(A)
Li 50.73(a)(2)(x)
Li 20.2203(a)(2)(iii)
E]
50.36(c)(2)
Li 50.73(a)(2)(v)(A)
[
73.71 (a)(4) 100 Ei 20.2203(a)(2)(iv)
[] 50.46(a)(3)(ii)
E]
50.73(a)(2)(v)(B)
Li 73.71 (a)(5)
E]
20.2203(a)(2)(v)
[I 50.73(a)(2)(i)(A)
E]
50.73(a)(2)(v)(C)
Li OTHER Specify in Abstract below
_ 20.2203(a)(2)(vi)
Z 50.73(a)(2)(i)(B)
L] 50.73(a)(2)(v)(D) or in
DESCRIPTION OF EVENT
Event Description
On September 28, 2009, during performance of the CAC 3 18-month Surveillance Test, when'the CAC fan was attempted to be shutdown from slow speed while aligned to electrical train 2, conflicting information was received as to whether the fan had shutdown or remained operating in slow speed.
When the Service Water Outlet Valve for CAC 3 was closed it immediately reopened, which also indicated a defective contact relay in the fan slow speed run circuit. CAC 3 was declared inoperable per TS LCO 3.6.3 Condition C and CAC 2 was placed in service. The two Service Water Outlet Valves for CAC 3 were also declared inoperable as Containment Isolation valves per TS LCO 3.6.3 Condition C due to the slow speed contact preventing the valves from being placed in the closed position, and manual isolation valves were closed to comply with the TS LCO.
During review of the issue with the CAC 3 fan slow speed contact relay a reference to NRC Information Notice 92-19, "Misapplication of Potter & Brumfield MDR Relays," was identified. This Information Notice describes misapplications of Potter and Brumfield MDR Relays like those installed in the CAC fan circuitry. Specifically, when the relay's own contacts are used in the electrical circuit to reset the relay coil, a relay race condition can occur. Consequently, the relays would fail in an intermediate mechanical position due to a partial alternating current waveform being applied to the relay's reset circuit. An earlier FirstEnergy BETA Laboratory failure report (completed July 21, 2009) from a CAC fan 1 relay failure in June 2009 showed that a short or partial alternating current waveform to the reset coil of the MDR relay could place the relay in an intermediate mechanical state (neither open nor closed).
On October 13, 2009, a review of the CAC fan wiring diagrams showed that the fan control circuit utilizes Potter and Brumfield MDR 4121 rotary relays [BK-RLY], to provide a start signal for slow speed operation and to provide an interlock to open the Service Water Inlet and Outlet Valves. The MDR relays in the CAC circuitry use their own contacts to reset the relay coil. Upon receipt of an emergency signal, the two CAC fans connected to the essential buses will automatically start and run in slow speed. The issue with the MDR relays may cause the CAC fan to fail to start due to the relay stopping in an intermediate mechanical position. In order for this failure to occur, the timing has to be just right.
The stopping of the relay in an intermediate mechanical position was able to be reproduced under laboratory conditions. When the relay fails in an intermediate position, the normally closed and normally open contacts are all in the open state. The relays used in the CAC fan circuitry are latch-in style relays, so once the relay is operated it will remain in that condition until a trip signal is received.
Because of this potential to stop in an intermediate position, both operating CAC trains, which were operating in fast speed, were declared inoperable at 1207 hours0.014 days <br />0.335 hours <br />0.002 weeks <br />4.592635e-4 months <br /> on October 13, 2009, and TS LCO 3.6.6 Condition E was entered. Additionally, with Service Water Train 2 and thus Containment Spray Pump 2 inoperable, TS LCO 3.6.6 Condition G was also entered. This required entry into LCO 3.0.3, which required actions to be initiated within one hour to place the unit in Mode 3 within 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br />. At 1222 hours0.0141 days <br />0.339 hours <br />0.00202 weeks <br />4.64971e-4 months <br />, CACs 1 and 2 were started in slow speed to eliminate the potential of the relay failing in an intermediate position, thereby restoring operability of CAC 1, and TS LCO 3.6.6 Conditions E and G and LCO 3.0.3 were exited. Service Water Train 2 operability was restored on October 13, 2009, at 1556 hours0.018 days <br />0.432 hours <br />0.00257 weeks <br />5.92058e-4 months <br />, which also restored the operability of CAC 2.
SSION
==DESCRIPTION OF EVENT==An extent of condition review identified approximately 2 dozen locations at the DBNPS that use these relays. However, only the four CAC fan circuits (CAC 1, CAC 2, CAC 3 as 1, and CAC 3 as 2) use the relay's own contacts to reset the relay coil.
CAUSE OF EVENT
The cause of the Potter and Brumfield MDR 4121 rotary relays to fail in mid-position was a misapplication of the relays. These relays were not designed to have the coil wired through its own contacts, and in doing so, in some instances the contacts can de-energize the coil before the relay has completely moved the contacts to the spring holding position. This causes all the relay contacts to remain open and not in the desired state. The relay manufacturer makes other relays that are specifically designed to use its own contacts to set or reset the latch and unlatch coils, and the physical point gap of the contact is manufactured to allow the correct timing of the device.
When NRC Information Notice 92-19 was issued in 1992 to describe this issue with the Potter and Brumfield MDR rotary relays, there were no MDR relays used at the DBNPS. In 1999, the MDR relays were identified as a replacement for Westinghouse Latching Type AR relays for which the mechanical latch attachments had been discontinued and the magnetic replacements were not seismically qualified. The Potter and Brumfield MDR rotary relays were installed in the CAC fan circuits in 2000 and 2001. Due to an inadequate design change, the operating experience with the Potter and Brumfield MDR rotary relays was not effectively used to prevent the problems identified.
ANALYSIS OF EVENT
While laboratory tests were successful in reproducing a single relay failures to the intermediate mechanical state, this condition occurs with a low incident rate in the field. Each of the eight relays (one to provide a start signal for slow speed operation and one to provide an interlock to open the Service Water Inlet and Outlet Valves) are tested at least twice during monthly surveillance testing, and this event manifested itself only a few times since the relays were installed eight to nine years ago. It is unlikely that both CACs would fail to switch to slow speed during an accident condition. Therefore, this issue was of very low safety significance. The Containment Spray System, which also serves to cool containment during accident conditions and for which two trains are required to be operable per TS LCO 3.6.6 during Modes 1 through 4, was unaffected by the Potter and Brumfield MDR rotary relay issue. Therefore, no loss of safety function occurred as a result of this issue.
Reportability Discussion:
TS LCO 3.6.6 requires two Containment Spray trains and two CAC trains to be Operable in Modes 1 through 4. Condition C allows one CAC train to be inoperable for up to 7 days, and Condition E allows two CAC trains to be inoperable for up to 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. With these relays installed, rendering all CACs inoperable, the plant operated for a period longer than allowed by TS LCO 3.6.6, which is reportable per 10 CFR 50.73(a)(2)(i)(B) as operation in a condition prohibited by the Technical Specifications.
CORRECTIVE ACTIONS
Upon discovery of this condition, CAC fans 1 and 2 were switched from fast speed to slow speed operation to eliminate the potential the fans would fail to start in slow speed during accident conditions.
An Engineering Change was issued on November 4, 2009, to modify the control circuitry of the Containment Air Cooler fans so that contacts from the MDR relays are not used to de-energize their own coils. Instead, auxiliary contacts will be installed on the slow speed starters for the CAC fan motors and the circuit will be re-wired to eliminate the use of MDR relay contacts to de-energize its own relay coil. The circuits for CAC 2, CAC 3 as 1, and CAC 3 as 2 were modified on November 14, November 19, and December 8, 2009, respectively; and until the circuit for CAC 1 is modified, this fan will only be considered operable per TS LCO 3.6.6 if it is operating in slow speed.
A change to procedure NOP-CC-2007, "Part/Component Equivalent Replacement Packages," is being made to include guidance to include an Operating Experience Review as an element in selecting a replacement part or component.
A review was performed to ensure the extended operation of the Containment Air Coolers in slow speed and the resultant increased Containment temperatures did not adversely affect the qualified life of equipment in Containment.
PREVIOUS SIMILAR EVENTS
A previous event in June 2004 when CAC 2 failed to start in slow speed during monthly Surveillance Testing was determined to be the stopping of the MDR relay in the intermediate position. At that time, the most probable causes were considered to be a mechanical problem with the relay, even though disassembly at an internal laboratory did not find any deficiencies. The suspect relay was replaced, and the CAC declared operable following successful post maintenance testing. A similar event also occurred in March 2008 when the CAC Service Water inlet isolation valve would not remain closed when CAC fan 1 was shutdown during system realignment.