05000260/LER-2011-001
| Browns Ferry Nuclear Plant Unit 2 | |
| Event date: | 08-08-2011 |
|---|---|
| Report date: | 10-07-2011 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 2602011001R00 - NRC Website | |
I. PLANT CONDITION(S)
Browns Ferry Nuclear Plant (BFN) - Unit 2 was at 100 percent power.
II. DESCRIPTION OF EVENT
A. Event:
On August 8, 2011, LCO 3.5.1.A, one low pressure ECCS injection/spray system inoperable or one low pressure coolant injection (LPCI) pump in both LPCI subsystems inoperable, was entered to perform Surveillance Procedure 2-SR-3.3.5.1.6 (CS II) "Core Spray [BG] System Logic Functional Test Loop II." The normal interval for performance of this surveillance is 24 months. Relay [RLY] 2-RLY-075-14A-K30B, DG PWR AVAIL 4KV SD BD C (Division II), at step 7.5 [3], contacts 3-7 and 4-8 for normally energized relay 14A-K30B were visually observed to be closed (de-energized state) instead of open as required by the procedure. On August 9, 2011, the cover on the relay was adjusted per WO 112545536 and the contacts went fully open. Surveillance Procedure 2-SR-3.3.5.1.6 (CS II) testing recommenced. Steps 7.5 [75] and 7.11 [5.1] both de- energize the 14A-K30B relay and the relay is subsequently reenergized. The surveillance was exited on August 11, 2011, at 05:45 CDT.
During a walkdown on August 12, 2011, System Engineering personnel determined contacts 3-7 and 4-8 for normally energized relay 14A-K3OB were again determined to be not fully open. The troubleshooting revealed there was a relay problem, which has since been attributed to intermittent relay binding due to a possible manufacturing defect. LCO 3.5.1.A, was again entered. On August 13, 2011, the relay was replaced by Work Order 112565983, the relay passed surveillance testing, and the surveillance was successfully completed.
TVA is submitting this report in accordance with 10 CFR 21.2(c), reporting of defects and non-compliance; 10 CFR 50.73 (a)(2)(i)(B), any operation or condition prohibited by the plant's Technical Specifications; and 10 CFR 50.73 (a)(2)(v)(D), as any event or condition that could have prevented the fulfillment of the safety function of structures or systems that are needed to mitigate the consequences of an accident. Past inoperability is based on the equipment history and cause of failure; and a conclusion was reached that the relay should be considered non-functional from April 27, 2011, until the relay was replaced and successfully passed PMT's on August 13, 2011, at 1355 CDT.
B. Inoperable Structures, Components, or Systems that Contributed to the
Event:
None C. Dates and Approximate Times of Major Occurrences:
April 27, 2011@1636 CDT Relay received receipt inspection. No discrepancies noted. Material inspection form signed on April 12, 2009.
Relay was bench tested prior to installation in the plant.
Bench test verifies the relay will energize and de-energize.
The test verifies relay contacts change state by checking continuity and visually inspects relay contacts for signs of pitting and corrosion. The PMT after installation verified continuity across normally closed relay contacts 3-7 and 4-8. The relay was also visually verified to be energized and that normally closed relay contacts 3-7 and 4-8 were open. The testing verified proper operation of the relay and contacts and their material condition. Relay BFN-2-RLY- 075-14A-K3OB is located in panel 2-PNLA-009-0033.
Relay verified to be energized in performing step 7.5 [3] of Procedure 2-SR-3.3.5.1.6 (CSII), CS System Logic Functional Test, Loop II.
Relay verified to be energized during performance of step 7.11 [8] A, SR 0-SR-3.8.1.9(C), Diesel Generator C Emergency Unit 1 Load Acceptance Test.
Thermography determined relay to be energized.
Loss of offsite power (LOSP) to Units 1, 2, and 3 occurred due to severe weather causing tornadoes and high winds in the immediate vicinity.
Units 1, 2, and 3 experienced a reactor scram due to loss of offsite power. The loss of offsite power condition resulted in the diesel generator supplying power to the 4KV shutdown boards C and D, [EB] This would have caused Relay 2 RLY-075-14A-K3OB to de-energize and subsequently re-energize (cycle the contacts).
No procedures or PMs that removed the relay cover.
OPS performed 2-SR-3.3.5.1.6 (CS11), "Core Spray System Logic FT-Loop II".
CS Loop II declared inoperable for performance of 2-SR-3.3.5.1.6 (CS II). Entered TS LCO 3.5.1 Condition A.
Required Action is to restore LP ECCS to Operable Status within 7 days.
April 8, 2009 May 5, 2009 August 8, 2009 November 13, 2010 February 10, 2010 April 27, 2011@1635 CDT November 13, 2010 to August 8, 2011 August 8, 2011@ 08:50 CDT August 8, 2011@ 10:40 CDT August 8, 2011@18:20 CDT August 9, 2011@ 1:47 CDT August 9, 2011@ 3:26 CDT August 9, 2011@ 4:00 CDT August 11, 2011@ 5:45 CDT Ops completed 2-SR-3.3.5.1.6 and exits action statement entered on August 8, 2011, at 10:40 CDT.
August 12, 2011@ 9:45 CDT Determined Relay 2-RLY-075-14AK3OB contacts in wrong position. Entered TS LCO 3.8.7.A.1 and A.2 for the C 4kV SDBD and TS 3.8.1.B.1, B.2, and B.3 for the C D/G.
(Second entry) August 12, 2011@ 11:05 CDT Entered TS 3.5.1.A, one low pressure ECCS injection/spray subsystem inoperable.
August 13, 2011@ 10:00 CDT W0112565983 initiated to replace relay for 2B Core Spray Pump.
August 13, 2011@ 13:55 CDT Relay energized correctly during post-maintenance testin g.
August 13, 2011@ 15:45 CDT Operated 2B Core Spray pump IAW-2-01-75 to verify pump operation and exited LCO that was entered on August 12, 2011, at 11:05 CDT.
D. Other Systems or Secondary Functions Affected
None
E. Method of Discovery
The contacts were discovered to be in the incorrect position during performance of surveillance testing, Procedure 2-SR-3.3.5.1.6(CS II), Core Spray System Logic Functional Test Loop II. This occurred two (2) times.
F. Operator Actions
Operations personnel declared one low pressure ECCS injection/spray subsystem inoperable (Unit 2 TS 3.5.1.A) from 10:40 on August 8, 2011, until 05:45 on August 11, 2011. Additionally, another period of inoperability was declared between 11:05 on August 12, 2011, until 15:45 on August 13, 2011.
G. Safety System Responses
There were no safety system responses.
III. CAUSE OF THE EVENT
A. Immediate Cause
The contacts for HGA relay 14A-K30B failed to open.
B. Root Cause
A Kepner-Trego (K-T) analysis was performed by BFN Site Engineering to analyze the equipment failure. The K-T analysis determined the most probable cause of the relay failure was a manufacturing defect causing intermittent binding of the relay contacts. A cursory inspection of the replaced relay revealed intermittent sticking/binding of the movable contacts. The hinged armature can intermittently become bound in an undesired position, limiting the ability of the relay to operate as expected.
The HGA relay was also shipped to the Tennessee Valley Authority's Central Labs to determine the failure mechanism. The failure report stated that the General Electric HGA relay's hinged armature can intermittently become bound in an undesired position, limiting the ability of the relay to operate as expected. External forces do not appear to be a factor, as in the misplacement of the cover. The binding takes place within the hinged armature mechanism, particularly on the right side; however, binding occurred intermittently in both directions. The failure mechanism for this relay is a bound hinged armature. A specific cause for the binding was not identified.
The relay was also sent to GE-Hitachi for their review and analysis.
IV. ANALYSIS OF THE EVENT
The condition being reported is the operation of BFN Unit 2 in a manner prohibited by Technical Specifications, the loss of a safety function, and a defect in a basic component.
LCO 3.3.5.1.A requires that with one or more channels inoperable, to immediately enter the condition referenced in Table 3.3.5.1-1 for the channel. For the Core Spray system, Function 1e in Table 3.3.5.1-1 is the core spray pump start with time delay relay and pumps A, B, C, and D (with diesel power) should start between 6 to 8 seconds. For the LPCI system, with time delay relay and pumps A, B, C, and D on diesel power, the pumps should start between 0 and 1 seconds. The condition associated with the HGA relay may not have allowed the RHR pumps and Core Spray pumps to sequence properly. Core Spray logic is such that the B and D pumps either both start or both do not start. RHR pumps in the LPCI mode may have one pump not start, but their logic is not tied together as that previously discussed for the Core Spray pumps. The relay failure caused the Core Spray pump motor sequencing logic to be inoperable. In addition, the opposite division was removed from service between April 27, 2011, until August 13, 2011, when the relay was replaced.
Therefore, over this time interval, the safety function provided by core spray was not always available.
V. ASSESSMENT OF SAFETY CONSEQUENCES
When normal power is lost at bus C, relay 14A-K29B de-energizes which drops out relay 14A-K124B. With relay 14A-K30B de-energized due to the malfunction, relay 14A-K123B remains energized, which maintains the start signal on the 2B Core Spray pump. (Note:
Bus C supplies power for the 2B Core Spray pump). The 2B Core Spray pump will be tripped by 27SCX and/or 27SCY relays. When power comes back on the C bus, the 27SCX and/or 27SCY relays will reset and the standing start signal will reclose the 2B Core Spray breaker without a time delay. Without this time delay, the Residual Heat Removal (RHR) [BO] pumps and the Core Spray would have attempted to start at the same time, which cannot be supported by the diesel. One or both of the pump motors would be expected to stall and trip on over current.
With normal power available, no problems occur as a result of a relay failure. If both the C and D bus lose power and regain a power supply from the diesel generators, the Core Spray and RHR pumps would have sequenced back on.
However, for the single failure of the C bus coincident with an accident signal, the RHR pumps and Core Spray pumps would have attempted to start at the same time and one or both of the pump motors would stall and trip on over current.
VI. CORRECTIVE ACTIONS
TVA will track completion of these actions in the Corrective Action Program.
A. Immediate Corrective Actions
The relay that was found in the incorrect position (BFN-2-RLY-075-14A-K30B) was replaced. The contacts for other HGA relays identified in the extent of condition have been visually verified to be in the correct position.
Work orders have been prepared to have HGA relays, which were received in the same lot, manually cycled a minimum of 20 times to verify no binding of the hinged relay armature occurs.
B. Corrective Actions to Prevent Recurrence
A PM that checks coils using thermography will be expanded to also include checking the contacts position Other corrective actions:
1. Address relay binding. Submit a revision to Preventive Maintenance (PM) to visually verify the relay contacts are in the correct position. (Projected completion date is October 28, 2011) 2. Address relay binding. Submit a revision to PM to visually verify relay contacts are in the correct position. (Projected completion date is November 14, 2011) 3. Address relay binding. Revise ECI-0-000-RLY003 (Electrical Corrective Instruction Replacement of relays) to test the HGA relays being installed by cycling the relay a minimum of 20 times to verify no binding of the hinged relay armature occurs. (Projected completion date is December 15, 2011)
VII. ADDITIONAL INFORMATION
A. Failed Components
GE HGA relay, PN 1376C6183P014, Model Number 12HGA111A1F.
B. Previous Similar Events
Browns Ferry Unit 3 Failure Number 484 (PERs 141604 and 141624:
On April 4, 2008, at 1200 hours0.0139 days <br />0.333 hours <br />0.00198 weeks <br />4.566e-4 months <br />, following implementation of DCN 51017 Stage 2, U3 CASA logic modifications, continuity checks were performed on all relay contacts, relay coils and wire terminations in Panel 3-9-32. During performance of this PMT, 3- RLY-74-10AK104C contacts 3-7 and 3-RLY-75-14A-K30A contacts 3-7 and 4-8 were identified to be "open" when they should have been closed. PERs 141604 and 141624, respectively, were initiated to document these findings. WOs 08-714071-000 and 08- 714101-000 were initiated to replace the relays. The relays were removed and inspected by Systems and Design Engineering. The initial inspection identified that the armature assembly was easily disturbed from its normally pulled forward de- energized state. The ease of movement was attributed to a weak spring that did not demonstrate the normal amount of tension on the armature to ensure the normally closed contacts were firmly made up. Multiple continuity checks were performed on these relays with similar findings as documented in the WOs. During the visual inspections performed, the armature was verified to be free from binding, all connections appeared to be secure, the contacts were not pitted or severely corroded and the normally closed contacts were verified to be made up. These relays were then sent to Central Labs (CL) for failure analysis. This evaluation of the relays at CL concluded that the control springs did not keep the normally closed contacts firmly pulled together due to the control spring tension. The corrective action was to replace the relays. Additionally, another action was to replace the relays listed in the extent of condition.
On November 5, 2008, at 03:45 CST, during performance of 1/2-ETU-SMI 1-C.4 (Relay Functional Checks on 4 kV Shutdown Board C) [EB], General Electric Company model 12HGA11A51F (12HGA11A51F), model 12HGA11A51F (12HGA11A51F relays, BFR-1-RLY -074- 10AK132B and BFN-2-RLY -074-10AK132B failed to de-energize. When the relay covers were removed and reinstalled on these relays, the Trip Signal Cancel Relay (TSCRN) (0-RLY-211-TSCRNC) relay located on Panel 25-45C energized as expected. The apparent cause was a weakened spring or bonding of the contacts. Corrective actions consisted of replacing the relays, initiated thermography scan PMs to proactively detect adverse trends associated with normally energized risk significant HGA relays.
C. Additional Information
The corrective action document for this report is PER 415242.
D. Safety System Functional Failure Consideration:
This event is considered a safety system functional failure. The fulfillment of a safety function (core spray) during a loss of offsite power could have been prevented; therefore in accordance with NEI 99-02 guidance, this event is considered a safety system functional failure.
E. Scram With Complications Consideration:
This event did not include a reactor scram.
F. 10 CFR 21 Reporting Requirements The following information is provided at this time to meet the requirements of 10 CFR Part 21.21(d)(4) (i)-(viii).
(i) Name and address of individual informing the Commission:
Mr. K. J. Poison Vice-President-Browns Ferry Nuclear Plant Tennessee Valley Authority P. 0. Box 2000 Decatur, Alabama 35609-2000 (ii) Identification of the facility, the activity, or the basic component supplied for such facility or such activity within the United States which fails to comply or contains a defect.
Facility: Browns Ferry Nuclear Plant Basic component which contains a defect: GE relay, Q Level QA 1, type:
electromagnetic, coil volts: 250 DC, construction: semi-flush mount, rear connections, Type HGA. Relay in located in the Core Spray System logic.
(iii) Identification of the firm constructing the facility or supplying the basic component which fails to comply or contains a defect.
Basic component supplier: GE-Hitachi Nuclear Energy 3901 Castle Hayne Rd Wilmington, NC 28402 GE Part No. is 1376C6183P014 (iv) Nature of the defect or failure to comply and the safety hazard which is created or could be created by such defect or failure to comply.
Nature of the defect:
The General Electric HGA relay's hinged armature can intermittently become bound in an undesired position, limiting the ability of the relay to operate as expected. External forces do not appear to be a factor, as in the misplacement of the cover. The binding takes place within the hinged armature mechanism, particularly on the right side; however binding occurred intermittently in both directions. The failure mechanism for this relay is a bound hinged armature.
Safety hazard which could be created by such defect:
When normal power is lost at bus C, relay 14A-K29B de-energizes which drops out relay 14A-K122224B. With relay 14A-K30B de-energized due to the malfunction, relay 14A-K123B remains energized, which maintains the start signal on the 2B core spray pump. (Note that bus C provides power to the 2B Core Spray pump). The 2B pump will be tripped by 27SCX and/or 27SCY relays. When power comes back on the C bus, the 27SCX and/or 27SCY relays will reset and the standing start signal will reclose the 2B Core Spray breaker without a time delay. Without this time delay, the RHR pumps and the Core Spray would have attempted to start at the same time which cannot be supported by the diesel. One or both of the pump motors would be expected to stall and trip on overcurrent.
(v) The date on which the information of such defect or failure was obtained.
October 6, 2011.
(vi) In the case of a basic component which contains a defect or fails to comply, the number and location of all such components in use at, supplied for, or being supplied for one or more facilities or activities subject to the regulation of this part.
Number and Location of all such components in use at BFN was restricted to those relays received on or after the receipt of the defective relay since no failure have been observed with those received earlier.
BFN Item Number 2-RLY-075-14A-K30A 2-RLY-074-10AK104B 2-RLY-075-14A-K32A 3-RLY-075-14A-K32A 2-RLY-074-10AK104C 3-RLY-074-10AK104A 2-RLY-074-10AK104A 3-RLY-074-10AK104B 2-RLY-075-14A-K3OB 3-RLY-074-10AK104D 2-RLY-075-14A-K32B 3-RLY-075-14A-K32B 2-RLY-074-10AK104D 3-RLY-075-14A-K3OB (vii) The corrective action which has been, is being, or will be taken; the name of the individual or organization responsible for the action; and the length of time that will be taken to complete the action.
The corrective actions in response to this event are discussed in the LER, Section VI., Corrective Actions.
(viii) Any advice related to the defect or failure to comply about the facility, activity, or basic component that has been, is being, or will be given to purchasers or licensees.
In addition to the immediate corrective actions taken and proposed corrective actions, WA has forwarded the relay and internal evaluation report to GE-Hitachi, who is performing a separate analysis.
VIII. COMMITMENTS
None