ML13109A234
ML13109A234 | |
Person / Time | |
---|---|
Site: | Wolf Creek ![]() |
Issue date: | 07/12/2013 |
From: | Mathew R K Division of Engineering |
To: | Chernoff H K NRC/NRR/DIRS/IOEB |
Sahay P P | |
References | |
TAC ME8004 | |
Download: ML13109A234 (12) | |
Text
July 12, 2013 MEMORANDUM TO: Harold K. Chernoff, Chief Operating Experience Branch Division of Inspection and Regional Support Office of Nuclear Reactor Regulation FROM: Roy K. Mathew, Acting Chief /RA/ Electrical Engineering Branch Division of Engineering Office of Nuclear Reactor Regulation
SUBJECT:
SAFETY EVALUATION REGARDING WOLF CREEK GENERATING STATION - AUGMENTED INSPECTION - LOSS OF OFFSITE POWER AND NOTIFICATION OF UNUSUAL EVENT, ISSUE FOR RESOLUTION 2012-004. (TAC NO. ME8004)
The Electrical Engineering Branch has reviewed the Issue For Resolution 2012-004, AWolf Creek Generating Station- Augmented Inspection - Loss Of Offsite Power and Notification Of Unusual Event.@ Enclosed is our safety evaluation. This completes our review and evaluation efforts for TAC NO. ME8004. Enclosure:
As stated
CONTACTS: Prem P. Sahay, NRR/DE/EEEB (301) 415-8439
Peter J. Kang, NRR/DE/EEEB (301) 415-6800
July 11, 2013 MEMORANDUM TO: Harold K. Chernoff, Chief Operating Experience Branch Division of Inspection and Regional Support Office of Nuclear Reactor Regulation FROM: Roy K. Mathew, Acting Chief /RA/ Electrical Engineering Branch Division of Engineering Office of Nuclear Reactor Regulation
SUBJECT:
SAFETY EVALUATION REGARDING WOLF CREEK GENERATING STATION - AUGMENTED INSPECTION - LOSS OF OFFSITE POWER AND NOTIFICATION OF UNUSUAL EVENT, ISSUE FOR RESOLUTION 2012-004. (TAC NO. ME8004)
The Electrical Engineering Branch has reviewed the Issue For Resolution 2012-004, AWolf Creek Generating Station- Augmented Inspection - Loss Of Offsite Power and Notification Of Unusual Event.@ Enclosed is our safety evaluation. This completes our review and evaluation efforts for TAC NO. ME8004. Enclosure:
As stated
CONTACTS: Prem P. Sahay, NRR/DE/EEEB (301) 415-8439
Peter J. Kang, NRR/DE/EEEB (301) 415-6800
DISTRIBUTION: JRobles FIssa RidsNrrDeEeeb NRR_DE_DPR ADAMS ACCESSION NO: ML13109A234 OFFICE DE/EEEB DE/EEEB DE/EEEB (A)BC NAME PSahay PKang RMathew DATE 7/11/2013 7/11/2013 7/12/2013 OFFICIAL RECORD COPY ENCLOSURE EVALUATION REGARDING WOLF CREEK GENERATING STATION AUGMENTED INSPECTION - LOSS OF OFFSITE POWER AND NOTIFICATION OF UNUSUAL EVENT, ISSUE FOR RESOLUTION 2012-004 (TAC NO. ME8004)
1.0 PURPOSE The purpose of this Issue For Resolution (IFR) is to evaluate the root cause of the LOOP event, the safety significance, determine similar events at other plants, and provide recommendations for issuing generic communications or other regulatory actions. This IFR applies to all operating and new reactors. 2.0 BACKGROUND According to the License Event Report (LER) 2012-001-00, Wolf Creek unit 1 experienced an unplanned automatic shutdown from full power operation and a coincident loss of offsite power (LOOP) for greater than 15 minutes, resulting in the declaration of a Notification of Unusual Event (NUE) on January 13, 2012 at 14:03 Central Standard Time (CST). A Wolf Creek Site Watch Operator reported to the Control Room that the main generator output breaker 345-60 had experienced a failure with substantial visual damage. At 1537 hours0.0178 days <br />0.427 hours <br />0.00254 weeks <br />5.848285e-4 months <br /> the Wolf Creek Turbine Building Watch notified the Control Room of a dropped flag from a Differential relay actuation (487/T1B) on a panel MA104D for the Startup Transformer. This failure and faulted condition of the breaker 345-60 resulted in actuation of protective relays for the East Bus Primary lockout, East Bus Secondary lockout, and 345-60 breaker failure. These relay actuations caused the opening of switchyard breakers and isolated the east switchyard bus, resulting in a turbine trip and subsequent reactor trip. To counter this loss of power, a "fast bus transfer" occurred that realigned the power supply for the non-safety buses to the Startup Transformer (XMR01). As soon as the fast bus transfer was complete, the 487/T1 'B' Phase Startup Transformer differential relay actuated per design. The Startup Transformer Differential relay (487/T) sends inputs to the 286/T1 lockout relay, which initiates de-energization of the Startup Transformer. This protective sequence generates a signal to isolate the Startup Transformer power supply to ESF Transformer XNB02. Additionally, this Startup Transformer 286/T1 lockout relay feeds protection circuitry in the WCGS switchyard to open switchyard breakers to isolate the West 345 kilo Volt (kV) bus and de-energize the Startup Transformer.
This sequence of events, which isolated and de-energized the WCGS switchyard East 345 kV and West 345 kV buses, resulted in the loss of offsite AC electrical power to the unit. The turbine trip initiated a reactor trip. All control rods inserted into the core and all reactor coolant pumps tripped per design. The undervoltage relays on safety related 4.16 kV engineered safety features (ESF) buses NB01 and NB02 actuated and initiated a loss of power DG start signal for the automatic start of both emergency diesel generators (DGs). Both DGs started and supplied power to the 4.16 kV ESF buses with loads sequenced onto the buses per design.
At 1700 CST offsite power was restored to the East 345 kV bus, Transformer XNB01, and the NB01 bus. With the restoration of an operable offsite circuit, the Unusual Event was terminated at 1709 CST. At 1721 CST the 'A' DG was placed in a standby condition.
- 2 - On February 13, 2012, at 2008 CST, an attempt was made to start the 'A' Reactor Coolant Pump (RCP) for troubleshooting. This start attempt resulted in the second loss of electrical power to the Startup Transformer. The licensee's investigation revealed that the Startup Transformer 487/T1 'B' phase Differential relay and the 286/T1 lockout relay were tripped as well as the 94F West Bus Secondary Differential relay in the Switchyard, which had initiated the automatic opening of Switchyard breakers resulting in de-energization of the switchyard West 345 kV bus.
In its Root Cause Report CR 47653 (IIT 2012-001), the licensee stated that the January 13, 2012 event was the third instance of LOOP experienced by Wolf Creek within past four years.
The first loss of offsite power occurred on April 7, 2008 (LER No. 4822008004), when one offsite source (the Switchyard No. 7 Transformer) was out of service, and a human error isolated the one remaining offsite source (the West Bus) from the Startup Transformer, resulting in a loss of offsite power. The unit was in Mode 6 with the Reactor de-fueled at the time. The second LOOP event occurred in August 19, 2009 (LER No. 4822009002), when both offsite sources were lost due to a lightning strike on its transmission line. The unit was in Mode 1 both times. The second LOOP was caused by offsite transmission grid events and conditions.
LOOP on January 13, 2013 was the third occurrence for the Wolf Creek Plant.
3.0 ROOT CAUSE EVALUATION The Wolf Creek Switchyard and its connections to the unit are shown in the Attachment 2. The Wolf Creek unit is connected to the Switchyard through main generator breaker to provide generator output to the Switchyard and transmission grid and through the main Step up Transformer to provide offsite power to the plant equipment.
In its root cause report, the licensee stated the following:
1. The most likely failure mechanism of the Generator Output breaker phase/pole 'C' was the internal particulate or contaminations in the breaker internal environment that set up a path for a flashover across the insulator surface. However, due to the extensive damage incurred by the breaker pole, it is not possible to definitively identify the exact internal location and source of the particulate material.
2. The initial testing and troubleshooting was performed on the Startup Transformer and associated protective relaying. Troubleshooting did not reveal any deficiencies that could have caused the LOOP on January 13, 2012. During the original troubleshooting effort the junction boxes that contained the Current Transformer (CT) junction boxes were not visually inspected. Standard/regular electrical tests individually verified XMR01 differential CT circuits for each phase as not having any short or ground. However, the standard electrical test configuration would not identify shorts between unused CT taps of different CTs or phases. Due to the lack of any monitoring at the time of the event the maintenance crew was not aware that the 'B' phase Differential relay circuit was not generating the expected current, and phase shift. The point-to-point meggering between phase checks that eventually detected the CT short on the unused taps, is not a typical industry standard test used and thus was not originally implemented. As a result, the
- 3 - Startup Transformer was reenergized from off-site power at 1207 hours0.014 days <br />0.335 hours <br />0.002 weeks <br />4.592635e-4 months <br /> on February 3, 2012, with the later identified CT short still existing. 3. After the February 13, 2012, Startup Transformer (XMR01) differential trip (Failure Mode), troubleshooting on February 16, 2012, identified a short on XMR01 high side CT wires that feeds the differential protection scheme. During the troubleshooting, megger testing indicated a phase-to-phase short circuit between two unused high side CT taps (CT Wire W2 on 'A' phase and W10 on 'B' phase). Subsequent field inspections found two missing insulation sleeves on these CT wires, originating from last refueling outage vendor work on the transformer.
The following sequence of events is based on the excerpt from the licensee=s root cause report of the Wolf Creek event (Attachment 2). The sequence of events occurred in the following order (only important events are listed):
Sequence of Events Date - 01/13/2012 Event Time Summary of Event Event Description Wolf Creek Unit was operating at 100% power prior to this event. 14:02:54.740 Wolf Creek East Bus Diff (T = 0) Switchyard supervisory point indicating the trip of an east buss differential relay. 14:02:54.753 Main Generator Switchyard BKR No. 345-60 is OPEN This point is a direct indication of breaker 345-60 opening. It indicates breaker open when all three phases of breaker 345-60 are open. 14:02:54.768 Wolf Creek 345-50 open 14:02:54.919 Startup Transformer lockout 286/T1 has actuated. 487/T1 "B" phase Start-up Transformer Differential target flagged (T) Indication that Startup Transformer lockout 286/T1 has actuated. 487/T1 "B" phase Start-up Transformer Differential target flagged (T). 14:02:54.922 Startup transformer trip. Indication in switchyard that the 286/T1 lockout has actuated in the plant. Switchyard cross trip relay 94P/W is in parallel with the plants 286/T1 lockout and they were both actuated by the S/U transformer 487/T1 differential relay. The 94P/W relay actuates the west bus lockout 86 P/W. 14:02:54.930 Wolf Creek West Bus Diff (Lockout) Loss of Offsite Power occurred. 14:02:54.994 REAC MAIN TRIP BKR B is TRIPD Reactor tripped. 14:02:55.006 REAC MAIN TRIP BKR A is TRIPD Reactor tripped. 14:03:02.656 DG NE02 FDR BKR is CLOSD Emergency Diesel Generator started as designed. 14:03:02.953 DG NE01 FDR BKR is CLOSD Emergency Diesel Generator started as designed.
- 4 -
- 5 - The Wolf Creek root cause investigation (CR 47653 (IIT 2012-001)) identified two key issues that resulted in the LOOP event, namely, failure to identify presence of the particulate or contaminations on the main generator output breaker phase/pole "C" and correct it and failure to identify two un-insulated terminal wires in contact (i.e. shorted) on the CT for Start up Transformer 487 differential relays. Both cases appear to be due to human errors.
The licensee's Hardware Failure Analysis identified the most likely failure mechanism of the 345-60 Generator Output breaker to be particulate contamination in the breaker internal environment that set up a path for a flashover across the insulator surface. An effective maintenance program by the licensee would have detected and resolved the presence of particulates and could have avoided main generator output breaker failure.
On February 16, 2012, at approximately 2008 hours0.0232 days <br />0.558 hours <br />0.00332 weeks <br />7.64044e-4 months <br />, with the plant in Mode 5, the 'A' Reactor Coolant Pump (RCP) was started. This start attempt resulted in another protective lockout of the Startup Transformer. This transformer lockout resulted in the isolation of the Wolf Creek substation West bus and the loss of electrical power to the non-safety related 13.8 kV buses as well as 'B' Train safety-related 4.16 kV bus NB02. While performing current transformer (CT) testing on interfaces with the Differential Relay protection zone, the licensee identified an electrical short in the phase 'A' and 'B' CT terminal wiring block. Two wires that were not insulated (as they should have been) were in contact, causing an electrical short and that resulted in Step up transformer differential relay actuation and tripping of transformer as designed. This CT terminal junction block was last worked during refueling outage RF18 by the vendor company, and had previously tested satisfactorily at the completion of work during RF18. Wolf Creek did not recognize the risk/consequence of having a vendor perform work in accordance with vendor procedures and processes, without established verification methods for ensuring work quality, thus resulting in an undetected human performance error.
Had the licensee established and implemented effective verification methods for ensuring work quality, an electrical short in the phase 'A' and 'B' CT terminal wiring block could have been identified and corrected and startup transformer differential relay actuation and startup transformer trip and LOOP could have been avoided.
The NRC dispatched an augmented inspection team to review the facts surrounding the event (NRC Inspection Report 05000482/2012008 dated March 4, 2012, Agencywide Documents Access and Management System (ADAMS) Accession Number ML12095A414). The team identified several unresolved items (URIs) requiring follow-up inspection. Specifically, one URI involved reviewing the root cause analysis of the main generator output breaker fault when it is completed and follow up actions, and the another URI involved reviewing the root cause analysis of the startup transformer fault when it is completed and follow up actions.
4.0 SAFETY SIGNIFICANCE The event is considered safety significant, as it resulted in unit trip, LOOP, and starting of the safety related equipment due to human performance errors.
In addition, the January 13, 2012, event was the second LOOP event caused by human error within four years. The first loss of offsite power occurred on April 7, 2008, when one offsite
- 6 - source (the Switchyard No. 7 Transformer) was out of service, and a human error isolated the one remaining offsite source (the West Bus) from the Startup Transformer, resulting in a loss of offsite power. The unit was in Mode 6 with the Reactor de-fueled at the time.
Had the licensee established and implemented an effective maintenance program including independent verification requirements for ensuring work quality and identification and elimination of human errors based on the lessons learned from April 7, 2008 plant event, Wolf Creek could have avoided another LOOP and unplanned plant trip on January 13, 2012.
5.0 RECENT LOOP EVENTS IN OTHER NUCLEAR PLANTS
- Oyster Creek experienced LOOP on July 23, 2012, due to an electrical fault on an offsite transmission line (outside plant). (ref. PNO-I-12-005)
- Byron Units 1 and 2 experienced LOOP on January 30, 2012, at 10:18 AM (CST). (ref. LER No. 2012-001-01)
- Point Beach Nuclear plant Unit 1 declared NOUE following LOOP on November 27, 2011, at 2:38 AM (CST). (ref. LER No. 2011-001-00)
- Browns Ferry Nuclear Plant Units 1, 2, and 3 declared NOUE due to LOOP on April 27, 2011, at 6:21 PM (EDT). (ref. LER No. 2011-001-00)
- Surry Power Station Units 1 and 2 declared NOUE due to LOOP from a tornado on April 16, 2011, at 7:48 PM. (ref.LER No. 2011-001-00)
- North Anna Units 1 and 2 experienced LOOP during seismic event on August 23, 2011. (ref. LER No. 2011-003-00)
- Calvert Cliff Unit 1 and 2 experienced Partial LOOP on February 18, 2010, at 8:24 AM (ref. 2010-001-01).
- Point Beach Nuclear plant Unit 1 declared NOUE following LOOP on January 15, 2008. (ref. LER No. 2008-001-00)
Two out of a total of eight LOOP events (including Wolf Creek January 13, 2012 event) in year 2011 and year 2012 were due to natural causes such as tornado and earthquakes. From above list of LOOP events, it is evident that overall LOOP events in years 2011 and 2012 have significantly increased (including Wolf Creek January 13, 2012 event) compare to year 2008, 2009 (Wolf Creek August 19, 2009 event) and 2010 and supports NRC concern on upward trend of the LOOP events in recent two years.
The staff reviewed the NRC Technical Review Group (TRG) Reports for the Year 2011 dated November 11, 2012 and 2012 (Agencywide Documents Access and Management System
- 7 - (ADAMS) Number ML12306A261) and finds that both reports highlighted the issue on the high number of LOOP/inoperable offsite power sources due to the equipment failure and/or human errors, consistent with the staff's finding on this IFR.
The staff also reviewed following active generic communications on the plant events resulting in LOOP:
1. Information Notice (IN) 2006-18, "Significant Loss of Safety-Related Electrical Power at Forsmark Unit 1 in Sweden."
2. IN 2007-14, "Loss of Offsite Power and Duel-Unit Trip at Catawba Nuclear Generating Station."
3. IN 2009-10, "Transformer Failures - Recent Operating Experience." 6.0 IMPACT OF FREQUENCY OF LOOP ON STATION BLACKOUT COPING DURATION Regulatory Guide 1.155, August 1988, "Station Blackout" states, "Based on Title 10 of the Code if Federal Regulation, Part 50.63 (10 CFR 50.63), all licensees and applicants are required to assess the capability of their plants to maintain adequate core cooling and appropriate containment integrity during a station blackout and to have procedures to cope with such an event. This guide presents a method acceptable to the NRC staff for determining the specified duration for which a plant should be able to withstand a station blackout in accordance with these requirements. The application of this method results in selecting a minimum acceptable station blackout duration capability from 2 to 16 hours1.851852e-4 days <br />0.00444 hours <br />2.645503e-5 weeks <br />6.088e-6 months <br />, depending on a comparison of the plant's characteristics with those factors that have been identified as significantly affecting the risk from station blackout. These factors include redundancy of the onsite emergency ac power system (i.e., the number of diesel generators available for decay heat removal minus the number needed for decay heat removal), the reliability of onsite emergency ac power sources (e.g., diesel generators), the frequency of LOOP, and the probable time to restore offsite power. Licensees may propose durations different from those specified in this guide. The basis for alternative durations would be predicated on plant-specific factors relating to the reliability of alternating current (AC) power systems such as those discussed in
Reference
2."
As discussed above, the frequency of LOOP is one of the factors affecting SBO coping duration. If the frequency of LOOP has changed since last analysis, the coping duration is no longer valid.
7.0 CONCLUSION
Based on the above, the staff concludes the following:
1. The Wolf Creek event on January 13, 2012, appears to be plant specific as staff review of the operating experience database did not find similar plant events occurring in other nuclear plants.
- 8 - 2. The LOOP events in the US nuclear industry continued to rise for the year 2011 and 2012 compared to year 2008 and 2010 as evident from the LOOP events listed in Section 5.00.
3. Despite NRC concerns communicated to the nuclear industry in INs 2006-18, 2007-14 and 2009-10, number of LOOP events continue to increase in year 2011 and 2012.
8.0 RECOMMENDATIONS 1. This Issue for Resolution (IFR) Evaluation should be sent to all the Engineering Branch Chiefs/Division of Reactor Safety in the Regions including Region 2 Division of Construction Inspection. The regional inspectors should refer to this IFR when conducting inspections in maintenance and surveillance areas.
2. Although the January 13, 2012 plant events at Wolf Creek appear to be plant specific, and similar plant events have already been addressed in above INs, the staff finds that number of LOOP events occurring in the nuclear industry have suddenly increased in year 2011 and 2012. As such, the staff recommends issuing appropriate generic communication to address its concern on overall increase in LOOP events in recent years.
9.0 ATTACHMENTS (1) Excerpt from Wolf Creek Root Cause Evaluation Report CR 47653 (IIT 2012-001) Executive Summary (2) Wolf Creek Nuclear Station Switchyard diagram and its connection to the unit (1 sheet)
Principal Contributor: Prem P. Sahay
- 9 - ATTACHMENT 1 Excerpt from Wolf Creek Root Cause Evaluation Report CR 47653 (IIT 2012-001) Executive Summary On January 13, 2012 Wolf Creek experienced a loss of offsite power, the third such instance experienced by Wolf Creek within the past four years. The first loss of offsite power (April 7, 2008) occurred when one offsite source (the Switchyard No. 7 Transformer) was out of service, and a human error isolated the one remaining offsite source (the West Bus) from the Startup Transformer, resulting in a loss of offsite power.
The unit was in Mode 6 with the Reactor de-fueled at the time. The second and third LOOP events occurred when both offsite sources were lost due to two failure mechanisms that occurred at approximately the same time (i.e., within cycles of one another). The unit was in Mode 1 both times. The second LOOP was caused by offsite transmission grid events and conditions. The first and third LOOPs initiated at Wolf Creek each involved a human error. Wolf Creek management established an Incident Investigation Team (IIT) on January 13, 2012 to perform a Root Cause Analysis (RCA) of the unplanned station shutdown coincident with a loss of offsite power (LOOP). The event occurred at approximately 1403 hours0.0162 days <br />0.39 hours <br />0.00232 weeks <br />5.338415e-4 months <br /> on January 13, 2012 and a Notification Of Unusual Event (NUE) was declared due to the LOOP lasting longer than 15 minutes. The IIT was chartered by and reported to the Vice President - Strategic Projects. An event focus meeting was held on January 14, 2012 to commence the investigation.
Additionally, on 2/13/12 at approximately 2008 hours0.0232 days <br />0.558 hours <br />0.00332 weeks <br />7.64044e-4 months <br />, with the plant in Mode 5 the 'A' Reactor Coolant Pump (RCP) was started. This start attempt resulted in another protective lockout of the Startup Transformer. This transformer lockout resulted in the isolation of the Wolf Creek substation West bus and the loss of electrical power to the non-safety related 13.8 kV buses as well as 'B' Train safety-related 4.16 kV bus NB02. Personnel interviews, reviews of pertinent documents, computer modeling of protective relay schemes of the applicable plant electrical buses, hardware failure analysis of the breaker 345-60 and potential transformer (PT) 113-1, and extensive hardware inspection and testing were used to generate the bases related to the causes and consequences associated with the causal factors identified. Two Event and Causal Factor (E&CF) Charts (overview and details) were developed using a detailed Sequence of Events (SOE) report. Fault Tree Analysis, Evidence and Action Matrix, Change Analysis, Barrier Analysis, Why Tree, and Safety Culture Analysis were used to identify causal factors and determine corrective actions. A Management Oversight and Risk Tree (MORT) analysis was used to evaluate completeness of the analysis and to identify or confirm areas of causality. Many Organizational and Programmatic elements, as well as technical, procedural, structural, and leadership factors, contributed to the Loss of Offsite Power event in January 2012. The IIT has developed corrective actions that are responsive to the Organizational and Programmatic contributors to the Loss of Offsite Power event. The collective data evaluated by the IIT demonstrates that Wolf Creek did not recognize the risk/consequence of having a vendor perform work, in accordance with vendor procedures and processes, without an established verification method for ensuring work quality, thus resulting in an undetected human performance error. Depending upon a third party to provide quality work was evident in the 2008 LOOP, as well as in the 2009 LOOP. The Hardware Failure Analysis identified the most likely failure mechanism of the 345-60 Generator Output breaker to be particulate in the breaker internal environment that set up a path for a flashover across the insulator surface. However, due to the extensive damage incurred by the breaker pole, it is not possible to definitively identify the exact internal location and source of the particulate material. Different theories exist regarding the source of the contamination and internal location of the initial arc. CAUSES
- 10 - The identified Root and Contributing Causes:
Root Cause 1 Statement:
Internal particulate contamination caused the failure of the Generator Output breaker 345-60.
Root Cause 2 Statement:
Wolf Creek did not recognize the risk/consequence of having a vendor perform work, in accordance with vendor procedures and processes, without a Wolf Creek approved verification method for ensuring work quality, resulting in an undetected human performance error.
Contributing Cause 1 Statement: Wolf Creek personnel, at all levels, failed to implement and enforce the company's accountability model, primarily with respect to procedural use and adherence.
- 11 -
ATTACHMENT 2
Wolf Creek Nuclear Station Switchyard diagram and its connection to the unit (1 sheet)