ML13109A234
ML13109A234 | |
Person / Time | |
---|---|
Site: | Wolf Creek |
Issue date: | 07/12/2013 |
From: | Roy Mathew Division of Engineering |
To: | Chernoff H NRC/NRR/DIRS/IOEB |
Sahay P | |
References | |
TAC ME8004 | |
Download: ML13109A234 (12) | |
Text
July 12, 2013
MEMORANDUM TO: Harold K. Chernoff, Chief Operating Experience Branch Division of Inspection and Regional Support Office of Nuclear Reactor Regulation FROM: Roy K. Mathew, Acting Chief
/RA/ Electrical Engineering Branch Division of Engineering Office of Nuclear Reactor Regulation
SUBJECT:
SAFETY EVALUATION REGARDING WOLF CREEK GENERATING STATION - AUGMENTED INSPECTION - LOSS OF OFFSITE POWER AND NOTIFICATION OF UNUSUAL EVENT, ISSUE FOR RESOLUTION 2012-004. (TAC NO. ME8004)
The Electrical Engineering Branch has reviewed the Issue For Resolution 2012-004, A Wolf Creek Generating Station- Augmented Inspection - Loss Of Offsite Power and Notification Of Unusual Event.
@ Enclosed is our safety evaluation. This completes our review and evaluation efforts for TAC NO. ME8004.
Enclosure:
As stated
CONTACTS: Prem P. Sahay, NRR/DE/EEEB (301) 415-8439
Peter J. Kang, NRR/DE/EEEB (301) 415-6800
July 11, 2013
MEMORANDUM TO: Harold K. Chernoff, Chief Operating Experience Branch Division of Inspection and Regional Support Office of Nuclear Reactor Regulation
FROM: Roy K. Mathew, Acting Chief
/RA/ Electrical Engineering Branch Division of Engineering Office of Nuclear Reactor Regulation
SUBJECT:
SAFETY EVALUATION REGARDING WOLF CREEK GENERATING STATION - AUGMENTED INSPECTION - LOSS OF OFFSITE POWER AND NOTIFICATION OF UNUSUAL EVENT, ISSUE FOR RESOLUTION 2012-004. (TAC NO. ME8004)
The Electrical Engineering Branch has reviewed the Issue For Resolution 2012-004, A Wolf Creek Generating Station- Augmented Inspection - Loss Of Offsite Power and Notification Of Unusual Event.
@ Enclosed is our safety evaluation. This completes our review and evaluation efforts for TAC NO. ME8004.
Enclosure:
As stated
CONTACTS: Prem P. Sahay, NRR/DE/EEEB (301) 415-8439
Peter J. Kang, NRR/DE/EEEB (301) 415-6800
DISTRIBUTION: JRobles FIssa RidsNrrDeEeeb NRR_DE_DPR ADAMS ACCESSION NO: ML13109A234 OFFICE DE/EEEB DE/EEEB DE/EEEB (A)BC NAME PSahay PKang RMathew DATE 7/11/2013 7/11/2013 7/12/2013 OFFICIAL RECORD COPY
ENCLOSURE EVALUATION REGARDING WOLF CREEK GENERATING STATION AUGMENTED INSPECTION - LOSS OF OFFSITE POWER AND NOTIFICATION OF UNUSUAL EVENT, ISSUE FOR RESOLUTION 2012-004 (TAC NO. ME8004)
1.0 PURPOSE The purpose of this Issue For Resolution (IFR) is to evaluate the root cause of the LOOP event, the safety significance, determine similar event s at other plants, and provide recommendations for issuing generic communications or other regulatory actions. This IFR applies to all operating
and new reactors.
2.0 BACKGROUND
According to the License Event Report (LER) 2012-001-00, Wolf Creek unit 1 experienced an
unplanned automatic shutdown from full power operation and a coincident loss of offsite power (LOOP) for greater than 15 minutes, resulting in the declaration of a Notification of Unusual
Event (NUE) on January 13, 2012 at 14:03 Central Standard Time (CST). A Wolf Creek Site
Watch Operator reported to the Control Room that the main generator output breaker 345-60
had experienced a failure with substantial visual damage. At 1537 hours0.0178 days <br />0.427 hours <br />0.00254 weeks <br />5.848285e-4 months <br /> the Wolf Creek
Turbine Building Watch notified the Control Room of a dropped flag from a Differential relay
actuation (487/T1B) on a panel MA104D for the Startup Transformer. This failure and faulted
condition of the breaker 345-60 resulted in actuation of protective relays for the East Bus
Primary lockout, East Bus Secondary lockout, and 345-60 breaker failure. These relay
actuations caused the opening of switchyard breakers and isolated the east switchyard bus, resulting in a turbine trip and subsequent reactor trip. To counter this loss of power, a "fast bus
transfer" occurred that realigned the power supply for the non-safety buses to the Startup Transformer (XMR01). As soon as the fast bus transfer was complete, the 487/T1 'B' Phase
Startup Transformer differential relay actuated per design. The Startup Transformer Differential
relay (487/T) sends inputs to the 286/T1 lockout relay, which initiates de-energization of the
Startup Transformer. This protective sequence generates a signal to isolate the Startup Transformer power supply to ESF Transformer XN B02. Additionally, this Startup Transformer 286/T1 lockout relay feeds protection circuitry in the WCGS switchyard to open switchyard
breakers to isolate the West 345 kilo Volt (kV) bus and de-energize the Startup Transformer.
This sequence of events, which isolated and de-energized the WCGS switchyard East 345 kV
and West 345 kV buses, resulted in the loss of offsite AC electrical power to the unit. The
turbine trip initiated a reactor trip. All control rods inserted into the core and all reactor coolant
pumps tripped per design. The undervoltage relays on safety related 4.16 kV engineered safety
features (ESF) buses NB01 and NB02 actuated and initiated a loss of power DG start signal for
the automatic start of both emergency diesel generators (DGs). Both DGs started and supplied
power to the 4.16 kV ESF buses with loads sequenced onto the buses per design.
At 1700 CST offsite power was restored to the East 345 kV bus, Transformer XNB01, and the
NB01 bus. With the restoration of an operable offsite circuit, the Unusual Event was terminated
at 1709 CST. At 1721 CST the 'A' DG was placed in a standby condition. On February 13, 2012, at 2008 CST, an attempt was made to start the 'A' Reactor Coolant Pump (RCP) for troubleshooting. This start attempt resulted in the second loss of electrical
power to the Startup Transformer. The licensee's investigation revealed that the Startup
Transformer 487/T1 'B' phase Differential relay and the 286/T1 lockout relay were tripped as
well as the 94F West Bus Secondary Differential relay in the Switchyard, which had initiated the
automatic opening of Switchyard breakers resulting in de-energization of the switchyard West
345 kV bus.
In its Root Cause Report CR 47653 (IIT 2012-001), the licensee stated that the January 13, 2012 event was the third instance of LOOP experienced by Wolf Creek within past four years.
The first loss of offsite power occurred on April 7, 2008 (LER No. 4822008004), when one
offsite source (the Switchyard No. 7 Transformer) was out of service, and a human error
isolated the one remaining offsite source (the West Bus) from the Startup Transformer, resulting
in a loss of offsite power. The unit was in Mode 6 with the Reactor de-fueled at the time. The
second LOOP event occurred in August 19, 2009 (LER No. 4822009002), when both offsite
sources were lost due to a lightning strike on its transmission line. The unit was in Mode 1 both
times. The second LOOP was caused by offsite transmission grid events and conditions.
LOOP on January 13, 2013 was the third occurrence for the Wolf Creek Plant.
3.0 ROOT CAUSE EVALUATION The Wolf Creek Switchyard and its connections to the unit are shown in the Attachment 2. The
Wolf Creek unit is connected to the Switchyard through main generator breaker to provide
generator output to the Switchyard and transmission grid and through the main Step up
Transformer to provide offsite power to the plant equipment.
In its root cause report, the licensee stated the following:
- 1. The most likely failure mechanism of the Generator Output breaker phase/pole 'C' was the internal particulate or contaminations in the breaker internal environment that set up
a path for a flashover across the insulator surface. However, due to the extensive
damage incurred by the breaker pole, it is not possible to definitively identify the exact
internal location and source of the particulate material.
- 2. The initial testing and troubleshooting was performed on the Startup Transformer and associated protective relaying. Troubleshooting did not reveal any deficiencies that
could have caused the LOOP on January 13, 2012. During the original troubleshooting
effort the junction boxes that contained the Current Transformer (CT) junction boxes were not visually inspected. Standard/regular electrical tests individually verified XMR01
differential CT circuits for each phase as not having any short or ground. However, the
standard electrical test configuration would not identify shorts between unused CT taps
of different CTs or phases. Due to the lack of any monitoring at the time of the event the
maintenance crew was not aware that the 'B' phase Differential relay circuit was not
generating the expected current, and phase shift. The point-to-point meggering between
phase checks that eventually detected the CT short on the unused taps, is not a typical
industry standard test used and thus was not originally implemented. As a result, the Startup Transformer was reenergized from off-site power at 1207 hours0.014 days <br />0.335 hours <br />0.002 weeks <br />4.592635e-4 months <br /> on February 3, 2012, with the later identified CT short still existing. 3. After the February 13, 2012, Startup Transformer (XMR01) differential trip (Failure Mode), troubleshooting on February 16, 2012, identified a short on XMR01 high side CT
wires that feeds the differential protection scheme. During the troubleshooting, megger
testing indicated a phase-to-phase short circuit between two unused high side CT taps (CT Wire W2 on 'A' phase and W10 on 'B' phase). Subsequent field inspections found
two missing insulation sleeves on these CT wires, originating from last refueling outage
vendor work on the transformer.
The following sequence of events is based on the excerpt from the licensee
=s root cause report of the Wolf Creek event (Attachment 2). The sequence of events occurred in the following order (only important events are listed):
Sequence of Events Date - 01/13/2012 Event Time Summary of Event Event Description Wolf Creek Unit was operating at 100% power prior to this event.
14:02:54.740 Wolf Creek East Bus Diff (T = 0) Switchyard supervisory point indicating the trip of an east buss differential relay. 14:02:54.753 Main Generator Switchyard BKR No.
345-60 is OPEN This point is a direct indication of breaker 345-60 opening. It
indicates breaker open when all
three phases of breaker 345-60
are open. 14:02:54.768 Wolf Creek 345-50 open 14:02:54.919 Startup Transformer lockout 286/T1 has actuated. 487/T1 "B" phase Start-
up Transformer Differential target
flagged (T)
Indication that Startup Transformer lockout 286/T1 has
actuated. 487/T1 "B" phase Start-
up Transformer Differential target
flagged (T). 14:02:54.922 Startup transformer trip. Indication in switchyard that the 286/T1 lockout has actuated in
the plant. Switchyard cross trip relay 94P/W is in parallel with the
plants 286/T1 lockout and they
were both actuated by the S/U
transformer 487/T1 differential
relay. The 94P/W relay actuates
the west bus lockout 86 P/W. 14:02:54.930 Wolf Creek West Bus Diff (Lockout) Loss of Offsite Power occurred. 14:02:54.994 REAC MAIN TRIP BKR B is TRIPD Reactor tripped. 14:02:55.006 REAC MAIN TRIP BKR A is TRIPD Reactor tripped. 14:03:02.656 DG NE02 FDR BKR is CLOSD Emergency Diesel Generator started as designed. 14:03:02.953 DG NE01 FDR BKR is CLOSD Emergency Diesel Generator started as designed. The Wolf Creek root cause investigation (CR 47653 (IIT 2012-001)) identified two key issues that resulted in the LOOP event, namely, failure to identify presence of the particulate or
contaminations on the main generator output breaker phase/pole "C" and correct it and failure to
identify two un-insulated terminal wires in contact (i.e. shorted) on the CT for Start up
Transformer 487 differential relays. Both cases appear to be due to human errors.
The licensee's Hardware Failure Analysis identified the most likely failure mechanism of the
345-60 Generator Output breaker to be particulate contamination in the breaker internal
environment that set up a path for a flashover across the insulator surface. An effective
maintenance program by the licensee would have detected and resolved the presence of
particulates and could have avoided main generator output breaker failure.
On February 16, 2012, at approximately 2008 hours0.0232 days <br />0.558 hours <br />0.00332 weeks <br />7.64044e-4 months <br />, with the plant in Mode 5, the 'A' Reactor
Coolant Pump (RCP) was started. This start attempt resulted in another protective lockout of
the Startup Transformer. This transformer lockout resulted in the isolation of the Wolf Creek
substation West bus and the loss of electrical power to the non-safety related 13.8 kV buses as
well as 'B' Train safety-related 4.16 kV bus NB02.
While performing current transformer (CT) testing on interfaces with the Differential Relay protection zone, the licensee identified an
electrical short in the phase 'A' and 'B' CT terminal wiring block. Two wires that were not
insulated (as they should have been) were in contact, causing an electrical short and that
resulted in Step up transformer differential relay actuation and tripping of transformer as
designed. This CT terminal junction block was last worked during refueling outage RF18 by the
vendor company, and had previously tested satisfac torily at the completion of work during RF18. Wolf Creek did not recognize the risk/consequence of having a vendor perform work in
accordance with vendor procedures and processes, without established verification methods for
ensuring work quality, thus resulting in an undetected human performance error.
Had the licensee established and implemented effective verification methods for ensuring work
quality, an electrical short in the phase 'A' and 'B' CT terminal wiring block could have been
identified and corrected and startup transformer differential relay actuation and startup
transformer trip and LOOP could have been avoided.
The NRC dispatched an augmented inspection team to review the facts surrounding the event (NRC Inspection Report 05000482/2012008 dated March 4, 2012, Agencywide Documents
Access and Management System (ADAMS) A ccession Number ML12095A414). The team identified several unresolved items (URIs) requiring follow-up inspection. Specifically, one URI
involved reviewing the root cause analysis of the main generator output breaker fault when it is
completed and follow up actions, and the another URI involved reviewing the root cause
analysis of the startup transformer fault when it is completed and follow up actions.
4.0 SAFETY SIGNIFICANCE
The event is considered safety significant, as it resulted in unit trip, LOOP, and starting of the
safety related equipment due to human performance errors.
In addition, the January 13, 2012, event was the second LOOP event caused by human error
within four years. The first loss of offsite power occurred on April 7, 2008, when one offsite source (the Switchyard No. 7 Transformer) was out of service, and a human error isolated the one remaining offsite source (the West Bus) from the Startup Transformer, resulting in a loss of
offsite power. The unit was in Mode 6 with the Reactor de-fueled at the time.
Had the licensee established and implemented an effective maintenance program including
independent verification requirements for ensuring work quality and identification and
elimination of human errors based on the lessons learned from April 7, 2008 plant event, Wolf
Creek could have avoided another LOOP and unplanned plant trip on January 13, 2012.
5.0 RECENT LOOP EVENTS IN OTHER NUCLEAR PLANTS
- Oyster Creek experienced LOOP on July 23, 2012, due to an electrical fault on an offsite transmission line (outside plant). (ref. PNO-I-12-005)
LER No. 2012-001-01)
- Point Beach Nuclear plant Unit 1 declared NOUE following LOOP on November 27, 2011, at 2:38 AM (CST). (ref. LER No. 2011-001-00)
- Browns Ferry Nuclear Plant Units 1, 2, and 3 declared NOUE due to LOOP on April 27, 2011, at 6:21 PM (EDT). (ref. LER No. 2011-001-00)
- Surry Power Station Units 1 and 2 declared NOUE due to LOOP from a tornado on April 16, 2011, at 7:48 PM. (ref.LER No. 2011-001-00)
- North Anna Units 1 and 2 experienced LOOP during seismic event on August 23, 2011. (ref. LER No. 2011-003-00)
- Calvert Cliff Unit 1 and 2 experienced Partial LOOP on February 18, 2010, at 8:24 AM (ref. 2010-001-01).
- Point Beach Nuclear plant Unit 1 declared NOUE following LOOP on January 15, 2008. (ref. LER No. 2008-001-00)
Two out of a total of eight LOOP events (including Wolf Creek January 13, 2012 event) in year
2011 and year 2012 were due to natural causes such as tornado and earthquakes. From above
list of LOOP events, it is evident that overall LOOP events in years 2011 and 2012 have significantly increased (including Wolf Creek January 13, 2012 event) compare to year 2008, 2009 (Wolf Creek August 19, 2009 event) and 2010 and supports NRC concern on upward
trend of the LOOP events in recent two years.
The staff reviewed the NRC Technical Review Group (TRG) Reports for the Year 2011 dated
November 11, 2012 and 2012 (Agencywide Do cuments Access and Management System (ADAMS) Number ML12306A261) and finds that both reports highlighted the issue on the high number of LOOP/inoperable offsite power sources due to the equipment failure and/or human
errors, consistent with the staff's finding on this IFR.
The staff also reviewed following active generic communications on the plant events resulting in
LOOP:
- 1. Information Notice (IN) 2006-18, "Significant Loss of Safety-Related Electrical Power at Forsmark Unit 1 in Sweden."
- 2. IN 2007-14, "Loss of Offsite Power and Duel-Unit Trip at Catawba Nuclear Generating Station."
- 3. IN 2009-10, "Transformer Failures - Recent Operating Experience."
6.0 IMPACT OF FREQUENCY OF LOOP ON STATION BLACKOUT COPING DURATION Regulatory Guide 1.155, August 1988, "Station Blackout" states, "Based on Title 10 of
the Code if Federal Regulation , Part 50.63 (10 CFR 50.63), all licensees and applicants are required to assess the capability of their plants to maintain adequate core cooling
and appropriate containment integrity during a station blackout and to have procedures
to cope with such an event. This guide presents a method acceptable to the NRC staff
for determining the specified duration for which a plant should be able to withstand a
station blackout in accordance with these requirements. The application of this method
results in selecting a minimum acceptable station blackout duration capability from 2 to
16 hours1.851852e-4 days <br />0.00444 hours <br />2.645503e-5 weeks <br />6.088e-6 months <br />, depending on a comparison of the plant's characteristics with those factors
that have been identified as significantly affecting the risk from station blackout. These
factors include redundancy of the onsite emergency ac power system (i.e., the number
of diesel generators available for decay heat removal minus the number needed for
decay heat removal), the reliability of onsite emergency ac power sources (e.g., diesel
generators), the frequency of LOOP, and the probable time to restore offsite power.
Licensees may propose durations different from those specified in this guide. The basis for alternative durations would be predicated on plant-specific factors relating to the
reliability of alternating current (AC) power systems such as those discussed in
Reference 2."
As discussed above, the frequency of LOOP is one of the factors affecting SBO coping duration. If the frequency of LOOP has changed since last analysis, the coping duration is no longer valid.
7.0 CONCLUSION
Based on the above, the staff concludes the following:
- 1. The Wolf Creek event on January 13, 2012, appears to be plant specific as staff review of the operating experience database did not find similar plant events occurring in other
nuclear plants. 2. The LOOP events in the US nuclear industry continued to rise for the year 2011 and 2012 compared to year 2008 and 2010 as evident from the LOOP events listed in Section 5.00.
- 3. Despite NRC concerns communicated to the nuclear industry in INs 2006-18, 2007-14 and 2009-10, number of LOOP events continue to increase in year 2011 and 2012.
8.0 RECOMMENDATIONS
- 1. This Issue for Resolution (IFR) Evaluation should be sent to all the Engineering Branch Chiefs/Division of Reactor Safety in the Regions including Region 2 Division
of Construction Inspection. The regional inspectors should refer to this IFR when
conducting inspections in maintenance and surveillance areas.
- 2. Although the January 13, 2012 plant events at Wolf Creek appear to be plant specific, and similar plant events have already been addressed in above INs, the staff finds that number of LOOP events occurring in the nuclear industry have suddenly increased in year 2011 and 2012. As such, the staff recommends issuing appropriate generic communication to address its concern on overall increase in LOOP events in recent years.
9.0 ATTACHMENTS
(1) Excerpt from Wolf Creek Root Cause Evaluation Report CR 47653 (IIT 2012-001) Executive Summary
(2) Wolf Creek Nuclear Station Switchyard diagram and its connection to the unit (1 sheet)
Principal Contributor: Prem P. Sahay
ATTACHMENT 1
Excerpt from Wolf Creek Root Cause Ev aluation Report CR 47653 (IIT 2012-001) Executive Summary On January 13, 2012 Wolf Creek experienced a loss of offsite power, the third such instance experienced
by Wolf Creek within the past four years. The first loss of offsite power (April 7, 2008) occurred when one
offsite source (the Switchyard No. 7 Transformer) was out of service, and a human error isolated the one
remaining offsite source (the West Bus) from the Startup Transformer, resulting in a loss of offsite power.
The unit was in Mode 6 with the Reactor de-fueled at the time. The second and third LOOP events
occurred when both offsite sources were lost due to two failure mechanisms that occurred at
approximately the same time (i.e., within cycles of one another). The unit was in Mode 1 both times. The second LOOP was caused by offsite transmission gr id events and conditions. The first and third LOOPs initiated at Wolf Creek each involved a human error.
Wolf Creek management established an Incident Investigation Team (IIT) on January 13, 2012 to per form a Root Cause Analysis (RCA) of the unplanned station shutdown coincident with a loss of offsite power (LOOP). The event occurred at approximately 1403 hours0.0162 days <br />0.39 hours <br />0.00232 weeks <br />5.338415e-4 months <br /> on January 13, 2012 and a Notification Of Unusual Event (NUE) was declared due to the
LOOP lasting longer than 15 minutes. The IIT was c hartered by and reported to the Vice President -
Strategic Projects. An event focus meeting was hel d on January 14, 2012 to commence the investigation.
Additionally, on 2/13/12 at approximately 2008 hours0.0232 days <br />0.558 hours <br />0.00332 weeks <br />7.64044e-4 months <br />, wit h the plant in Mode 5 the 'A' Reactor Coolant Pump (RCP) was started. This start attempt resulted in another protective lockout of the Startup
Transformer. This transformer lockout resulted in the isolation of the Wolf Creek substation West bus and
the loss of electrical power to the non-safety related 13.8 kV buses as well as 'B' Train safety-related 4.16 kV bus NB02. Personnel interviews, reviews of perti nent documents, computer modeling of protective relay schemes of the applicable plant electrical bus es, hardware failure analysis of the breaker 345-60 and potential transformer (PT) 113-1, and extensive hardw are inspection and testing were used to generate the bases related to the causes and consequences a ssociated with the causal factors identified. Two Event and Causal Factor (E&CF) Charts (ove rview and details) were developed using a detailed Sequence of Events (SOE) report. Fault Tree Analys is, Evidence and Action Matrix, Change Analysis, Barrier Analysis, Why Tree, and Safety Culture A nalysis were used to identify causal factors and determine corrective actions. A Management Oversight and Risk Tree (MORT) analysis was used to
evaluate completeness of the analysis and to identify or confirm areas of causality. Many Organizational and Programmatic elements, as well as technical, procedural, structural, and leadership factors, contributed to the Loss of Offsite Power event in January 2012. The IIT has developed corrective actions that are responsive to the Organizational and Programmatic contributors to the Loss of Offsite Power
event. The collective data evaluated by the IIT demonstrates that Wolf Creek did not recognize the
risk/consequence of having a vendor perform work, in accordance with vendor procedures and processes, without an established verification method for ensuri ng work quality, thus resulting in an undetected human performance error. Depending upon a third party to provide quality work was evident in the 2008
LOOP, as well as in the 2009 LOOP. The Hardware Fa ilure Analysis identified the most likely failure mechanism of the 345-60 Generator Output breaker to be particulate in the breaker internal environment
that set up a path for a flashover across the insulator surface. However, due to the extensive damage
incurred by the breaker pole, it is not possible to def initively identify the exact internal location and source of the particulate material. Different theories exist regarding the source of the contamination and internal location of the initial arc.
CAUSES The identified Root and Contributing Causes:
Root Cause 1 Statement:
Internal particulate contamination caused the failure of the Generator Output breaker 345-60.
Root Cause 2 Statement:
Wolf Creek did not recognize the risk/consequence of having a vendor perform work, in accordance with vendor procedures and processes, without a Wolf Creek approved verification method for ensuring work quality, resulting in an undetected human performance error.
Contributing Cause 1 Statement: Wolf Creek personnel, at all levels, failed to implement and enforce the company's accountability model, primarily with respect to procedural use and adherence.
ATTACHMENT 2
Wolf Creek Nuclear Station Switchyard diagram and its connection to the unit (1 sheet)