IR 05000325/2004011

From kanterella
Jump to navigation Jump to search
IR 05000325-04-011, on 08/23/2004 - 08/27/2004, Brunswick Steam Electric Plant, Unit 1; Special Inspection Report
ML042710443
Person / Time
Site: Brunswick Duke Energy icon.png
Issue date: 09/26/2004
From: Mccree V
Division Reactor Projects II
To: Gannon C
Carolina Power & Light Co
References
IR-04-011
Download: ML042710443 (29)


Text

September 26, 2004

SUBJECT:

BRUNSWICK STEAM ELECTRIC PLANT - NRC SPECIAL INSPECTION REPORT 05000325/2004011

Dear Mr. Gannon:

On August 27, 2004, the Nuclear Regulatory Commission (NRC) completed a special inspection at your Brunswick Unit 1 facility. The enclosed report documents the inspection findings which were discussed on August 27, 2004, with you and other members of your staff, and on September 22, 2004, with you.

On August 14, 2004, at 12:59 p.m., Brunswick Unit 1 was manually tripped from approximately 67% power, following a loss of offsite power to a switchyard bus. The loss of offsite power resulted from the combination of a fault on an offsite power line and the failure of a switchyard breaker, which was removed from service. Your post-trip review determined that a hinged-armature relay in switchgear associated with an emergency diesel generator had been mechanically bound by a modified dust cover. This problem prevented several loads from shedding before the emergency diesel generator connected to the affected bus. Your staff removed the mispositioned dust cover. To determine the extent-of-condition, your staff conducted two reviews to look for similar problems, and imposed a mode restraint while completing a formal operability evaluation of conditions observed during those reviews. In addition, several related operability issues were resolved before restarting the unit on August 18.

On August 20, 2004, a Special Inspection Team (SIT) was established by NRC Region II management using the guidance contained in Management Directive 8.3, NRC Incident Investigation Program. The SIT was chartered to inspect and assess the circumstances associated with the loss of offsite power to Unit 1 that occurred on August 14; operator responses to that event; the subsequent discovery of a failed hinged-armature relay; your related operability evaluation and extent-of-condition reviews; and your initial onsite declaration of an Alert. The inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license. The inspectors reviewed selected procedures and records, conducted field walkdowns, observed activities, and interviewed personnel.

CP&L

Based on the results of this inspection, no findings of significance were identified. We have determined that the cause of the loss of offsite power event was well understood, your staff conducted a comprehensive review of the event, and identified problems were appropriately placed into your corrective active program.

In accordance with 10 CFR 2.390 of the NRCs "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publically Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web-site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA by L. Wert for/

Victor M. McCree, Director Division of Reactor Projects Docket No.: 50-325 License No.: DPR-71

Enclosure:

Inspection Report 05000325/2004011 Attachments: 1. Supplemental Information 2. Brunswick Unit 1 Special Inspection Team Charter

REGION II==

Docket No:

50-325 License No:

DPR-71 Report No:

05000325/2004011 Licensee:

Carolina Power & Light (CP&L)

Facility:

Brunswick Steam Electric Plant, Unit 1 Location:

8470 River Road SE Southport, NC 28461 Dates:

August 23 - 27, 2004 Inspectors:

R. Hagar, Senior Resident Inspector - Robinson R. Monk, Resident Inspector - Browns Ferry J. Austin, Resident Inspector Approved by:

Victor M. McCree, Director Division of Reactor Projects

Enclosure

SUMMARY OF FINDINGS

IR 05000325/2004011; 08/23/2004 - 08/27/2004; Brunswick Steam Electric Plant, Unit 1;

Special Inspection Report.

The special inspection team (SIT) inspection was conducted by a senior resident inspector and two resident inspectors. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 3, dated July 2000. No findings were identified during this inspection.

  • The cause of the loss-of-offsite power was the internal failure of a switchyard breaker as it responded to a line fault outside the units switchyard: that failure led to loss of power on the 1B bus, which caused, in turn, a loss of power to the unit 1 startup transformer, and the loss of both recirculation pumps.

The special inspection team noted that changes could be made in the switchyard configuration and some switchyard equipment which could significantly reduce the units vulnerability to similar events in the future. The licensee initiated efforts to review and evaluate enhancements.

  • A load-shed permissive HGA relay on emergency bus 1 failed when the relay dust cover prevented the relay armature from actuating. Several loads were not shed from the bus before emergency diesel generator (EDG)-1 picked up the loads on that bus. Upon identifying the relay problem, the licensee corrected the involved relay problem, completed an adequate operability determination of EDG-1 and also performed the Technical Specifications-required common-cause analysis of the other EDGs.
  • To verify that no other important HGA relays had mispositioned dust covers, the licensee examined a larger population of relays in other applications. The initial relay examination identified a number of conditions that needed to be corrected, however, none of those conditions prevented the proper operation of any relay. Because the initial examination had been completed using an informal methodology, the licensee had not developed documentation that was adequate to support an operability determination.

Some Operations personnel and management were not aware of how the identified relay conditions had been addressed. The licensee subsequently re-examined the subject relays, using a more formal and approved process. The re-examination was completed and the operability determination was formally documented prior to continuing the unit restart.

REPORT DETAILS

Initial Conditions On August 14, 2004, Brunswick Unit 2 was at full power and Unit 1 was at 67 percent power, following a dispatcher-directed downpower from full power. The Unit 1 reactor recirculation pumps were powered from the startup auxiliary transformer, which was aligned to receive power from switchyard bus 1B, while other major plant loads were powered from the unit auxiliary transformer, which received power directly from the main generator.

In addition, the licensee had declared a Notification of Unusual Event (NOUE), on August 14, due to a hurricane warning that was issued for the plant area. In the morning of August 14, the licensee partially staffed their Emergency Response Organization in anticipation of hurricane landfall.

Event Description On August 14, an undervoltage condition was sensed on the Unit 1 startup auxiliary transformer, resulting in an automatic start of emergency diesel generator (EDG)-1 and tripping of both reactor recirculation pumps. The operators then manually tripped the reactor and the main turbine. The other three EDGs started, and EDG-1 and -2 tied-in to their respective emergency buses, based on sensed undervoltage conditions on those buses. The operators reported that during the trip recovery, they did not have to start three pumps powered from emergency bus E-1, because those pumps were already running. The operators therefore questioned proper load shedding of bus E-1.

On August 15, Unit 1 entered mode 4. A review determined that bus E1 did not fully load shed before it was energized by EDG-1. Consequently, EDG-1 had immediately picked up the loads from several large pumps and a 480-volt bus upon closure of its output breaker. An examination of the emergency bus switchgear found that bus E1 had failed to shed load because a load-shed permissive relay had failed to actuate. The relay dust cover had been mispositioned such that the cover mechanically bound the relay armature in the closed position.

Following this discovery, EDG-1 was declared inoperable, and Action Request 134802 was initiated to address the operability of EDG-1. An investigation of the cause of the mispositioned relay was begun, but was not completed before the special inspection team completed its inspection.

On August 16, the licensee removed the mispositioned dust cover and returned the affected relay to service. After completing an analysis of common-cause failure possibilities and assessing the effects on EDG-1 of picking up loads upon closure of its output breaker, the licensee returned EDG-1 to operable status.

On August 17, the licensee completed an ad-hoc extent-of-condition review of other relays in the EDG circuitry, to determine whether any of those relays had also experienced mispositioned dust covers. They completed that review coincident with preparations for restarting the reactor; shortly after that review was completed, the reactor achieved criticality. Meanwhile, although the licensee determined that no other relays had been affected by mispositioned dust covers, questions related to the determination methodology and documentation details prompted the licensee to re-examine the subject relays on August 18, within the context of a formal operability evaluation of the EDG circuitry, and to implement a mode restraint to ensure that the operability evaluation was completed before the unit entered power operation. After the evaluation was completed on August 18, the EDG circuitry was considered fully operable, the mode restraint was lifted, and the unit subsequently entered mode 1 and was synchronized to the grid.

Special Inspection Team (SIT) Charter Based on the criteria specified in Management Directive 8.3, NRC Incident Investigation Program, and Inspection Procedure 71153, Event Follow-up, a special inspection was initiated in accordance with Inspection Procedure 93812, Special Inspection. The objectives of the inspection, described in the charter, are listed below and are addressed in the identified sections:

(1) determine the facts surrounding the failure of switchyard breaker 24B and how this failure ultimately resulted in a loss of offsite power to Unit 1 (Section 4OA3.1);
(2) review the cause of the EDG-1 emergency bus HGA relay failure (Section 4OA3.2.1);
(3) determine the adequacy of the licensees extent of condition review for the relay failure (Section 4OA3.2.2);
(4) assess the adequacy of the site evaluation, operations communications, and operators decision making, associated with the HGA relay problem prior to startup (Sections 4OA3.2 and 4OA3.2.2);
(5) review the adequacy of the surveillance procedure for testing the EDG load-shed permissive relays, especially as it relates to this event (Section 4OA3.3.3);
(6) review the failure of standby gas treatment train B and the subsequent repair and operability determination (Section 4OA3.3.1);
(7) review the licensee's decision making process on initially declaring the Alert and then subsequently changing the declaration to an Unusual Event (Section 4OA3.3.2); and
(8) review the adequacy of the operators response to the reactor scram and loss of offsite power (Sections 4OA3.2, 4OA3.2.1, and 4OA3.2.2).

OTHER ACTIVITIES

4OA3 Event Followup

.1 Loss of Offsite Power Issues (Objective 1)

a. Inspection Scope

The SIT inspectors interviewed licensee staff and reviewed selected site procedures and drawings to determine the facts surrounding the failure of switchyard breaker 24B and to understand how this failure ultimately resulted in a loss of offsite power to Unit 1. In addition, the inspectors evaluated the switchyard design against the requirements of General Design Criterion (GDC) 17, Electrical Power Systems.

b. Observations and Findings

On August 14, while the 230-kv Weatherspoon transmission line near the Brunswick plant was experiencing high winds from hurricane Charlie, an insulator supporting the B phase of that line mechanically failed (snapped in half). This failure allowed the line to fall and come into contact with the towers cross structure, causing a non-clearing phase-to-ground fault. In the Brunswick switchyard, the breakers that connected the Weatherspoon line to the switchyard (breakers 24A and 24B) sensed that fault and opened. After 15 seconds, breaker 24B closed, sensed the fault again, and re-opened.

After another 50 seconds, the breaker again closed, still sensed the fault, and re-opened to a lockout condition. To this point, the breakers response to the line fault was in accordance with its design. However, almost immediately after the breaker re-opened the second time, breaker 24B itself experienced an internal fault that allowed one phase of power to arc from its corona ring to the side of its tank. This fault was sensed by the breakers breaker failure relay, and that relay actuated to open all breakers connected to the B bus. This included opening the Unit 1 main generator output breaker to the B bus, the motor-operated disconnect between the startup auxiliary transformer and the B bus, and all other offsite lines to the B bus. Because the motor-operated disconnect between the startup auxiliary transformer and the A bus was already open, this meant that the startup auxiliary transformer was de-energized.

Because the startup auxiliary transformer was the power supply to the reactor recirculation pumps, this precipitated a loss of recirculation flow, which prompted the operators to manually scram the reactor. Scramming the reactor de-energized the unit auxiliary transformer, which in turn de-energized the boards supplying the emergency busses. At this point in time, even though switchyard bus A remained energized, no off-site power was being supplied to the on-site Unit 1 distribution system. Emergency Diesel Generators (EDGs) 1 and 2 started and re-energized their respective emergency buses.

The inspectors determined that the Brunswick switchyard design is in compliance with the requirements described in GDC 17. However, one requirement in GDC 17 is as follows:

Provisions shall be included to minimize the probability of losing electric power from any of the remaining supplies as a result of, or coincident with, the loss of power generated by the nuclear power unit, the loss of power from the transmission network, or the loss of power from the onsite electric power supplies.

With respect to this requirement, the inspectors observed that:

  • Plant loads in Unit 1 are distributed between the startup auxiliary transformer and the unit auxiliary transformer. In particular, the reactor recirculation pumps are powered from the startup auxiliary transformer, while other large plant loads are powered from the unit auxiliary transformer.
  • The startup auxiliary transformer is aligned to receive power from either switchyard bus A or switchyard bus B, while the unit auxiliary transformer is aligned to receive power from the Unit 1 main generator.
  • The alignment described above necessarily means that if a switchyard fault affects the bus to which the startup auxiliary transformer is aligned, that fault will remove power from the startup auxiliary transformer, which will in turn remove power from the reactor recirculation pumps and necessitate a reactor scram. With respect to the requirement above, this alignment means that if a switchyard fault affects the bus to which the startup auxiliary transformer is aligned, loss of power from the transmission network will necessarily lead to loss of power generated by the nuclear power unit.
  • This alignment described above also necessarily means that any fault that affects the startup auxiliary transformer itself will also necessitate a reactor scram. With respect to the requirement above, this alignment means that a loss of power from the startup auxiliary transformer (one onsite power source) will necessarily lead to loss of power generated by the nuclear power unit.

The inspectors noted that, although the switchyard design is in compliance with the requirements of GDC 17, the probability of losing electrical power (as described in the GDC requirements) would be reduced if the reactor recirculation pumps were not powered from the startup auxiliary transformer.

No findings of significance were identified.

.2 Operability / Extent-of-Condition Issues

.2.1 Operability of EDG-1 (Objectives 2 and 8)

a. Inspection Scope

The SIT inspectors interviewed licensee staff, reviewed site procedures and drawings and inspected subject relays installed in the emergency bus switchgear to assess the licensee cause determination of the EDG-1 emergency bus HGA relay failure and assess the adequacy of the site evaluation, operations communications, and operators decision-making associated with the HGA relay problem prior to startup.

b. Observations and Findings

The loss of power from the startup auxiliary transformer directly started all four EDGs.

This start signal occurred simultaneously with the loss of power to the recirculation pumps. Hence, the EDGs were starting while the plant operators were assessing plant conditions and tripping Unit 1. Tripping Unit 1 caused the loss of the unit auxiliary transformer, which de-energized balance-of-plant busses 1C and 1D which, in turn, de-energized emergency busses E2 and E1. Emergency bus E2 load shed and was energized per design by EDG-2. However, emergency bus E1 did not fully load shed before it was energized by EDG-1. Consequently, EDG-1 immediately picked up approximately 1300 kw of load upon closure of its output breaker.

Observations by the control-room operators of equipment that continued to run following the event led the licensee to examine the emergency bus switchgear. That examination found that bus E1 had failed to shed load because in the EDG-1 circuitry, relay CL-B (a hinged-armature load-shed permissive relay) had failed to actuate because its dust cover had been mispositioned such that the cover mechanically bound the relay armature in the closed position. In the logic circuit of the EDG output breaker, this locked in a signal indicating that load shedding for the emergency bus was complete, when in fact load shed had not fully occurred. The licensee subsequently removed the mispositioned cover from relay CL-B, successfully tested the relay, and restored the relay to operability.

The dust cover that was found mispositioned had been substantially modified, apparently during initial plant construction. The original dust cover was a solid, heavy-gauge plastic box approximately 5" x 3" x 3" deep, with one open side, normally mounted vertically on an electrical cabinet inner wall; the dust cover was configured such that when the dust cover is properly installed, it completely enclosed the relay, so the relay was not exposed in any way to the environment around the dust cover. The original dust cover included detents that were machined into two opposing inside surfaces of the cover, and the cover was held in place by the engagement of those detents with heavy spring clips that were mounted on the relay body; when the detents on the cover aligned with the clips on the body, the cover snapped into place. When the original cover was installed on a relay, tension in the clips held the cover firmly against the relay body, such that when a relay was vertically mounted, the cover could not be rotated from side-to-side, or from top-to-bottom. However, the dust cover that had been mispositioned was not as originally fabricated, but had been modified, in that parts of the top and bottom surfaces, and parts of the sides adjacent to those surfaces had been removed, apparently to allow clearance for the relatively heavy-gauge wires that were connected to terminals on the relay body. (Those wires were apparently of a larger diameter than the wires for which the relay was designed). The dust covers on many other similar relays had also been modified somewhat, apparently during original plant construction. Most of the covers were not modified as significantly as the mispositioned cover (less material was removed).

The surfaces that were removed from the affected dust cover included approximately 3/4 of the top and bottom surfaces of the cover and the top and bottom thirds of both sides, above and below the center third which contained the detents; all that remained of the dust cover sides were strips approximately 1" wide, located in the approximate center of each side, coincident with the detents. Besides providing clearance for wires, removal of this material had the effect of greatly reducing the amount of dust cover material that was in contact with the relay body. With these modifications, and because of the flexibility that is inherent in the connection between the clips on the relay body and the detents in the dust cover, after the modified dust cover was snapped into place on a relay, the cover was not held firmly against the relay body, but could be rotated relatively easily from top-to-bottom, around the intersection of the clips with the detents. When the dust cover was rotated up, the top of the cover moved closer to the relay, while the bottom of the cover moved farther away. Furthermore, a cover that is mispositioned by rotation would tend to stay mispositioned, because of the tension between the clips and the detents. Apparently, the dust cover that was found mispositioned had been rotated up far enough that the top of the cover moved close enough to the relay to contact the hinged armature of the relay, and forced closed the contacts on that armature. The licensees investigation into how the relay came to be mispositioned has not yet been completed.

Upon discovering the failed relay in the EDG-1 circuitry, the licensee declared EDG-1 inoperable. In this circumstance, Technical Specification (TS) 3.8.1, [Alternating Current] Sources - Operating, required the licensee to determine within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> that the other EDGs were not also inoperable due to common-cause failures. Consequently, Operations management asked Engineering to complete a common-cause failure analysis of the other EDGs. Because EDG-1 had picked up load immediately upon closure of its output breaker, Operations management also asked Engineering to determine whether energizing the bus with loads already connected had affected the operability of EDG-1.

In response to Operations requests:

  • Engineering completed the common-cause analysis by first developing an engineering change package that allowed removal of any dust covers that were mispositioned in a manner similar to the dust cover on CL-B, and then inspecting seven relays in the EDG circuits that were similar to CL-B. That inspection found no additional mispositioned covers.
  • Engineering analyzed the effect of energizing the bus with loads already connected by first identifying the loads that had failed to shed. Using archived data from the Unit 1 scram and operator statements, Engineering determined that the 2C and 1B conventional service water pumps, the 1A nuclear service water pump, the 1A control rod drive pump and the 480v E5 bus had been connected when the EDG-1 output breaker closed. Engineering then determined that these loads totaled 1273 Kw. By comparing that load to previously analyzed load scenarios, both analytical and field test, which indicated that the EDGs were capable of instantly connecting to as much as 1773 Kw without damage to either the engine or generator, Engineering concluded that EDG-1 was operable.

Engineering also began a Significant Adverse Condition investigation of the cause(s) of the mispositioned relay, in accordance with Assignment 1 of AR 134802134802 At the time of this special inspection, that investigation was not complete.

The timeline below describes the major events related to the EDG-1 operability issues:

Date Time Event Description 8/14 1258 An undervoltage condition was sensed on the startup auxiliary transformer, resulting in an automatic start of EDG-1 and tripping of both reactor recirculation pumps.

Operators manually tripped the reactor and the main turbine.

2245 Operators reported that during the trip recovery, they did not have to start the 1A control rod drive pump, 1B conventional service water pump, and 2C conventional service water pump, because those pumps were already running. Operators therefore questioned proper load shedding of Bus E-1.

8/15 1640 Unit 1 entered mode 4.

1700 Discussions with the on-shift operating crew confirmed that no operator action had been taken to start the 1A control rod drive pump, 1B conventional service water pump, and 2C conventional service water pump.

2155 During their post-trip review, the licensee found EDG-1 relay CL-B with its dust cover contacting the relay armature, making up the relays contacts.

2156 The licensee declared EDG-1 inoperable.

2215 The Shift Outage Manager (SOM) discussed with the Engineering Outage Manager (EOM) the identified condition of the relay CL-B on emergency bus E1, and informed the EOM of the TS 3.8.1 condition D requirement for Engineering to document a common-cause analysis of Busses E2, E3, and E4 within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> (before 8/16, 2159). The SOM also directed the EOM to have Engineering evaluate the scenario in which the EDG output breaker closed on a sensed undervoltage prior to the individual load breakers tripping on individual load undervoltage, resulting in the load breakers not tripping open (E bus voltage recovered prior to load undervoltage condition being achieved.)

2300 The licensee initiated AR 134802134802 task 10 addressed the impact on EDG-1 operability 8/16 1804 Outage management realized that the operability determination in AR 134802134802task 10 will be needed to evaluate if EDG-1 was damaged by the transient experienced when it closed on the bus with loads running.

2151 The Operations Shift Supervisor approved assignment 10 of AR 1348021348022223 Operations returned EDG-1 to operable status, and canceled LCOs A2-04-0826 and T1-04-0828.

After reviewing the control wiring diagrams and the events described above, the inspectors considered that the licensees explanation of the cause of the load shed failure on bus E1 was reasonable, and based on the extent of the inspectors review, Assignment 10 of AR 134802134802adequately covered both the common-cause analysis and the operability determination associated with EDG-1.

No findings of significance were identified.

.2.2 Extent-of-Condition Reviews (Objectives 3, 4, and 8)

a. Inspection Scope

The SIT inspectors reviewed the circumstances associated with the licensees activities on August 16, when the licensee examined 208 relays which were similar to the CL-B relay that had been mechanically bound and determined that none of those relays were inoperable. The inspectors also reviewed similar activities on August 17, when the licensee re-examined the same relays. The inspectors focused specifically on the information used to determine operability of the four emergency busses.

b. Observations and Findings

After the licensee repaired and successfully tested the CL-B relay whose cover had mechanically bound the relay armature, and with the awareness that many other relays in the plant were similar to CL-B, the licensee decided to verify that none of those relays had experienced similar failures before restarting the unit. Consequently, a designated dayshift Relay Project Manager (RPM) began developing an action plan for completing that verification. The dayshift RPM turned over to the nightshift RPM a plan which included a one-sentence description of the required inspections, with the note that no work order was required, and the instructions that their personnel should coordinate with the Outage Control Center (OCC), inspect one bus at a time, note conditions, and resolve any identified deficiencies prior to inspecting the next bus. In the nightshift briefing on August 16, the nightshift RPM told the oncoming crew that their scope of work for that shift would include inspections of relays similar to CL-B, to verify that none were mechanically bound like CL-B. In that meeting, no one was actually dispatched to complete those inspections.

Several hours into the shift, and before OCC managers had actually dispatched anyone to complete the subject inspections, OCC managers received a call from an instrumentation & controls (I&C) technician who had begun the inspections, and wanted guidance on how to resolve some identified deficiencies. The OCC managers asked an experienced system engineer to assist the I&C technician. In a subsequent meeting, that system engineer, the I&C technician, the nightshift RPM, and possibly other personnel met to decide how to inspect the relays, and how to disposition any identified deficiencies.

Shortly before midnight on August 16, the system engineer and the I&C technician (the relay verification team) determined that 208 relays similar to CL-B were located in the switchgear of the four emergency diesel generators. The relay verification team obtained permission from the OCC to verify the identified relays, and then began verifying those relays as agreed in their earlier meeting. Two aspects of their verification method are noteworthy:

  • When the relay verification team first observed a relay, they wrote down the conditions they observed that were not what they expected.
  • If the relay verification team suspected that any condition(s) could have possibly caused the relay cover to mechanically bind the relay, they simulated the observed conditions on an identical relay and cover that they had withdrawn from plant stores, and physically tested the armature of their hand-held relay, to determine whether the observed conditions impacted the relays armature.

By early morning on August 17, the relay verification team had examined all 208 relays, and had determined by the method described above that no relay was mechanically bound. The team transcribed their handwritten notes into a list of observed deficiencies that should be corrected, and e-mailed that list to the dayshift RPM and several other personnel, along with a one-sentence description of their testing methodology and their conclusion that no relays had conditions that would prevent the relays from performing their functions. Besides that list and the subject e-mail note, the relay verification team developed no other documentation of their activities. After receiving the information sent by the relay verification team, OCC management told the nightshift Operations Shift Superintendent (SS) only that the relay verifications had been completed, and that no operability issues were identified. That summary was turned over to the oncoming dayshift SS.

During the dayshift on August 17, while the unit was operating in operational mode 2 and while the licensee was preparing the unit to enter mode 1, the resident inspectors reviewed the list of conditions associated with the relays that had been verified during the preceding nightshift. With respect to the noted deficiencies, the resident inspectors questioned several dayshift managers about the verification method with respect to dynamic effects (due to seismic action, breaker actuation in adjacent cabinets, etc.) and the extent to which Operations had been involved in the verifications. Upon realizing that the dayshift managers did not have satisfactory answers for the resident inspectors questions, the Operations Manager decided that satisfactory answers should be developed before the unit was placed in mode 1. The Operations Manager therefore asked Engineering to complete a formal operability assessment in accordance with Procedure 0OI-01.08, Control of Equipment and System Status, and initiated a mode restraint via LCO A-1-04-0841, to ensure that the operability assessment would be completed before the unit was allowed to enter mode 1.

In response to this request and within AR 134802134802 Engineering initiated assignment number 16, Evaluate Operability Based on HGA Relay Inspection Results of Bus E1, E2, E3, & E4. Engineering and I&C personnel subsequently re-inspected the 208 relays, and documented not only their inspection methodology, but also:

  • the criteria and considerations used during their inspections;
  • as-found descriptions of all relays;
  • for those relays which were found with covers not fully engaged, their evaluation of the potential for seismic or other vibration events to cause those covers to become dislodged; and
  • their conclusion that none of the observed conditions could adversely affect the operation of any relay.

In particular, these personnel noted that:

  • Most of the relays were found with both sides of their covers mounted firmly against their relay bases.
  • Most of the remaining relays were found with at least one side of the relay covers engaged firmly enough to preclude any movement of the covers.
  • For those relays whose covers were not found firmly against their bases, none was mounted above components that could be damaged by the cover being released and falling away from the base.

Engineering management approved the documentation and submitted it to the SS, who reviewed and approved the documentation. Following that approval, the SS withdrew LCO A-1-04-0841, and the operating crew continued preparations for and subsequently entered operational mode 1.

The timeline below describes the events associated with the relay extent-of-condition issues.

Date Time Event Description 8/14 An undervoltage condition was sensed on the startup auxiliary transformer, resulting in an automatic start of EDG-1 and tripping of both reactor recirculation pumps.

Operators manually tripped the reactor and the main turbine.

1547 The licensee re-energized the startup auxiliary transformer from switchyard bus 1A.

8/15 2155 During their post-trip review, the licensee found EDG-1 relay CL-B with its dust cover contacting the relay armature, making up the relays contacts.

2215 The Shift Outage Manager (SOM) discussed with Engineering Outage Manager (EOM) the identified condition of the relay CL-B on emergency bus E1.

2300 The licensee initiated AR 134802134802on Bus E1 CL-B relay; task 16 addressed the operability of the emergency busses 8/16 0930 The SOM named a recovery manager for the relay issue, including extent of condition.

8/17 0255 I&C / Engineering completed extent-of-condition inspections of relays in Busses E-1, E-2, E-3, and E-4, and found no conditions that required evaluation for operability impact. They compiled a list of identified deficiencies.

0452 Operators placed the Reactor Mode Switch in Startup; Unit 1 entered Mode 2.

0456 I&C / Engineering sent to Outage management a list of E-bus relay dust covers with identified deficiencies, plus their conclusion that none of the deficiencies represented operability concerns.

0620 Operators began withdrawing control rods to take the reactor critical.

<1200 After reviewing the list of E-bus relay dust covers with identified deficiencies, the NRC Resident Inspectors identified concerns related to the verification method with respect to dynamic effects and the extent to which Operations had been involved in the verifications.

1146 Operators declared the reactor was critical.

1230 With the Licensing Supervisor and Operations Manager, NRC Resident Inspectors discussed their concerns related to operability of E-bus relays with covers with identified deficiencies.

1749 The Operations Manager initiated a formal operability assessment of the E-bus relays in accordance with Procedure 0OI-01.08. As directed by the Operations Manager, the Shift Superintendent implemented an associated mode-change restraint via LCO A1-04-0841.

2010 Engineering / I&C personnel obtained permission to inspect the E-bus relays to support resolution of the 0OI-01.08 operability determination.

8/18 0310 I&C / Engineering completed extent-of-condition inspections of Busses E-1, E-2, E-3, and E-4, and reported no operability concerns.

0821 Engineering completed the 0OI-01.08 operability assessment for E bus relays, documented it under AR 134802134802task 16, and submitted it to Operations for final approval.

0846 The Operations Shift Supervisor approved AR 134802134802task 16.

0854 Operations exited the LCO A1-04-0841 mode restraint.

0859 Operators placed the Reactor Mode Switch in Run; Unit 1 entered Mode 1.

After reviewing the circumstances and activities described above, the SIT inspectors made the following observations:

  • The licensee made a reasonable decision to verify that no other relays were mechanically bound like relay CL-B had been, and to complete that verification before restarting the unit.
  • The action plan begun on August 16 and further developed during the next shift was an informal, uncontrolled document. The level of detail in that plan was too general to provide useful guidance to working-level personnel.
  • The licensee had no established formal process for completing and adequately documenting the type of extent-of-condition review envisioned, within the short time period desired. Consequently, the personnel who actually inspected the subject relays determined in real time what to do, how to do it, and how to document what was done; the subject relays were inspected outside of any established process.

However, the inspectors noted that the personnel who inspected the subject relays only examined the relays visually, and did not actually touch or otherwise manipulate the relays. The inspectors also noted that no regulation requires the licensee to use an established process to look at plant equipment.

  • The assessment methodology developed and used by the relay verification team before the early morning of August 17 included an innovative and effective way to assess the operational impact of observed conditions. However, by recording the observed conditions without thoroughly documenting their assessment methodology, the relay verification team in effect highlighted the observed conditions without providing enough information to enable others to readily understand the operational impact of those conditions. The resulting questions raised by the resident inspectors prompted the licensees second review.
  • The licensees second extent-of-condition review, completed before the early morning of August 18, was completed within established processes, and with an awareness of the concerns that the review documentation must address.

Consequently, the second review was well-documented, and provided a firm basis for Operations to proceed with restarting the unit.

No findings of significance were identified.

.3 Other Concerns

.3.1 Failure of Standby Gas Treatment Train B (Objective 6)

a. Inspection Scope

The SIT inspectors reviewed the details surrounding the failure of the B Train of standby gas treatment to initiate following the Unit 1 scram. The inspectors interviewed maintenance operators and engineering personnel to determine why the unit failed to start and what corrective actions had been performed.

b. Observations and Findings

The inspectors determined that the licensee had performed extensive troubleshooting into the cause of the B train standby gas failure. The licensee determined that cause of the failure to start was actuation of the B trains fire protection lockout circuitry, but was unable to identify a specific component failure. Consequently, the licensee checked and replaced equipment with the potential to affect the lockout circuitry, and declared the system operable.

No findings of significance were identified.

.3.2 Incorrect Emergency Action Level Determination (Objective 7)

a. Inspection Scope

The SIT inspectors examined the circumstances associated with the licensees initial determination that the events of August 14 satisfied the Brunswick Emergency Action Level (EAL) criteria for an Alert, and their later soon-after determination that those events satisfied the criteria for a Notification of Unusual Event, to determine which of the licensees determinations was correct, and to understand the reasons why the licensees other determination was incorrect. The inspectors interviewed involved personnel and reviewed related records.

b. Observations and Findings

Shortly after the control-room operators tripped the reactor on August 14, the Shift Technical Advisor (STA) used Procedure 0PEP-02.1, Initial Emergency Actions, to determine that an Alert was the appropriate EAL classification. The STA initiated an emergency notification form (ENF) that showed that classification, and submitted the form to the SS for approval. The SS signed the form that the STA had prepared, and directed the STA to make the appropriate notifications. The STA announced the Alert over the site public address system, and then directed the site Security organization to activate the Emergency Notification (BEN) system to send the appropriate code for Alert classification to beepers carried by selected site personnel. The Security organization activated the BEN system with that code at 1315:55. Meanwhile, the STA directed the Emergency Communicator to transmit the signed ENF to offsite officials and agencies.

Sometime after the STA gave that direction but before the Emergency Communicator had completed any offsite notifications, the STA realized that his initial determination had been incorrect. The STA then notified the SS of the error, stopped the Emergency Communicator from transmitting the incorrect ENF, initiated a new ENF with an NOUE classification, obtained SS approval of the new form, directed Security to send the code for NOUE to site beepers, and directed the Emergency Communicator to transmit the new ENF to offsite officials and agencies. The Security organization subsequently terminated the Alert classification, and transmitted the NOUE code.

The inspectors noted that Procedure 0PEP-02.1 assigns to the SS the responsibility for determining the appropriate EAL classification, and that, because the SS signed the ENF form prepared by the STA, the SS retained that responsibility.

Interviews with the STA and the SS indicated that:

  • To determine the EAL classification that was appropriate, the STA first attempted to find the EAL logic charts that he had been accustomed to using. However, he was not able to find those logic charts, because the charts were not stored in their normal location.
  • The STA was not accustomed to using the EAL logic descriptions found in Procedure 0PEP-02.1, Initial Emergency Actions, because in training he had been allowed to use either the charts or the procedure, and he had consistently chosen to use the charts. Consequently, the STA had never practiced using the procedure.
  • When the STA presented for his approval the ENF that declared an Alert, the SS did not independently determine the appropriate EAL classification, and did not closely review the STAs determination, but signed the ENF after only a cursory review.

This was because the SSs attention was focused on the response of the plant and the performance of the control-room operators, and not on the EAL classification.

  • The STA realized that his initial determination of an Alert had been incorrect when he recognized that the scenario the plant was experiencing was similar to scenarios to which the operating crews had been exposed in training sessions, and when he remembered that during the training scenarios, the appropriate EAL classification had been NOUE. This occurred after the STA directed the Emergency Communicator to transmit the first signed ENF to offsite officials and agencies, but before the Emergency Communicator had completed any offsite notifications. After this realization, the STA re-examined the EAL logic descriptions he had used, and realized that his initial determination had been incorrect.

Using the EAL logic descriptions in Procedure 0PEP-02.1, the inspectors determined that NOUE was the correct classification for the event. In addition, the inspectors noted that the licensee initiated AR 134854134854 Wrong EAL Classification, to address the incorrect initial classification.

No findings of significance were identified.

.3.3 Adequacy of Surveillance Testing of EDG Load-Shed Permissive Relays (Objective 5)

a. Inspection Scope

To determine whether and how surveillance testing of EDG load-shed permissive relays had contributed to the events of August 14 and to verify compliance with related surveillance requirements in the site TS, the SIT inspectors reviewed the TS and the licensees procedures for testing those relays. The inspectors also interviewed Maintenance and Engineering personnel.

b. Observations and Findings

TS 3.8.1 addresses the operability of alternating-current electrical power sources, and includes fourteen associated surveillance requirements (SRs). Among those SRs, TS SR 3.8.1.14 is the only SR that mentions load shedding. The only requirement imposed by that SR with respect to load shedding is for the licensee to verify, on actual or simulated loss of offsite power signal in conjunction with an actual or simulated emergency-core-cooling-system initiation signal, load shedding from emergency buses, on a 24-month frequency. In particular, TS 3.8.1 does not require logic system functional testing of the load-shedding function. Therefore, no TS SR imposed logic system functional testing on the EDG load-shed permissive relays.

Procedure 0MST-DG 11R, DG 1 Loading Test, Rev. 1, is used to satisfy TS SR 3.8.1.1.14. The inspectors found that the licensee had successfully completed that procedure within the previous 24 months.

The inspectors therefore determined that surveillance testing of EDG load-shed permissive relays did not contribute to the events of August 14, and that the licensee was in compliance with related surveillance requirements in the site TS.

No findings of significance were identified.

4OA6 Meetings, Including Exit

The SIT inspectors presented the inspection results to Mr. C. Gannon and other members of licensee management at the conclusion of the inspection on August 27, 2004. The inspectors discussed the inspection results further with Mr. Gannon on September 22, 2004.

The inspectors confirmed with the licensee that material examined during the inspection was not proprietary.

ATTACHMENTS:

1.

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee personnel

G. Atkinson, Supervisor - Emergency Preparedness
J. Bass, Lead Engineer
L. Beller, Supervisor - Licensing/Regulatory Programs
J. Cannon, System Engineer
T. Cleary, Director Site Operations
B. Cowen, Lead Engineer
P. Dubroillet, Shift Superintendent
C. Dunsmore, Shift Technical Advisor
T. Hackler, Shift Superintendent
D. Hardin, Shift Superintendent
D. Hinds, Plant General Manager
D. Jester, Superintendent of Shift Operations
K. Karp, Supervisor, Transmission Substation Maintenance
L. Kufel, System Engineer
E. ONeil, Manager - Site Support Services
E. Quidley, Manager - Outage and Scheduling
P. Smith, Recovery Manager for Relay Issues
N. Smith, Recovery Manager for Relay Issues
S. Tabor, Lead Engineer - Technical Support
L. Troutman, Lead Engineer
M. Turkal, Lead Engineer
M. Williams, Manager - Operations

NRC personnel

G. DiPaolo, Senior Resident Inspector, Brunswick
P. Fredrickson, Chief, Branch 4, Division of Reactor Projects
L. Wert, Deputy Director, Division of Reactor Projects

LIST OF ITEMS

OPENED, CLOSED AND DISCUSSED

None

LIST OF DOCUMENTS REVIEWED