IR 05000325/2007010

IR 05000325-07-010, on 7/9/2007 - 8/30/2007; Brunswick Steam Electric Plant; Supplemental Inspection IP 95002 for Degraded Mitigating Systems Cornerstone; a Violation Identified in the Area of Maintenance Practices

ML072840504
Person / Time
Site:	Brunswick
Issue date:	10/11/2007
From:	Casto C Division Reactor Projects II
To:	Scarola J Carolina Power & Light Co
References
IR-07-010
Download: ML072840504 (31)

Text

ber 11, 2007

SUBJECT:

BRUNSWICK STEAM ELECTRIC PLANT - NRC SUPPLEMENTAL INSPECTION REPORT NO. 05000325/2007010

Dear Mr. Scarola:

On August 30, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection at your Brunswick Facility, Unit 1. The enclosed report documents the inspection results, which were discussed on July 20 and August 30, 2007, with you and other members of your staff.

The purpose of this supplemental inspection, performed in accordance with Inspection Procedure 95002, was to examine your problem identification, root cause evaluation, extent-of-condition and extent-of-cause determinations, and corrective actions associated with multiple issues that placed Unit 1 in the Degraded Cornerstone Column of the NRC Reactor Oversight Process Action Matrix. This inspection also included an independent NRC review of the extent-of-condition and extent-of-cause for these same issues and an assessment of whether any safety culture component caused or significantly contributed to the issues. The issues, which were in the Mitigating Systems Cornerstone, included the performance indicator for the Mitigating Systems Performance Index, Emergency AC Power System, which crossed the threshold from very low risk significance (Green) to low to moderate risk significance (White) in the second quarter of 2006, and an additional White inspection finding related to the failure of the emergency diesel generator #1 on November 2, 2006. The Mitigating Systems Performance Index was previously evaluated in Supplemental Inspection Report 05000325,324/2006008. Consequently, this inspection focused on the emergency diesel generator failure and changes to the Emergency AC Power System performance indicator after completion of the previous supplemental inspection.

The inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.

The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

The report documents one NRC-identified finding of very low significance (Green) that was also a violation of NRC requirements. However, because of the very low significance and because it was entered into your corrective action program, the NRC is treating the finding as a non-cited

CP&L 2 violation consistent with Section VI.A.1 of the NRC Enforcement Policy. If you contest any non-cited violation in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN.:

Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at Brunswick.

Given your performance in addressing the failure of the emergency diesel generator following an automatic start, the White finding associated with that issue will only be considered in assessing plant performance for a total of four quarters in accordance with the guidance in IMC 0305, Operating Reactor Assessment Program. In addition, the White performance indicator for the Emergency Power systems will be allowed to run its due course. The NRC will review the implementation of your corrective actions during a future inspection.

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRCs document system (ADAMS). Adams is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA by Harold Christensen Acting For/

Charles A. Casto, Director Division of Reactor Projects Docket No.: 50-325 License No.: DPR-71

Enclosure:

Inspection Report 05000325/2007010 w/Attachment: Supplemental Information

REGION II==

Docket No: 50-325 License No: DPR-71 Report No: 05000325/2007010 Licensee: Carolina Power and Light (CP&L)

Facility: Brunswick Steam Electric Plant, Unit 1 Location: 8470 River Road SE Southport, NC 28461 Dates: July 9, 2007- August 30, 2007 Inspectors: S. Freeman, Senior Resident Inspector, Sequoyah (Team Lead)

P. OBryan, Senior Resident Inspector, Shearon Harris S. Stewart, Senior Resident Inspector, Turkey Point D. Jones, Senior Reactor Inspector, Region II H. Gepford, Senior Health Physicist, Region II G. Gardner, Reactor Inspector, Region II (Training)

M. Coursey, Reactor Inspector, Region II (Training)

Approved by: R. Musser, Chief Reactor Projects Branch 4 Division of Reactor Projects Enclosure

SUMMARY OF FINDINGS

IR 05000325/2007010; 7/9/2007 - 8/30/2007; Brunswick Steam Electric Plant; Supplemental

Inspection IP 95002 for Degraded Mitigating Systems Cornerstone; A violation was identified in the area of maintenance practices.

This inspection was conducted by three senior resident inspectors, one senior reactor inspector, and one senior health physicist. One Green finding of very low safety significance was identified during this inspection and was classified as a non-cited violation (NCV). The significance of most findings is identified by the color (Green, White, Yellow, Red) using IMC 0609, Significance Determination Process (SDP). Findings for which the SDP does not apply may be Green or be assigned a severity level after NRC management review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 4, dated December, 2006.

Cornerstone: Mitigating Systems

The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess the licensees evaluation associated with multiple issues in the mitigating systems cornerstone for Unit 1. Due to a number of failures between July 1, 2003 and June 30, 2006, the performance indicator for the Mitigating Systems Performance Index, Emergency AC Power Systems, increased to the point where it was characterized as low to moderate significance (White). In addition, between July 1, 2006 and June 30, 2007, three more failures occurred in the Emergency AC Power System. One of these involved the failure of an emergency diesel generator following automatic start and was also characterized as

White.

During this supplemental inspection, performed in accordance with Inspection Procedure 95002, the inspection team determined that the licensee performed a comprehensive review of each failure and a common cause review of all failures collectively. The licensees collective evaluation, while acknowledging there was no common cause, identified that work practices and equipment performance were significant contributors to emergency diesel generator performance and that the use of trending codes for not applicable and undetermined causes could indicate a lack of thoroughness in cause evaluations. The licensee has relied on ongoing actions for previously identified human performance trends to address the work practice issue and has established an emergency diesel generator reliability improvement team to deal with equipment issues. To address the cause code concern, the licensee has established corrective actions to address the level of rigor to be used in cause evaluations.

The licensees evaluation of the White emergency diesel generator failure identified the primary root cause to be inadequate foreign material controls, which resulted in maintenance workers leaving a cleaning cloth in the crankcase. The licensee also identified that increased clearance between the main crankshaft and the diesel casing, which resulted in a poor bearing fit, contributed to the failure. The licensee has taken corrective actions to tighten foreign material controls and has initiated plans to replace the bearings on the affected emergency diesel generator with a larger size.

In addition to assessing the licensees evaluation, the inspection team performed an independent extent-of-condition and extent-of-cause review and a focused inspection of the site safety culture. Overall, the team concluded that the licensees cause and corrective actions were adequate, that adequate extent-of-condition and extent-of-cause was done, and that the safety culture did not contribute to the issues in question. However, the team did identify some missed opportunities to identify the degrading trend in emergency diesel generator performance earlier than actually occurred. The team also identified one potential weakness in the licensees handling of operating experience information.

Given the licensees acceptable performance in addressing the failure of the emergency diesel generator following an automatic start, the White finding associated with that issue will only be considered in assessing plant performance for a total of four quarters in accordance with the guidance in IMC 0305, Operating Reactor Assessment Program. In addition, the White performance indicator for the Emergency AC Power Systems will be allowed to run its due course. Implementation of the licensees corrective actions will be reviewed during a future inspection.

Cornerstone: Mitigating Systems

Green.

The team identified a Green non-cited violation (NCV) of 10 CFR 50.65 (maintenance rule) for failure to demonstrate that the performance or condition of structures, systems, or components is being effectively controlled through the performance of appropriate preventive maintenance. An inadequate maintenance rule evaluation was performed after an emergency diesel generator exceeded its maintenance rule (a)(2) performance criteria and, as a result, goal setting and monitoring was not performed as required by Paragraph (a)(1) of the maintenance rule.

This finding was more than minor because it was associated with the equipment performance attribute and affected the Mitigating Systems Cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The lack of proper attention by the maintenance rule program to the degraded performance of Emergency Diesel Generator 3 allowed degraded performance to continue for all emergency diesel generators. This finding was of very low safety significance because it was not a design or qualification deficiency, did not directly result in an actual loss of safety function for a system or train, and was not risk significant due to a seismic, fire, flooding, or severe weather initiating event. The cause of the finding directly involved the cross-cutting area of human performance, in the decision making component under the aspect of using conservative assumptions because the expert panel decided to keep Emergency Diesel Generator 3 under maintenance rule Paragraph (a)(2) without fully supporting that conclusion. The licensee made this decision even though other evidence indicated that preventive maintenance was not effectively controlling Emergency Diesel Generator 3 performance H.1(b).

Licensee-Identified Violations

None.

REPORT DETAILS

INSPECTION SCOPE

This supplemental inspection included four main objectives: 1) to assess the licensees cause evaluation associated with the two issues that led to a degraded cornerstone for mitigating systems, the White performance indicator for Mitigating Systems Performance Index (MSPI), Emergency Alternating Current (AC) Power Systems and the White finding for failure to identify and implement corrective actions on the emergency diesel generator (EDG) that failed in November 2006; 2) to assess the corrective actions for the same issues; 3) to conduct an independent NRC review of the extent-of-condition and extent-of-cause for these same issues and; 4) to conduct an assessment of whether any safety culture component caused or significantly contributed to the issues.

Because functional failures that contributed to the MSPI between July 1, 2003 and June 30, 2006 were previously inspected, this inspection focused on those additional inputs to the indicator since the last inspection. Specifically, the licensee experienced three functional failures of EDGs between July 1, 2006 and June 30, 2007. The effect of these failures was to keep the MSPI for Emergency AC Power systems on both units in the White band. One of these failures also involved an inspection finding that was in the White band. The three failures reviewed were as follows:

July 31, 2006. An engine jacket water temperature control valve failure caused a functional failure of EDG #1. The valve failed due to a malfunction of two sub-components within the valve, called power pills. The licensee addressed this failure in Nuclear Condition Report (NCR) 201700. They attributed the failure to a manufacturing defect, replaced the power pills, restored EDG #1 to service, and increased the power pill replacement frequency.

November 2, 2006. Following an automatic start in response to a loss of offsite power on Unit 2, EDG #1 tripped on low lube oil pressure trip while maintenance personnel were attempting to place a clean lube oil strainer in service. After further investigation the licensee determined that one of the main crankshaft bearings had also failed. The licensee addressed this failure in NCR 211236.

They identified that a cleaning towel had been left in the crankcase after maintenance and that caused lube oil strainer clogging and contributed, along with clearance problems, to the bearing failure. The licensee tightened foreign material exclusion (FME) controls and replaced the bearing. This incident was characterized as a Notice of Violation (White) for Unit 1 and NCV (Green) for Unit 2 (Report 05000325,324/2007009).

February 19, 2007. A control circuit relay failure caused a functional failure of EDG #2. After the EDG was started during a surveillance test, the failed relay caused the EDG to trip due to a low lubricating oil trip protective signal. The licensee addressed this failure in NCR 223012. They attributed the failure to random equipment failure and replaced all relays that could impact operability with less than 10 years of service life.

The licensee evaluated the cumulative effect of these three failures, along with those that occurred between July 1, 2003 and June 30, 2006, in a common cause evaluation done in Action Request (AR) 232815. The team reviewed that evaluation, reviewed the licensees actions associated with the three most recent events, and conducted interviews with licensee personnel to ensure that the collective and individual root and contributing causes were identified and understood and that appropriate corrective actions were initiated. Throughout the report enclosure, the functional failures will be referred to by their numerical designators listed above.

To conduct the extent of condition and extent of cause review, the team independently inspected the licensees FME and operating experience (OE) programs along with maintenance practices and operational decision making on the high pressure coolant injection (HPCI), reactor core isolation cooling (RCIC), EDGs, and direct current (DC)

Power Systems. The team also conducted a focused safety culture review. The individual scope of inspection to meet these objectives is contained in that section.

EVALUATION OF INSPECTION REQUIREMENTS 02.01 Problem Identification a.

Determination of who identified the issues and under what conditions

(1) Licensee Evaluation The licensee determined that all three of the most recent functional failures were self-revealing. Failure number 1 was revealed when an alarm alerted operators to an abnormal rise in lubricating oil temperature. Failure numbers 2 and 3 were revealed when the respective EDGs tripped after engine starts.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

b.

Determination of how long the issues existed, and prior opportunities for identification

(1) Licensee Evaluation The licensee concluded that Failure 1 occurred sometime between July 3 and July 31, 2006 because EDG #1 successfully passed a surveillance run on July 3, 2006. The licensee concluded that Failure 3 occurred sometime between January 22 and February 19, 2007, because EDG #2 successfully passed a surveillance run on January 22, 2007.

The licensee concluded that the cleaning towel, which led to Failure 2, had been left in the EDG #1 crankcase on October 27, 2006 and that the clearance problems that contributed to the bearing failure had most likely been present since 1992 when the bearing was installed. For both of these items, the licensee concluded that there were prior opportunities to identify the condition. The EDG #1 lube oil strainers experienced high differential pressure three times during post maintenance testing on October 27, 2006, and again prior to trip on November 2, 2006. Each time, the licensee identified that fibers were present in the strainers but attributed them to residue from engine cleaning activities instead of the cleaning cloth itself. In 1992 the same crankshaft bearing failed for similar reasons as the failure on November 2, 2006. At that time the licensees investigation did not discover the root cause of the failure and therefore did not correct the condition that led to the failure on November 2, 2006.

(2) NRC Assessment No significant concerns were identified with the licensees assessment of the individual failures. However, the team determined that additional questions remained concerning the licensees approach to identifying a degrading trend in EDG performance. After the sixth MSPI failure in August 2005, the licensee initiated AR 166409 to follow up on the failures and determine any areas for additional effort. This AR identified that there was a degrading trend in EDG performance. However, instead of an NCR, which would have required a root cause investigation and corrective actions to prevent recurrence, the licensee issued AR 166409 as a self-assessment. Later, in November 2005, the licensee changed it to a bench marking report.

In addition to this use of a bench marking report, the team determined that weaknesses in the corrective action program (CAP) constituted a missed opportunity to correct preventive maintenance and human performance issues that affected the availability and reliability of the EDGs. The team reviewed several ARs that documented the licensees previous failures to take adequate corrective actions to address weaknesses in these two areas. This review showed that the inadequacy of previous corrective actions was identified by the corporate and site nuclear assurance sections (NAS). The team noted that while new ARs were initiated to address the recurring preventive maintenance and human performance issues, no evaluation(s) or corrective action(s)addressed the failure of the CAP to effectively deal with the previously identified deficiencies. The licensee did have open corrective active actions to address preventive maintenance and human performance issues that were originally identified in 2003 and 2004. The licensees ARs related to these issues are listed below.

AR 102339 (2003), WANO Peer Review Shortfalls in Preventive Maintenance AR 129173 (2004), Maintenance Adverse Trend in Human Performance AR 135289 (2004), PES Issue for Plant Performance AR 169120 (2005), NAS Weakness in Equipment Reliability Process AR 206696 (2006), 2006 INPO Midcycle - Gap in Preventive Maintenance AR 204325 (2006), NAS Assessment - Conduct of Maintenance These items will remain unresolved pending further inspection of the impact they had on the continuing problems with the EDGs. This will include licensee handling of both AR 166409 and the previously identified preventive maintenance and human performance problems. This matter is identified as Part 1 of URI 05000325, 324/2007010-01, Handling of Diesel Generator Problems by the CAP.

c.

Determination of the plant-specific risk consequences (as applicable) and compliance concerns associated with the issues both individually and collectively

(1) Licensee Evaluation Since Failure 2 was associated with an NRC finding, the risk associated with this failure was quantitatively evaluated as a change in core damage frequency of 1.3E-6. The licensee did a qualitative risk evaluation for Failures 1 and 3. The potential time exposure for Failure 1 was 28 days, making the risk potentially more than minimal.

However, the risk was mitigated by the ability of the operator to manually adjust the temperature control valve, as was demonstrated during the event on July 31, 2007. The potential time exposure for Failure 3 was also 28 days, but there were no mitigating actions available to operators. The licensee evaluated the risk for this failure as potentially more than minimal. The licensee also evaluated the cumulative risk for overall EDG performance. Since none of the three individual events represented a condition outside the plants design basis (failure of a single EDG), no actual loss of power to an emergency bus occurred, and the failures did not represent a common mode failure concern, the licensee considered the impact of the three additional EDG failures were not more than minimal.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

02.02 Root Cause, Extent-of-Condition, and Extent of Cause Evaluation a.

Determination that systematic methods were used to identify root causes and contributing causes

(1) Licensee Evaluation The licensee used a variety of methods to identify the root causes of the failures, including laboratory analyses, fault tree analyses, support/refute matrixes, and independent industry consultants.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

b.

Determination that the level of detail of the root cause evaluation was commensurate with the significance of the issues

(1) Licensee Evaluation Based on laboratory analyses, independent organization/contractor review, and management review, the licensee concluded that the root cause analyses were performed at the appropriate level of detail.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

c. Determination that the root cause evaluation considered prior occurrences of the issues and knowledge of prior operating experience

(1) Licensee Evaluation The licensees evaluations found that there were previous failures similar to the three failures discussed in this report. For Failure 1, a temperature control valve similarly failed in 1999 on EDG #3 and industry operational experience indicated that similar failures occurred at least three times at other nuclear plants (Shoreham Station in 1986, Wolf Creek in 1992, and Seabrook in 2004). As discussed earlier, a bearing failure similar to Failure 2 occurred in 1992. For Failure 3, there were several examples of EDG system relay failures.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

d. Determination that the root cause evaluation addressed extent-of-condition and extent-of-cause of the issues

(1) Licensee Evaluation For all three of the failures, the licensee performed an evaluation of the extent of condition and the extent of cause. In each case, all four of the EDGs were evaluated for the existence of a common cause failure potential and the extent of the failing condition.

For Failure 1, the power pills failed due to a manufacturing defect, which led to the loss of the power pill filler material. For Failure 2, poor FME practices led to the cleaning cloth being left in the EDG #1 crankcase and the failed bearing was attributed to loss of crush caused by a combination of issues including a larger than normal bore size for the EDG #1 bearings. Failure 3 was caused by the failed relay. Laboratory analysis did not reveal the exact cause of the relay failure.

The licensee also reviewed priority 1 ARs associated with the EDGs that did not impact the MSPI and priority 2 ARs associated with the EDGs. These were used to help determine the contributors to the MSPI decline. For two of the main contributors; work practices and equipment performance, the licensee determined that NCRs were already in place to deal with the issue. For the use of cause codes the licensee initiated NCR 236415 to evaluate the high use of L cause codes (inadequate or indeterminate cause).

In reviewing cause codes, the licensee looked at other safety related systems to identify any common elements that would warrant additional action. This list included the HPCI, RCIC, residual heat removal (RHR), and Service Water Systems.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

02.03 Corrective Actions a.

Determination that appropriate corrective actions were specified for each root or contributing cause

(1) Licensee Evaluation Causes and corrective actions for the three failures included:

Failure 1: The licensee considered this a random failure due to a manufacturing defect because the power pills failed three years after installation. The licensee reduced the replacement frequency of the power pills from once per six years to once per four years in order to increase reliability. This was within the original equipment manufacturers recommended replacement frequency.

Failure 2: The licensee identified the cause the EDG #1 trip after auto start as inadequate FME controls resulting in the cleaning cloth being left in the crankcase.

They addressed this by increasing the rigor of FME control and the degree of cleanliness in the EDG crankcases. The licensee identified the cause of the EDG #1 crankcase bearing failure as increased clearance between the crankshaft and the casing that resulted in less than adequate bearing fit (lower initial bearing crush) and greater operational stresses on the bearing. The licensee also indicated that the presence of the cleaning towel contributed to the timing of the failure. They addressed these causes by replacing the failed bearing and scheduling replacement of the crankshaft bearings on EDG #1 with larger bearings.

Failure 3: The licensee considered this a random failure because the failed relay had not reached its end of life and because they could not identify a specific failure contributor.

The licensee replaced all of the applicable EDG relays (Allen Bradley type 700) which could affect EDG operability and that were greater than ten years old. To increase reliability of relays in general, the licensee planned future replacements on a periodic basis and implemented a monitoring plan.

As part of their common cause analysis in AR 232815, the licensee identified two significant contributors to EDG performance: work practices and equipment performance, and that a high use of L cause codes (those for not applicable or undetermined) could indicate a lack of thoroughness. To address the work practice cause the licensee relied on ongoing actions to address site human performance problems. These included ongoing work to address a maintenance human performance trend (NCR 129173), an operations human performance trend (NCR 228956), an emergent site wide trend (NCR 227583), and a fleet wide trend in human performance (NCR 234828). To address the equipment performance cause the licensee relied on actions from an existing EDG Reliability Improvement Team (AR 230789) and preventive maintenance optimization plans initiated in 2006 (NCR 206696). To address the use of L cause codes the licensee initiated NCR 236415. This NCR contained corrective actions to address the level of rigor to be used in cause investigations.

(2) NRC Assessment No significant concerns were identified with the licensees assessment of the individual failures. However, in attempting to verify that all specified corrective actions for improving EDG performance were properly tracked in the CAP, the team found the actions to be contained in many ARs, that most had not been implemented, and that many were scheduled out several years. This item will remain unresolved pending further verification of proper corrective action tracking in the CAP and further inspection of corrective action implementation. This matter is identified as Part 2 of URI 05000325, 324/2007010-01, Handling of Diesel Generator Problems by the CAP.

b. Determination that corrective actions were prioritized with consideration for risk significance and regulatory compliance

(1) Licensee Evaluation The licensee has completed the corrective actions for Failures 1 and 3 and part of the actions for Failure 2. They have modified FME control procedures to increase the control of the EDG crankcases during maintenance. The licensee had scheduled, but not implemented, the corrective action to replace the crankshaft bearings on EDG #1 with larger bearings to ensure adequate bearing crush. The licensee reasoned that this action could be delayed due to the long time-based degradation of the bearings. The bearing failure occurred approximately 15 years after installation. The new bearings are scheduled to be replaced in 2009.

The priority for implementing corrective actions to address EDG reliability was based on studies done by the EDG reliability improvement team using the MSPI, maintenance rule, and system health reports.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

c. A schedule has been established for implementing and completing the corrective actions

(1) Licensee Evaluation The licensee had scheduled the final corrective action for Failure 2, as discussed in the previous section, for completion in 2009. They had scheduled corrective actions for human performance to be complete in December 2007 as part of a fleet-wide initiative to improve human performance at all Progress Energy nuclear plants. As discussed in Section 02.03a above, not all of the remaining actions for EDG reliability had been scheduled.

(2) NRC Assessment No significant concerns were identified with the licensees schedule.

d.

Determination that quantitative or qualitative measures of success were established for determining the effectiveness of the corrective actions to prevent recurrence

(1) Licensee Evaluation The licensee considered Failure 1 to be a random equipment failure, and therefore assigned it a priority two investigation under the CAP. The licensee tracked effectiveness for these actions in equipment reliability programs (e.g. maintenance rule program). For Failures 2 and 3, the licensee assigned priority one investigations under the CAP, with effectiveness reviews planned accordingly three to six months after all corrective actions are complete.

(2) NRC Assessment No significant concerns were identified with the licensees assessment.

02.04 Independent Assessment of Extent of Condition and Extent of Cause a.

Foreign Material Exclusion Program

(1) Inspection Scope The team performed an independent review of the licensees FME program to assess the validity of the licensee's conclusions regarding the extent to which FME controls have caused similar issues in the past and to verify that these controls have been properly implemented throughout the plant. The team reviewed the two implementing procedures for the FME program; MNT-NGGC-0007, Foreign Material Exclusion Program, Revision 6, and 0MMM-0055, Cleanliness and Flushing Requirements, Revision 8 and reviewed ARs related to FME control (ARs 194882, 2006678, and 211469) to determine if the licensee addressed the culture of inspect and remove that led to the FME control decisions made during maintenance on EDG #1 before it tripped on November 2, 2006. The team discussed the implementation, philosophy, and culture of the FME program with individuals at all levels of the site organization, including the plant general manager, the maintenance manager, maintenance supervisors, select job supervisors, mechanics, the FME coordinator, and a training supervisor to understand the basis for, and results of, the pre-job FME implemental review and single point accountability for high FME risk jobs. The team walked-down the Unit 1 spent fuel pool area and reactor building to evaluate FME controls for staged work and work-in-progress. The team also reviewed the FME control plans for work on the Unit 2 CST and the Unit 2 refueling outage (B218R1) turbine project for adequacy and appropriate controls, including prevention techniques. In addition, the team reviewed Maintenance Department CAP Roll-Up and Trend Analysis reports from April and June 2007 and supervisor observations completed during B218R1 related to FME. The team reviewed select maintenance training materials, newsletters, and information contained on the maintenance website to evaluate communication of FME control practices and culture.

(2) Findings and Assessment The licensee's extent-of-condition and extent-of-cause evaluation for the EDG #1 trip was limited to review of the remaining EDGs; corrective actions taken to address FME controls subsequent to the trip included the other three EDGs. The actions included increasing the FME job classification for EDG crankcases to high, revising procedures that addressed maintenance activities within EDG crankcases to reflect a high FME job classification, revising 0MMM-0055 to provide single point accountability during cleanliness inspections for high FME risk activities, and performing a stand down with EDG crew mechanics. Discussions with the licensee determined that the high FME job classification meant that a documented FME control plan was required, but did not require specific FME control methods (e.g. control logs); the job supervisor selected the control methods. The team therefore noted that, although the corrective actions should prevent recurrence in the short term, sensitivity to the event will likely decrease with time and/or personnel changes, which would bring into question sustainability. The team further concluded that the above listed actions were in place and added more restrictive controls than those that were in place at the time of the EDG trip, but that additional actions such as specifying appropriate control methods, which may have ensured sustainability, were not incorporated. However, as discussed in the following paragraphs, the licensee implemented a number of means to communicate the importance of FME controls to workers.

The team determined that concerns with the licensees FME program have existed since at least 2000 when an industry peer review identified two areas for improvement involving FME practices. Subsequent assessments in 2003 and 2006 also identified concerns with the FME program. In addition, a 2006 NAS assessment identified a weakness in implementation of the FME program related to a culture of removal rather than foreign material intrusion prevention. Because of these earlier assessment findings concerning the site's level of FME controls, and the fact that corrective actions for those items were already in place, additional corrective actions beyond the diesel lube oil system were deemed unnecessary by the licensee. Based on independent review of the corrective actions that were implemented subsequent to the assessment findings, discussions with individuals responsible for FME control, and an apparent trend of improving performance in FME controls identified through CAP trending and supervisor observations, the team had no concerns with the licensee decision to limit FME extent-of-condition and extent-of-cause evaluations to the EDGs.

The team determined that the licensee recently participated on an industry working group to develop improved guidance on debris mitigation and control methods for ensuring fuel integrity. The guidance, which focused on FME practices, was being evaluated by the licensee as an operating experience item under AR 232312 to determine what changes should be made to the site's FME program. The evaluation was to determine if the FME risk classification for specific systems, defined in procedure 0MMM-055, needed to be increased. The team determined that, although not included or tracked as part of the extent-of-condition and extent-of-cause evaluation of FME as the cause of the EDG 1 trip, the corrective actions being implemented via other CAP documents should ensure the FME program would not cause similar incidents in other parts of the plant.

The team reviewed selected ARs to evaluate the licensees efforts towards changing the FME culture from one of inspect and remove (retrieval) to foreign material prevention.

AR 194883 documented the deficiencies in implementation of the FME program identified by NAS in 2006. The corrective actions included an action to retrain on FME before B218R1; this was completed via inclusion of FME refresher training as part of the B218R1 outage training presentation; coverage of FME in the first quarter continuing training handout for individual reading; and a review of FME controls in individual training/briefing sessions held with each of the 19 maintenance crews. Based on review of training materials, the team confirmed a shift in culture from retrieval methods to prevention of foreign material intrusion.

AR 206678 documented FME concerns identified by an industry peer assessment in 2006. The corrective actions were to communicate prevention expectations to the FME committee and maintenance supervisors; reinforce the prevention expectations to the site; revise the B218R1 turbine project FME plans to focus on prevention; determine if FME tooling could be improved; and evaluate FME training for newly hired employees.

For each of these completed corrective actions, the focus on changing the sites FME culture was evident.

AR 211469 documented the identification of foreign material in the EDG #1 lube oil system subsequent to the trip on November 2, 2006. An immediate corrective action for this AR required a stand down with diesel crew mechanics emphasizing the importance of self-checking and reiterating the standards and expectations for utilizing adequate FME controls and inspections; specifically, the expectation was for the mechanic introducing material into a system to be responsible for removing it when exiting FME controlled area. The licensee also reinforced the expectations associated with peer inspector duties. These corrective actions focused on retrieval rather than prevention of FME. However, additional corrective actions identified in AR 211469 and tracked by AR 211236 included: revising procedures involving site work on EDG engine crankcases; revising OMMM-055 to establish single point FME accountability; and establishing high FME area classification for EDG crankcases. These actions were indirectly tied to prevention rather than retrieval because the high risk classification for FME on EDG crankcase jobs required greater attention to job planning and accountability.

Since 2006, licensee management has focused on changing the culture from "inspect and remove" to prevention of foreign material intrusion. This was communicated to maintenance staff via training, newsletters, the maintenance website, implemental review of jobs, pre-job briefs, and coaching as required. The team reviewed FME observations and ARs written during the B218R1 refueling outage and noted several positive FME practices by craft personnel including: identification and removal of legacy foreign material from systems; an apparent decrease in FME occurrences; use of plexiglass instead of mesh around the spent fuel pool; use of FME covers; around the shaft of service water pumps during maintenance; requests for machining FME covers; and the request for a clean room for performing pump and valve maintenance. The team further noted that licensee personnel identified and corrected two trends involving inappropriate FME prevention. Based on this and discussions with licensee personnel and document reviews, the team concluded that progress was being made in improving the implementation of the FME program as well as changing the culture from "inspect and remove" to prevention of foreign material intrusion.

b. Operating Experience Program

(1) Inspection Scope The team performed an independent review of the OE program to assess the validity of the licensee's conclusions regarding the extent to which that program has caused similar issues in the past and to verify the vital information has been properly entered into procedures and drawings. To accomplish this, the team reviewed procedures, select operating experience items, and Nordberg Service Bulletin 72:3. In addition, the team discussed the OE program and its implementation with cognizant licensee personnel including the OE Coordinator and select system engineers.

(2) Findings and Assessment Subsequent to the trip of EDG #1 on November 2, 2006, the licensee learned that a service bulletin had been released by Nordberg in 1972 addressing aluminum bearing service life. The licensee determined that failure to have a record of all applicable diesel generator service bulletins might have been a contributing cause to the failed crankshaft bearing. The licensee was in the process of reviewing other applicable service bulletins from Nordberg during this inspection and indicated that after review, necessary actions would be initiated. The licensee also determined that industry operating experience indicated that the crankshaft bearing that failed on EDG #1 (number 9) was inherently more prone to failure on Nordberg diesels due to operational stresses.

The team determined that procedure CAP-NGGC-0202, Operating Experience Program, was the corporate-level procedure used to implement the OE program. The procedure required select types of operating experience, such as NRC Information Notices and Part 21 reports, to be included in the program. For those types of OE, the OE Coordinator was to initiate an operating experience AR. If initial screening determined the OE item to be applicable, the AR was to be assigned to the appropriate department for review. Subsequent to the review, actions were to be generated as appropriate, including procedure revision requests. Other types of OE, including vendor information, were not required to be included in the OE program, but were screened by the OE Coordinator for inclusion. If deemed not applicable, the OE Coordinator could distribute the item by email to individuals (e.g. system engineer, maintenance) who may have interest as "information only" items. Tracking was not required nor routinely performed for these items. If the item was determined by the email recipient to be applicable, that individual would then either originate an operating experience AR or notify the OE Coordinator that one was needed. The recipient was also expected to forward the OE to any other individuals who may have interest. Individual system engineers, upon review, could choose to maintain an electronic copy of information determined to not be applicable for future reference. In addition, when OE was obtained through mechanisms other than via the OE Coordinator daily reviews, such as boiling water reactor users group, RHR users group, or informal communication with peers, no formal program requirements guided recipients in processing or tracking the OE information.

The failure to formally track the content, review, and basis for not including OE items deemed not applicable or not originating with the OE coordinator represented a potential weakness in the program. However, the team concluded that this potential weakness did not contribute to the EDG problems.

Based on review of the content of Nordberg Service Bulletin 72:3, the team concluded that knowledge of the information contained therein would not have prevented the bearing failure. The information included the fact that dirt and metallic debris in engine lubricating oil systems tended to shorted service life of aluminum bearings. The bulletin provided suggestions to assure maximum service life and further stated that loss of bearing crush would be difficult to determine in the field. Therefore, the bearing should be returned to the factory to determine it. Although the EDG bearing failure was related to inadequate crush on the crankshaft bearing, the inadequate crush was due to the bearing bore being oversized, a fact not addressed in the bulletin. From this the team concluded that, although the licensee's OE program had not captured all the information available on Nordberg diesels and aluminum bearings, the use of OE did not contribute to the bearing failure.

c. Maintenance Practices

(1) Inspection Scope The team completed an independent assessment of the operating performance of the HPCI, RCIC, EDG, and DC power systems. The team focused on equipment reliability, maintenance practices, and effectiveness of corrective actions for identified problems.

Licensee Event Reports were reviewed for system reliability issues. The team conducted a walkdown of the systems with system engineering personnel to check for any temporary modifications or deficiencies that could affect operational readiness. The team reviewed relevant corrective action documents, system health reports, completed surveillance tests, and modification history to verify that problems were being identified and resolved using the normal licensee processes. Maintenance in progress and selected maintenance work packages were reviewed using the inspection requirements of NRC Inspection Procedure 62700, Maintenance Program Implementation, to verify that maintenance activities were being conducted in a manner that resulted in reliable and safe operation.

(2) Findings and Assessment

Introduction.

No findings of significance were identified for the HPCI, RCIC, and DC power systems. However, the team identified a Green NCV of 10 CFR 50.65 (Maintenance Rule) for failure to properly administer the maintenance rule on one of the EDGs. The licensee failed to perform an adequate maintenance rule evaluation after EDG #3 exceeded its maintenance rule performance criteria and, as a result, did not set goals and monitoring as required.

Description.

In November 2004, EDG #3 experienced a fourth functional failure in 36 months, which exceeded the licensees established maintenance rule performance criteria of three failures. The licensee performed an investigation (AR 143328) for the four failures, which included an auto voltage regulator circuit board that failed after replacement, a significant jacket water leak due to maintenance processes and human performance deficiencies, an inadvertent loss of control power due to a maintenance planning error, and a failed relay coil due to aging caused by the deferral of a preventive maintenance activity. The licensees investigation did not identify a common cause(s)from any of the causal factors for the four EDG failures.

In February 2005 the maintenance rule expert panel met in accordance with procedure ADM-NGGC-0101, Maintenance Rule Program, to determine if performance goals should be set for EDG #3 as required by Paragraph (a)(1) of the maintenance rule because EDG #3 had exceeded its performance criteria of three failures in 36 months.

Based on discussion, the review of AR 143328, and other related documents the expert panel decided that the performance monitoring group did not require increased management oversight and that EDG #3 would remain in the preventive maintenance program as allowed by Paragraph (a)(2) of the maintenance rule.

The licensees evaluation was inadequate because it did not adequately assess generic, organizational, programmatic, or common causes as required by Section 9.8.2 of procedure ADM-NGGC-0101. At the time of the expert panels decision to keep the EDG in (a)(2) status the site had:

(1) open corrective actions (AR 108100) for the replacement of relays on all EDGs due to past failures;

(2) an adverse human performance trend in maintenance (AR 129173), including a failure of EDG #1 because the cylinder indicator cocks were left open by maintenance personnel; and

(3) open corrective actions (AR 135289) to review the preventive maintenance standards for critical components to properly address aging considerations on non-metallic parts such as elastomer parts. Also, the inspectors determined that the four EDG #3 failures displayed a common theme of human performance inadequacies in maintenance practices. Collectively, these issues indicated that the performance of EDG #3 was not effectively controlled through preventive maintenance. The result of not instituting performance goals for EDG #3 as required by maintenance rule Paragraph (a)(1) in 2005 was an additional (Section 02.01b) missed opportunity to improve availability and reliability of the EDGs and possibly avoid the degradation of the Emergency AC Power systems performance indicator and subsequent EDG failures.

The lack of proper attention by the maintenance rule program to the degraded performance of the EDG #3 allowed the degraded performance to continue for all EDGs. Specifically, the team noted two subsequent failures that were related to the issues mentioned above: a preventive maintenance human performance error (a cleaning towel left in the crankcase) was a contributing cause to the November 2006 failure of EDG #1; and a relay failure that resulted in a failure of EDG #2 in February 2007 was identified by the licensee as a similar failure of EDG #3 in November 2004.

Due to these continued performance problems the licensee classified all EDGs in accordance with Paragraph (a)(1) of the maintenance rule in 2007.

Analysis.

The finding was more than minor because it was associated with the equipment performance attribute and affected the Mitigating Systems Cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The lack of proper attention by the maintenance rule program to the degraded performance of EDG #3 allowed continued degraded performance for all EDGs. The team determined that the finding was of very low safety significance (Green) because it was not a design or qualification deficiency, did not directly result in an actual loss of safety function for a system or train, and was not risk significant due to a seismic, fire, flooding, or severe weather initiating event. The cause of the finding directly involved the cross-cutting area of human performance, in the decision making component under the aspect of using conservative assumptions because the expert panel decided to keep EDG #3 under maintenance rule Paragraph (a)(2) without fully supporting that conclusion. The panel based its decision mainly on the non-conservative assumption that performance of the other EDGs was good and they did not exhibit the same kind of failures as EDG #3. They made this decision even though other evidence, like that mentioned above, indicated that preventive maintenance was not effectively controlling EDG #3 performance H.1(b).

Enforcement.

Title 10 Code of Federal Regulations, Part 50.65 (10 CFR 50.65)

Paragraph (a)(1) requires, in part, that the licensee monitor the performance or condition of structures, systems, or components (SSC) against licensee-established goals, in a manner sufficient to provide reasonable assurance that such structures, systems, and components are capable of fulfilling their intended functions.

10 CFR 50.65 Paragraph (a)(2) requires, that monitoring as specified in paragraph (a)(1) of this section is not required where it has been demonstrated that the performance or condition of a SSC is being effectively controlled through the performance of appropriate preventive maintenance, such that the SSC remains capable of performing its intended function.

Contrary to the above, in February 2005, after an EDG had exceeded its maintenance rule (a)(2) performance criteria, which demonstrated that its performance was not being effectively controlled through the performance of appropriate preventive maintenance, the licensee performed an inadequate maintenance rule evaluation and, as a result, goal setting and monitoring was required, but not performed. Because the finding was of very low safety significance, and has been entered in the Corrective Action Program (AR 240192), it is being treated as an NCV, consistent with Section VI.A.1 of the NRC Enforcement Policy. This matter is identified as NCV 05000325, 324/2007010-02, Goal Setting and Monitoring not Performed for an Emergency Diesel Generator.

d.

Operational Decision Making Practices

(1) Inspection Scope The team completed an independent assessment of problem identification and operational decision making practices by the licensee regarding the HPCI and RCIC systems. The team specifically focused on identification and resolution of problems that could impact system reliability. The inspection included system walkdowns with engineering personnel, review of system health reports, and discussion of system reliability with operations personnel. Additionally, system operating procedures were reviewed for embedded operator workarounds and caution tags were evaluated for longstanding equipment problems that may be controlled outside of normal procedures or not otherwise corrected by maintenance. Completed surveillance test documents were reviewed for any equipment issues that had not been corrected using either the maintenance or corrective action programs.

(2) Findings and Assessment No findings of significance were identified.

02.05 Safety Culture Consideration

a. Inspection Scope

The team conducted a focused inspection of the safety culture at the Brunswick Station to independently determine that the licensee root cause evaluation appropriately considered safety culture components as a cause or significant contributor to these issues. The team identified the safety culture components that could reasonably have been the root cause or a significant contributing cause. The team then reviewed the licensees evaluations and interviewed affected personnel to verify the licensee considered weaknesses in safety culture components in the cause evaluation and that they considered those components as determined by the team.

b. Findings and Observations

The team considered that the following components could reasonably have been a root or significant contributing cause to the problems with the Emergency AC Power System and the failure of EDG #1 on November 2, 2006:

Human Performance-Decision Making Human Performance-Resources Problem Identification & Resolution-Corrective Action Program Problem Identification & Resolution-Operating Experience Other-Accountability

(1) Decision Making. No issues of significance were identified. This safety culture component would be the cause of the Emergency AC Power System problems if operations personnel tolerated recurring problems in the plant. This would constitute making safety-significant decisions outside of a systematic process (H.1.a). The licensee did consider decision making in the evaluation for the EDG #1 failure of November 2, 2006, and in some of the failures that contributed to the Emergency AC Power System PI decline and determined it was a contributor to the EDG #1 failure.

The team also looked for instances of tolerance for recurring problems by operations personnel as part of the independent extent of condition.

(2) Resources. No issues of significance were identified. This safety culture component would be the cause of Emergency AC Power System problems if the licensee tolerated long standing equipment problems or excessive preventive maintenance deferrals with the EDGs (H.2.a). The licensee did consider resources in the evaluation for the EDG #1 failure of November 2, 2006 and in some of the failures that contributed to the Emergency AC Power System PI decline and determined that inadequacies in the preventive maintenance program were a contributor.

(3) Corrective Action Program. No issues of significance were identified. This safety culture component could be the cause of Emergency AC Power System problems because the amount of time over which the problems occurred, July 2003 to February 2007, could indicate that the licensee did not address an adverse trend in a timely manner (P.1.d). The licensee did consider the corrective action program in a safety culture review for the EDG #1 failure of November 2, 2006 and as part of the collective common cause evaluation for the issues that led to this inspection (AR232815). They determined that there was not a breakdown of the corrective action program but that the use of cause codes designated for not applicable causes was a significant contributor to EDG performance. They initiated NCR 236415 to evaluate the high usage of these cause codes. While the licensee did acknowledge weaknesses in the corrective action program contributed to the Emergency AC Power System problems, the team noted that several potential corrective action program weaknesses needed further review. These were discussed as part of the URI in Sections 02.01.b & 02.03.a.

(4) Operating Experience. No issues of significance were identified. This safety culture component would be the cause of Emergency AC Power System problems if the use or missed use of Nordberg Bulletin 72:3 was a contributor to the EDG #1 failure of November 2, 2006 (P.2.b). The licensee did consider this as a possible cause of the EDG #1 failure and determined it did not contribute. The team looked at the Nordberg bulletin as part of the independent extent of condition and found that the bulletin did not contribute to the EDG #1 failure. The team did identify a potential weakness in the OE program regarding informal communication of information not required to be screened and tracked by procedure CAP-NGGC-0202, Operating Experience Program, including vendor information such as the Nordberg Service Bulletin, but determined that it did not contribute to the EDG #1 failure or the Emergency AC Power System problems.

(5) Accountability. No issues of significance were identified. This safety culture component would be the cause of Emergency AC Power System problems if actual management expectations, as opposed to those publicly stated, did not reflect nuclear safety as an overriding priority (O.1.b). The licensee did consider this as a possible cause of the EDG #1 failure and determined it did contribute. The team looked at accountability through interviews and found a very consistent belief that nuclear safety was an overriding priority.

MANAGEMENT MEETINGS

Exit Meeting Summary

The team presented the results of the supplemental inspection to Mr. Ben Waldrep and other members of licensee management and staff on July 20, 2007. The team leader conducted a second presentation by phone with Mr. Jim Scarola and other members of licensee management on August 30, 2007. The team confirmed that any proprietary information provided or examined during the inspection was returned.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

J. Anderson, Systems Engineer

D. Crawford, Work Week Manager

D. Griffith, Manager - Outage and Scheduling

J. Harrel, Diesel Generator Program Manager

A. Herring, Mechanical Maintenance Supervisor

T. Hobbs, Plant General Manager

S. Howard, Manager - Operations

K. Hughes, Sr. Mechanic

R. Ivey, Manager - Site Support Services

G. Jones, Sr. Mechanic

T. King, Mechanical Maintenance Superintendent

J. McIntyre, Preventive Maintenance Optimization Manager

W. Murray, Licensing Specialist

A. Pope, Supervisor - Licensing/Regulatory Programs

M. Potter, Operations Shift Superintendent

G. Raker, Site CAP Coordinator

W. Raker, CAP Trending

W. Richardson, System Engineer

L. Rieman, Training Supervisor

M. Rogers, System Engineer

R. Rogers, System Engineer

S. Rogers, Manager - Maintenance

J. Scarola, Site Vice President

T. Sherrill, Licensing Engineer

B. Smith, Systems Engineer

N. Smith, I&C Engineering Supervisor

D. Strong, Supervisory Engineer

G. Thearling, OE Coordinator

T. Trask, Manager - Engineering

T. Vereen, Mechanical Maintenance Supervisor

B. Waldrep, Plant General Manager

F. Ward, Sr. Mechanic

B. Wilton, Engineering Supervisor

O. Wrisbon, I&C Maintenance Supervisor

NRC Personnel

E. DiPaolo, NRC Senior Resident Inspector

J. Austin, NRC Resident Inspector

ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

05000325, 324/2007010-01 URI Handling of Diesel Generator Problems by CAP [Sections 02.01.b(2), 02.03.a(2)]

Opened and Closed

05000325, 324/2007010-02 NCV Goal Setting and Monitoring not Performed for an Emergency Diesel Generator

[Section 02.04.c.(2)]