IR 05000341/2004005
| ML041400423 | |
| Person / Time | |
|---|---|
| Site: | Fermi  |
| Issue date: | 05/20/2004 |
| From: | Eric Duncan Division Reactor Projects III |
| To: | O'Connor W Detroit Edison |
| References | |
| IR-04-005 | |
| Download: ML041400423 (16) | |
Text
SUBJECT:
FERMI POWER PLANT, UNIT 2 NRC SUPPLEMENTAL INSPECTION REPORT 05000341/2004005
Dear Mr. OConnor:
On April 23, 2004, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection at your Fermi Power Plant, Unit 2 facility. The enclosed report documents the inspection results which were discussed on April 23, 2004, with Mr. Cobb and other members of your staff.
The NRC performed this supplemental inspection to assess your evaluation of a White performance indicator in the Emergency Alternating Current Power System Unavailability area of the Mitigating Systems cornerstone. This inspection was conducted in accordance with Inspection Procedure 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area, and examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.
Based on the results of this inspection, we concluded that you have adequately completed a root cause analysis of the event and have identified appropriate corrective actions. No findings of significance were identified concerning the root cause evaluation and corrective actions. At the end of the inspection, a review of the impact of the potential loss of seismic qualification of an emergency diesel generator which primarily contributed to the White performance indicator had not been completed. The results of this review, and any findings and enforcement actions directly associated with the performance deficiencies which led to the emergency diesel generator inoperability, will be documented in a future inspection report.
W. OConnor, Jr. -2-In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA/
Eric R. Duncan, Chief Division of Reactor Projects Docket No. 50-341 License No. NPF-43
Enclosure:
Inspection Report 50-341/2004005 w/Attachment: Supplemental Information
REGION III==
Docket No: 50-341 License No: NPF-43 Report No: 05000341/2004005 Licensee: Detroit Edison Company Facility: Fermi Power Plant, Unit 2 Location: 6400 N. Dixie Hwy.
Newport, MI 48166 Dates: April 19 through April 23, 2004 Inspectors: J. Ellegood, Team Leader, Perry Resident Inspector T. Steadham, Fermi Resident Inspector Approved by: E. Duncan, Chief Branch 6 Division of Reactor Projects Enclosure
SUMMARY OF FINDINGS
IR 05000341/2004005; 04/19/2004 - 04/23/2004; Fermi Power Plant, Unit 2; Supplemental
Inspection; IP 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area.
Cornerstone: Mitigating Systems
The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess the licensees evaluation for a White performance indicator in the Emergency Alternating Current (AC) Power System Unavailability area of the Mitigating Systems cornerstone. This supplemental inspection was performed in accordance with Inspection Procedure 95001, Inspection For One Or Two White Inputs In A Strategic Performance Area, and concluded that the licensee performed a comprehensive evaluation of the unavailability of Emergency Diesel Generator (EDG) 12 from June 2, 2003, to November 8, 2003, which primarily contributed to the performance indicator crossing the Green-to-White threshold.
The licensees evaluation identified inadequate maintenance procedures, an inadequate review of maintenance procedures, and inadequate communication as the primary causes. The licensee had planned corrective actions, including training and procedural changes, to address these root causes.
Based on the results of the inspection, the inspectors concluded that the licensee had adequately completed a root cause analysis of the event and had identified appropriate corrective actions.
REPORT DETAILS
Inspection Scope The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess the licensees evaluation of a White performance indicator in the Emergency Alternating Current (AC) Power System area of the Mitigating Systems cornerstone. The performance indicator exceeded the Green-to-White threshold due to the improper connection of a fitting in a lube oil pressure sensing line associated with Emergency Diesel Generator (EDG) 12.
On June 2, 2003, licensee personnel performed maintenance on a lube oil pressure sensing line fitting to address a small oil leak associated with EDG-12. The fitting penetrated the engine bulkhead and was configured such that when the fitting was unscrewed from the bulkhead, an internal threaded fitting unscrewed as well. During the performance of this maintenance, the interior of EDG-12 was not accessible and the mechanic performing the work was unaware of the internal connection or the configuration vulnerability which would allow the internal fitting to loosen during maintenance. When the mechanic reconnected the fitting, full thread engagement on the internal fitting was not obtained. The loose connection resulted in indicated lube oil pressures lower than what was observed prior to the work, but above the 26 pound per square inch gauge (psig) procedural limit and the low pressure trip setpoint of 23 psig. After the problem was discovered, although the EDG successfully had satisfied surveillance testing requirements, the licensee could not demonstrate that the fitting would remain intact during a seismic event. Therefore, the licensee concluded that EDG-12 was inoperable from June 2, 2003, until the fitting was repaired and EDG-12 was returned to an operable status on November 8, 2003.
Evaluation of Inspection Requirements 02.01 Problem Identification a.
Determination of who (i.e. licensee, self-revealing, or NRC) identified the issue and under what conditions.
Following the completion of EDG-12 maintenance on June 6, 2003, licensee personnel conducted post maintenance testing to verify that the maintenance had been properly performed. Since the work on the fitting was believed to affect only the external portion of the fitting, the post maintenance test (PMT) consisted solely of an inservice leak test of the affected fittings. The PMT specified was not designed to detect an internal leak in the lube oil pressure sensing line. However, licensee procedures prescribed monitoring lube oil pressure during EDG testing and system engineers recorded these readings to identify adverse trends in EDG performance.
During EDG-12 surveillance testing in July, system engineers noted that lube oil pressure had decreased about 1.5 psig from the pressure measured prior to the performance of the EDG maintenance the previous month. Based on this decrease, licensee personnel entered the issue into their corrective action program as Condition Assessment Resolution Document (CARD) 03-10487.
On October 2, licensee personnel noted that lube pressure had decreased an additional 0.5 psig and again entered the issue into their corrective action program. Based on this additional drop in lube oil pressure, licensee personnel performed troubleshooting activities on EDG-12 on November 6 and November 7. This troubleshooting identified the loose connection which was then corrected.
Since the loose fitting was identified by the licensee using a program specifically intended to detect adverse trends in EDG performance, the inspectors concluded that the issue was licensee-identified.
b. Determination of how long the issue existed and prior opportunities for identification
.
Licensee personnel were able to determine that the fitting became loose during the June 2 to June 6 EDG maintenance outage. During the outage, licensee personnel disassembled a fitting for a lube oil pressure sensing line in order to repair a small oil leak. Although the leak was external, the disassembly of the fitting resulted in disconnecting the interior tubing as well. The work package in use did not address disassembly of the interior portion and due to the configuration of the fitting, licensee personnel were unaware that it had been disconnected. Therefore, the licensees post maintenance testing did not include tests to verify proper re-assembly of the interior portion of the fitting.
As part of the PMT for the work performed on EDG-12 during the June EDG outage, the licensee ran the EDG. During this PMT, the licensee recorded lube oil pressure as about 1.0 psig below pre-maintenance levels, but above required action levels. Since lube oil pressure was known to fluctuate slightly, this decrease did not concern cognizant licensee personnel. During the July EDG surveillance, the licensee recorded a drop of an additional 0.5 psig and entered the condition into their corrective action program.
In the months that followed, the licensee focused additional attention on the pressure drop and discussed potential causes with the emergency diesel generator owners group. Based on these discussions, the licensee calibrated the pressure gauge in September. This calibration identified that the as-found values were within specified limits. During a routine EDG-12 surveillance test in October, licensee personnel noted a further decrease in lube oil pressure. Based on this drop, licensee personnel placed EDG-12 out of service on November 6. Troubleshooting identified the loose internal connection in the lube oil pressure sensing line.
Based on a review of the sequence of events, the inspectors concurred with the licenseess evaluation of the duration of the EDG-12 inoperability. The inspectors also concluded that the licensees actions to identify this issue was accomplished in a reasonable amount of time.
c. Determination of the plant-specific risk consequences (as applicable) and compliance concerns associated with the issue.
The licensee concluded that EDG-12 would function as required during most accident scenarios, but could not conclusively determine that the EDG would remain operable during a seismic event. The licensee noted that EDG-12 had run successfully for about 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> during an August 14, 2003, grid blackout event. This supported the licensees conclusion that EDG-12 remained operable during non-seismic events. The inspectors reviewed the lube oil pressure data recorded during the event and concluded that the data indicated that the EDG would run successfully for its mission time for non-seismic events. Seismic data for Fermi indicated that even a low magnitude earthquake was much less likely than a loss of offsite power. Therefore, the licensee reported in Licensee Event Report (LER) 2003-004, EDG Lube Oil Pressure Low, Revision 0, that the event had an insignificant impact on the health and safety of the public.
The LER also identified that the licensee had operated the plant from June 6, 2003, through November 8, 2003, with an inoperable emergency diesel generator, contrary to Technical Specification 3.8.1 requirements. In addition, the root cause report identified that the work procedure did not contain adequate instructions for the work performed.
Since this event involved a seismic qualification issue, the Region III Senior Reactor Analyst planned to review the event to determine the impact of the issue on plant risk, and document the results of that review including the identification of any findings and enforcement actions as part of the review of LER 2003-004.
02.02 Root Cause and Extent of Condition a.
Evaluation of method(s) used to identify root causes and contributing cause(s).
The licensees root cause evaluation utilized three root cause methods: 1) Management Oversight and Risk Tree (MORT), 2) Event and Causal Factors Charting, and 3) Change
Analysis.
In addition, the licensee contracted an industry expert to assist in the root cause analysis.
The root cause report concluded that three factors contributed to the event: 1) a less than adequate procedure, 2) a less than adequate review of the procedure, and 3) poor communications between the personnel planning the work package and those performing the work.
The inspectors reviewed the root cause analysis methods employed and concluded that a formal, structured approach was utilized to perform the root cause analysis to identify root causes and contributing causes.
b.
Level of detail of the root cause evaluation.
The inspectors determined that the root cause evaluation was conducted to a sufficient level of detail. The licensee utilized multiple root cause methods to evaluate the issues and probed organizational factors to better understand the underlying causes of the problem.
c.
Consideration of prior occurrences of the problem and knowledge of prior operating experience.
The licensees review evaluated industry operating experience as well as internal records to determine if similar events had occurred previously. This review identified a similar occurrence at another nuclear facility where an identical fitting was removed for replacement, but the replacement part did not have the same configuration as the original. During this event, the facility detected the problem immediately since the indicated lube oil pressure was much lower than expected.
Based upon these reviews, the inspectors concluded that the licensee appropriately considered prior occurrences of the problem.
d.
Consideration of extent of condition and extent of cause of the problem.
The licenseess review focused primarily on two fittings with the same part numbers on the EDGs. During the inspection, the inspectors performed walkdowns of the EDGs and a walkdown of a training diesel generator with several cover plates removed. The inspectors noted that there were other engine bulkhead penetration fittings where work on the external portion could cause an internal threaded coupling to become disconnected. The inspectors discussed this observation with members of the licensee staff and concluded that validation that these fittings remained connected was appropriate to the extent of condition. Since this inspection has not been completed, the inspectors consider it to be an Unresolved Item (URI 05000341/2004005-01) pending NRC review of the results.
When the licensee discovered the problem with EDG-12, in accordance with Technical Specification 3.8.1 the licensee performed an extent of condition review specifically to determine if the other EDGs had a similar problem. During this review, they identified that the identical fitting on EDG-14 had been worked as well. However, upon review of the work actually performed, they confirmed the work did not disconnect the internal portion of the fitting.
02.03 Corrective Actions a.
Appropriateness of corrective action(s).
The licensee took immediate actions to reconnect the loose lube oil pressure sensing line fitting and verified that the other EDGs did not have a similar problem. In addition, mechanics and planners were briefed on the issue. As part of their long-term plans, licensee personnel developed corrective actions to improve the number and quality of work package reviews; improve the training of planners and mechanics; increase the level of detail in the procedures governing work order development; and improve drawings in the EDG maintenance procedures.
The corrective actions addressed not only the specific fitting, but the broader issues of procedural detail and internal communications.
Since the corrective actions addressed each of the root causes identified, the inspectors concluded they were appropriate to prevent recurrence.
b.
Prioritization of corrective actions.
Discussions with maintenance mechanics indicated that they were aware of the issues surrounding bulkhead fittings. Long-term actions to address the broader issues of work order content, work package reviews, and communications were scheduled for completion in June 2004.
Prioritization of the corrective actions was not directly based on risk perspectives or analysis, but rather based on a deterministic approach considering the significance of the problem.
The inspectors reviewed the prioritization of the corrective actions and verified that actions of a generally higher priority were scheduled for completion ahead of those of a lower priority. Therefore, the inspectors concluded the prioritization of corrective actions was appropriate.
c.
Establishment of schedule for implementing and completing corrective actions.
The licensees corrective action plan provided dates for the completion of corrective actions. The inspectors reviewed the licensees progress on selected corrective actions and determined that the corrective actions could reasonably be accomplished by the dates specified.
The inspectors concluded that the schedule for completion was reasonable.
d.
Determine that quantitative or qualitative measures of success have been developed for determining the effectiveness of the corrective actions to prevent recurrence.
The corrective action plan initially provided to the inspectors did not include an effectiveness review as specified in MQA11, Condition Assessment Resolution Document.
The inspectors discussed this with licensee personnel and an effectiveness review plan was subsequently developed and provided to the inspectors. The plan used qualitative measures to verify the effectiveness of corrective actions which focused on correcting the broader organizational issues. The licensee scheduled this effectiveness review for October 2004.
Management Meetings
Exit Meeting Summary
The inspectors presented the inspection results to Mr. D. Cobb and other members of licensee management at the conclusion of the inspection on April 23, 2004. The licensee acknowledged the findings presented. No proprietary information was discussed.
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
- D. Cobb, Director, Nuclear Production
- D. Chupurdy, Human Performance Coordinator
- M. Eghigian, EDG System Engineer
- M. Hobbs, Supervisor System Engineering, Electrical
- R. Johnson, Supervisor Licensing
- N. Peterson, Manager, Nuclear Licensing
- P. Roelant, EDG System Engineer
- C. Schumann, Principal Engineer, Maintenance
List of Items Opened, Closed and Discussed
Opened
05000341/2004005-01 URI Extent of Condition Review Associated With EDG Fittings
Closed
None.
Discussed
None.
Attachment
List of Documents Reviewed
CARD 03-12686; Loose Connection in Lube Oil Pressure Sensing Line of EDG 12; dated
November 8, 2003
CARD 03-12856; Review of VME8-1.1, ABN 9197-1, SIL 19, DER95-1042, and 34.307.001
Shows Inconsistent EDG Blower Clearances for Impeller to Outer Bearing Plate; dated
May 6, 2003
CARD 04-21720; VME8-1.5 Renewal Parts for Stationary Diesel catalog 4549.3, and 5987.1
not Updated for Present Configuration; dated April 20, 2004
CARD 04-21719; VME8-1.5 Renewal Parts for Stationary Diesel, Catalog 1.7, page 2 is
Missing; dated April 20, 2004
CARD 04-21703; NRC Concern / EDG Dowel and Nut Configuration on EDG #14 Front Cover;
dated April 19, 2004 (NRC-Identified issue)
CARD 04-21307; Ensure Internal Lube Oil Tubing Fitting is Tight; dated March 25, 2004
CARD 03-10847; EDG12 Lube Oil Pressure Slowly Trending Down; dated July 3, 2003
CARD 03-21311; EDG12 Lube Oil Pressure Continues to Drop; dated October 2, 2003
Root Cause Analysis Report for CARD 04-20185; NRC EDG Unavailability Performance
Indicator MS01 Change to White Status; dated March 16, 2004
35.307.008; Emergency Diesel Generator- Engine General Maintenance; Rev. 30
34.307.001; Emergency Diesel Generators - Inspection and Preventive Maintenance; Rev. 57
MWC 02; Work Package Preparation; draft
MWC 02; Work Package Preparation; Rev. 27
MQA 11; Condition Assessment Resolution Document; Rev. 9
MQA 12; Fermi-2 Ombudsman; Rev. 1
WR 000Z034098; EDG 12 Lube Oil Pressure Continues to Drop; dated November 8, 2003
WR 000Z020877; Repair Leaks on EDG-12; dated June 2, 2003
WR 000Z023206; Repair Oil Leaks on EDG 14; dated June 16, 2003
OE8001; Diesel Generator Tube Modification Results in Separation of Internal Lube Oil Line;
dated February 1996
Attachment
Document No. TMPE-04-004; Seismic Evaluation of EDG Lube Oil Line; dated
January 12, 2004
VME8-1.5; Colt Industries (Fairbanks Morse) Model 38TD8 1/8; dated May 8, 2002
LER 2003-004; EDG 12 Lube Oil Pressure Low; Rev. 0
Summary of Sequence of Events, EDG #12 Lube Oil Line
E-mail from
- P. Roelant to M. Eghigian, et al; CARD solution team meeting; dated
October 2, 2003
Document No. TMTE-03-0137; Fairbanks-Morse Owners Group Meeting Trip Report; dated
September 9, 2003
Selected Control Room Operator Logs from June 2, 2003 through June 3, 2003
Selected Control Room Operator Logs from November 8, 2003 through November 9, 2003
Attachment
LIST OF ACRONYMS USED
CARD Condition Assessment Resolution Document
CFR Code of Federal Regulations
EDG Emergency Diesel Generator
LER Licensee Event Report
NRC Nuclear Regulatory Commission
PMT Post Maintenance Test
WR Work Request
Attachment