IR 05000272/2003010
| ML040330012 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 01/30/2004 |
| From: | Lanning W Division of Reactor Safety I |
| To: | Richard Anderson Public Service Enterprise Group |
| Fuhrmeister R | |
| References | |
| IR-03-010 | |
| Download: ML040330012 (14) | |
Text
ary 30, 2004
SUBJECT:
SALEM GENERATING STATION - NRC SUPPLEMENTAL INSPECTION REPORT 05000272/2003010, 05000311/2003010
Dear Mr. Anderson:
On October 24, 2003, the U. S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection at your Salem Generating Station, Units 1 and 2. The enclosed report documents the inspection findings which were discussed with Messrs. J. Carlin, D. Garchow and other members of your staff on October 24, 2003, and at a formal inspection exit with yourself and members of your staff on December 16, 2003.
The inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.
The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel. The inspection reviewed PSEG Nuclears actions in response emergency diesel generator (EDG) turbocharger failures. These issues were identified in our inspection report dated May 1, 2003, and were determined to be of low to moderate safety significance (White).
This inspection examined actions you have taken to address both specific technical concerns and a broader issue associated with the application of some elements of your corrective action program. We consider your specific corrective actions dealing with the technical aspects of EDG turbocharger failures to be adequate. The broader issue involves ensuring that controls and procedural requirements that are determined to be necessary based on evaluations of equipment failures are reliably tracked and implemented. A significant contribution to the recurrent failure of the EDG turbocharger was deletion of certain maintenance requirements which had been identified in previous reviews. We believe additional NRC inspection is necessary to confirm the adequacy of the efforts you are taking to address this issue.
Accordingly, we are extending the applicability of this White finding beyond the four quarters it would normally apply in the Operation Reactor Assessment Program Action Matrix. We recognize that you have ongoing efforts for improvements to address NRC-identified substantive cross cutting issues in the problem identification and resolution areas of your corrective action program that were identified in both the last annual and mid-cycle performance review letters dated March 3, 2003 and August 27, 2003, respectively.
Mr. Roy In accordance with 10 CFR 2.790 of the NRCs Rules of Practice, a copy of this letter, its enclosure, and your response, if any, will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC website at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
If you have any questions, please contact John F. Rogge of my staff at (610) 337-5146.
Sincerely,
/RA/
Wayne D. Lanning, Director Division of Reactor Safety Docket Nos.: 50-272; 50-311 License Nos.: DPR-70; DPR-75 Enclosure: Inspection Report 05000272/2003010 and 05000311/2003010 cc w/encl:
A. C. Bakken, Senior Vice President Site Operations J. T. Carlin, Vice President Nuclear Assurance D. F. Garchow, Vice President, Engineering and Technical Support W. F. Sperry, Director Business Support S. Mannon, Manager - Licensing C. J. Fricker, Salem Plant Manager R. Kankus, Joint Owner Affairs J. J. Keenan, Esquire Consumer Advocate, Office of Consumer Advocate F. Pompper, Chief of Police and Emergency Management Coordinator M. Wetterhahn, Esquire State of New Jersey State of Delaware N. Cohen, Coordinator - Unplug Salem Campaign W. Costanzo, Technical Advisor - Jersey Shore Nuclear Watch E. Zobian, Coordinator - Jersey Shore Anti Nuclear Alliance
Mr. Roy
SUMMARY OF FINDINGS
IR 05000272/2003-010, 05000311/2003-010; 10/20/03 - 10/24/03; Salem Generating Station,
Units 1 and 2; Supplemental Inspection
Cornerstone: Mitigating Systems
The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess PSEG Nuclears evaluation associated with the ineffective implementation of corrective actions which led to the failure of the turbocharger on the 1C emergency diesel generator (EDG) in September 2002. This performance issue was previously characterized as having low to moderate risk significance (White) in NRC Inspection Report No. 50-272&311/02-10. During this supplemental inspection, performed in accordance with Inspection Procedure (IP) 95001, the inspectors determined that PSEG Nuclear performed a comprehensive evaluation of the failed turbocharger. PSEG Nuclears extent of condition evaluation of corrective action implementation problems was not completed at the time of the inspection. PSEG Nuclear attributed the failure to adequately implement corrective actions to the loss of the technical and historical bases for corrective actions.
Given that PSEG Nuclears extent of condition evaluation had not been completed as of the time of the inspection, the White finding associated with the ineffective implementation of corrective actions will not be closed at this time.
Report Details 01
INSPECTION SCOPE
The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess PSEG Nuclears evaluation associated with the ineffective implementation of corrective actions which led to the failure of the 1C diesel generator turbocharger in September 2002.
This performance issue was previously characterized as white in NRC Inspection Report No.
50-272&311/02-10, and is related to the mitigating systems cornerstone in the reactor safety strategic performance area.
EVALUATION OF INSPECTION REQUIREMENTS 02.01 Problem Identification a.
Determination of who (i.e., licensee, self-revealing, or NRC) identified the issue and under what conditions The ineffective implementation of corrective actions to prevent recurrence (CAPR) from prior turbocharger failures was self-revealing when the 1C emergency diesel generator (EDG) turbocharger failed during an endurance run being conducted on September 13, 2002.
b.
Determination of how long the issue existed, and prior opportunities for identification The issue of ineffective implementation of corrective actions has existed since October 1999. In both 1999 and 2001 the procedural requirement for turbocharger non-destructive examination (NDE) was deleted without understanding the basis for the requirement. The procedural requirement was a CAPR to address previous turbocharger failures. Subsequently, PSEG deleted the requirement in late 2001 because it was not a vendor requirement and again failed to recognize the basis for the CAPR requirement.
c.
Determination of the plant-specific risk consequences (as applicable) and compliance concerns associated with the issue PSEG Nuclear has not performed a risk evaluation of the 1C EDG turbocharger failure.
The NRC performed a Phase 3 SDP using SPAR Model Revision 3.02 and determined the change in core damage frequency to be 8.64x10-6 per year due to internal initiating events. Insufficient data existed to estimate the risk from external initiators.
02.02 Root Cause and Extent of Condition Evaluation a.
Evaluation of method(s) used to identify root cause(s) and contributing cause(s).
To evaluate this issue, PSEG Nuclear used a combination of structured root cause analysis techniques including fault tree analysis, Organizational and Programmatic failure charting and events and causal factor analysis. The inspectors determined that PSEG Nuclear followed its procedural guidance for performing level 1 root cause analysis. The procedure required conducting interviews with key personnel and preserving evidence associated with the issue. To evaluate the extent of the corrective action implementation issues, PSEG Nuclear is conducting a review of the SAP computer databases to identify CAPR resulting from Significance Level 1 (SL-1) root cause determinations. At the time of the inspection, the extent of condition had not been completed.
b.
Level of detail of the root cause evaluation The PSEG Nuclear root cause analysis of the turbocharger failure was divided into two phases. Phase 1 consisted of a detailed analysis of the rotating element of the turbocharger. The failure was due to fatigue failure that initiated at a pit-like surface on the concave side of the blade, along the hub to blade fillet.
Phase 2, to determine the cause of fatigue, crack initiation, and crack propagation had not been completed at the time of the inspection. This phase includes evaluating the turbocharger design conditions, running a rebuilt turbocharger at a test facility up to the equivalent of 110% engine load to evaluate vibration and surging, and obtaining in-plant operating information by instrumenting turbochargers during scheduled endurance runs.
The test facility turbocharger runs have been delayed due to test facility problems.
PSEG Nuclears evaluation of the cause of the ineffective corrective actions included a review of the history of the preventive maintenance (PM) program for the diesel generators. The review looked at the changes made to the PM program and the rationale for these changes. PSEG concluded that the deletion of the turbocharger NDE PM in the 2001 time frame resulted from a loss of the technical and historical bases for its creation.
c.
Consideration of prior occurrences of the problem and knowledge of prior operating experience PSEG Nuclear determined that there had been 4 previous turbocharger failures among the Salem EDGs. Failures in 1990 and 1998 were attributed to fatigue with the 1998 failure being the result of rebuilding the turbocharger with reverse-engineered, non-OEM parts, which did not meet dimensional specifications. Causes for the earlier failures were not available in all cases. One failure, during factory testing in the mid-1970s, resulted in redesign to eliminate surging during operation at rated conditions.
As discussed in the Annual Assessment Letter dated March 3, 2003, and the Mid-Cycle performance Review dated August 27, 2003, inspection findings have been documented during the past year at Salem in the area of corrective actions for identified problems.
These shortcomings resulted in both of those letters identifying a substantive cross-cutting issue in the area of problem identification and resolution.
d.
Consideration of potential common cause(s) and extent of condition of the problem PSEG Nuclears evaluation considered common cause failure to not be a concern for the turbochargers. This judgement was based on not identifying a maintenance practice, design configuration, operating procedure or material deficiency common to all EDGs which would lead to fatigue. OEM design information indicated there was no engineering or material limit on turbocharger life. Industry operating experience indicates that turbocharger failures are normally due to improper storage, poor refurbishment practices, and foreign object ingestion. It also indicated that turbine blade fatigue failures are very uncommon, and inducer blade failures are almost non-existent.
At the time of the inspection, PSEG was in the process of determining the extent of the issue of loss of CAPR resulting from prior root cause determinations. This review consisted of a review of the CAPR from prior SL-1 root cause evaluations since July 1999 (implementation date of the SAP computer system) to determine if they were still in effect.
02.03 Corrective Actions a.
Appropriateness of corrective action(s)
PSEG Nuclear took immediate corrective actions to make the 1C diesel generator operable. The failed turbocharger was replaced and the EDG was satisfactorily tested, including its 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> endurance run. PSEG Nuclear has scheduled NDE of the inducer blades after each endurance run with 110% engine load. This is intended to identify cracked inducer blades before the crack propagates to failure. Any cracking will be cause for turbocharger replacement and refurbishment. In early December 2003, because the cause for the fatigue cracking and crack propagation may never be fully understood, PSEG corrective action plans were changed to include scheduled replacement of the turbochargers for all of the Salem EDGs with rebuilt units using new compressors. The replacements are scheduled to be performed during the next refueling outage, and every four refueling outages thereafter.
The Root Cause Report Addendum, issued October 2003, identified several organizational contributors to the turbocharger failure. New actions were developed and assigned to identify document changes and revisions instituted as corrective actions to prevent recurrence and to tie them back to the original issue, and to review the SAP database to ensure that all CAPRs initiated as a result of Level 1 Root Cause reviews were still active. If any are not still active, they will either be reactivated, or the rationale for their deletion reviewed and reapproved. These actions were scheduled to run through the end of March 2004.
b.
Prioritization of corrective actions PSEG Nuclears immediate corrective actions restored the 1C diesel generator to operability within the allowed by the outage time in the technical specification (TS).
Identification of the programmatic contributors to the deletion of prior CAPR did not occur until Mid-October 2003, nearly a year after the event, and just prior to the followup inspection. The delay was due, in part, to PSEG Nuclear depending on laboratory testing of the turbocharger. Due to delays with the laboratory, PSEG Nuclear changed the approach to establish corrective actions that would bound and preclude further fatigue failures of turbocharger blades. PSEG Nuclear has scheduled replacement of all the EDG turbochargers with rebuilt units using new compressors at the next refueling outage, and every four refuelings thereafter.
The extent of condition review for the programmatic issues related to cancellation of prior CAPRs, and corrective actions to ensure that prior CAPR from previous SL-1 root cause evaluations are still active were being addressed by the licensee.
c.
Establishment of schedule for implementing and completing the corrective actions PSEG Nuclear has already completed actions to confirm the configuration of the failed turbocharger matches its design, instrumented turbochargers to determine in-plant operating conditions, verified serial numbers of the turbochargers, revised purchasing documents to prohibit changing of serial numbers during refurbishment and to require traceability of parts is maintained, performed NDE on the inducer blades, and scheduled post endurance run NDE for those EDGs which have not yet had their 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> endurance runs. After the inspection, PSEG scheduled replacement of the turbochargers with rebuilt units using new compressors at the next refueling outage, and subsequent replacement every four refuelings thereafter.
However, completion of the extent of condition review for the programmatic issues related to cancellation of prior CAPRs, and corrective actions to ensure that prior CAPR from previous SL-1 root cause evaluations are still active.
d.
Establishment of quantitative or qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence PSEG Nuclear has specified a no cracks acceptance criterion for the periodic NDE of the turbocharger inducers. This NDE will be conducted after each 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> endurance run (or once per refueling cycle). In addition, a periodic replacement schedule has been established.
PSEG Nuclear will ensure that all corrective actions to prevent recurrence from prior Significance Level 1 root cause evaluations are still in effect, or that the rationale for canceling the action is valid.
03. MANAGEMENT MEETINGS The inspector conducted a debrief at the end of the on-site portion of the inspection on October 24, 2003. At that meeting, the inspector discussed the preliminary inspection findings, and possible directions NRC actions might take.
An exit meeting was conducted at the site with Mr. Roy Anderson and other members of the PSEG Nuclear staff on December 16, 2003. At that meeting, senior NRC managers discussed the specifics of the inspection findings, as well as PSEGs broader corrective action initiatives.
Following the exit, PSEG Nuclear staff continued to provide the inspector with updates on their progress regarding Performance Improvement Program enhancements and extent of condition review.
Documents reviewed during the inspection included several marked as proprietary. All proprietary documents were returned to PSEG Nuclear at the end of the inspection.
ATTACHMENT
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
PSEG Nuclear
- R. Anderson, Chief Nuclear Officer
- D. Garchow, Vice President - Operations
- J. Carlin, Vice President - Nuclear
- C. Fricker, Salem Operations Manager
- A. Khanppour, System Engineering Manager
- G. Salamon, Licensing Manager
- J. Nagle, Licensing Supervisor
- J. Morrison, Reliability Engineer
- H. Burrick, licensing Engineer
- D. Kolasinski, System Engineer
New Jersey Bureau of Nuclear Engineering
- J. Lipoti, Director
- E. Rosenfeld, Inspector
U.S. Nuclear Regulatory Commission
- H. Miller, Regional Administrator, Region I
- D. Roberts, Director, Project Directorate 1-2B, NRR
- J. Rogge, Chief, Electrical Branch, Region I
- G. Meyer, Chief, Projects Branch 3, Region I
- D. Orr, Senior Resident Inspector
- M. Gray, Resident inspector
- M. Ferdas, Resident Inspector