ML20206C004
| ML20206C004 | |
| Person / Time | |
|---|---|
| Site: | Crystal River  |
| Issue date: | 10/31/1988 |
| From: | Morgan T, True D ERIN ENGINEERING & RESEARCH, INC. |
| To: | |
| Shared Package | |
| ML20206B990 | List: |
| References | |
| FRN-57FR14514, REF-GTECI-B-56, REF-GTECI-EL, TASK-B-56, TASK-OR AE06-1-011, AE6-1-11, NUDOCS 8811160072 | |
| Download: ML20206C004 (132) | |
Text
--_ __
c> +
\
ENCLOSURE 4, Ref. 11-2-8B NUMARC/EPRI/NRC Meeting
SUBJECT:
EDG RELIABILITY INVESTIGATION OF AN EMERGENCY DIESEL GENERATOR RELIABILITY PROGRAM:
A CASE STUDY OF CRYSTAL RIVER UNIT 3
'f1(6)pe !7 hf$$MI[: 7 October, 1988 Prepared by -
ERIN ENGINEERING AND RESEARCH, INC.
1850 Mt. Diablo Blvd., Suite 600 Walnut Creek, California 94596 i
Principal Investigators T. A. Morgan D. E. True Prepared for Electric Power Research Institute 3412 Hillview Avenue Palo Alto, California 94304 EPRI Project Manager J. P. Gaertner
\
i l
fN"to?NRUyljpg,
~U i
~
ABSTRACT In support of Industry efforts to address the station blackout and diesel generator (EOG) reliability issues, the Electric Power Research Institute (EPRI) is developing candidate emergency diesel reliability program guidelines. These guidelines are based on the concept of reliability grcuth through maintenance and operational imprevements and include use of a systematic evaluation process to identify and address dominant system failure modes such as-reliability-centered maintenance (RCM). This report describes 'a case study of the reliabi!ity improvements realized on the Crystal River Unit 3 imergency diesel generators.
The Crystal River Unit 3 plant had experienced some EDG unreliability problems during early years of operation. These problems were resolved through the implementation of various operational and maintenance program improvements al'ong with some minor design modific4tions. Additionally, Crystal River has performed and implemented a systematic evaluation of experienced failure modes to further improve EDG reliability. The combination of the systematic evaluation of experienced failure modes and other actions provided a plant history from which the effectiveness of many of the EPRI program elements could be assessed.
Utilizing the Crystal River experience, each of the primary elements of the EPRI EDG reliability program guidelines are evaluated for applicability and effectiveness.
1
ACKNOWLEDGMENT The investigation described herein would not have been possible without the timely support and cooperation of Florida Power Corporation personnel. In particular, Mr. T. P. Montgomery, Mr. J. Warren, Mr. B. Crane, and Mr. J. Andrews provided invaluable assistance in the collection and interpretation of plant data for this project.
9 O
e 11
l.
r s .
l
~
l CONTENTS Section g 1 INTRODUCTION 11 1.1 Background 11 1.2 Project Overview 13 1.3 Project Summary and Conclusions ..
1-5 2 CRYSTAL RIVER EXPERIENCE 21 1.2 Crystal River Emergency Diesel 22
- Generator Experience 2.2 Sunnary of Crystal River EDG 17 Improvement Actions 3 APPLICATION OF EPRI PROGRAM ELEMENTS 31 3.1 Application of Reliability Targets 32 3.2 Important Failure Review 36 3.3 Tracking of EDG Corrective Maintenance History 37 3.4 Calculation of EDG Unavailabilities 3-13 3.5 Comparison of Systematic Evaluation Program Content 3 14 3.6 Expected Reliability Application 3-15 3.7 Documentation Inputs and Requirements 3 16 1
4
SUMMARY
AND CONCLUSIONS 41 l
4.1 Summary of Crystal River Experience 41 4.2 Results of EPRI Program Application 41 5 REFERENCES 51 APPENDIX A EMERGENCY DIESEL GENERATOR RELIABILITY PROGRAM GUIDELINES A1 APPENDIX B METHODS FOR CALCULATING EXPECTED EDG RELIABILITY 81 111
. s ILLUSTRATIONS Fiaure g 1-1 EPRI EDG Reliability Program Flowchart 12 2-1 Crystal River EDG Failure :listory Timeline 26 3-1 Number of EDG Failures In Past 50 Demands Per Year '- 34 __
32 Number of EDG Failures In Past 100 Demands Per Year 35 L=
33 Crystal River EDG CM History, Total Safety 3-10
. Functional Failures 3-4 Crystal River EDG CM History, Subsystems: Fuel, -
I & C, Lubrication and Starting Functional Failures 3-11 .
35 Crystal River EDG CM History, Subsystem: Generator, Cooling, Engine and Intake & Exhaust Functional Failures 3-12 . ,
m I
l l *
( iv ,
. TABLES E E191 2-1 Crystal River EDG Failure History 2-3 3-1 Crystal River EDG Demand Between Failure History 33 r
4 e
t i
0
)
l l
l I l
E I
l 6
I t l l f .
L i
l i
f
~ .
. Section 1 INTRODUCTION
1.1 BACKGROUND
During the past few years, the nuclear utility industry and the Nuclear Regulatory Commission (NRC) staff have developed guidelines and steps which will assure that the risk from station blackout at nuclear plants is acceptably low. These guidelines and steps are described in NUMARC 87-00, ' Guidelines and Technical Bases for NUMARC Initiatives Addressing Station Blackout at Light Water Reactors *
(1) and in draft Regulatory Guide 1.155, "Station Blackout" (2). One element of these guidelines is an emergency diesel generator reliability .
program.
To assist a utility in maintaining Emergency Diesel Generator (EDG) reliability and to provide a program of potential value in addressing the station blackout issue described above, EPRI is developing proposed guidelines for an EDG reliability program (Appendix A). This program makes use of the principle of reliability growth as a means to effectively improve EDG reliability through changes to the EDG maintenance, testing and design.
As part of the NUMARC station blackout initiatives, each plant will be required to select an overall EDG reliability target based on a plant specific analysis of station blackout coping capability. Should EDG test data indicate an
- unreliability problem, some action is req'iired. The EPRI diesel generator reliability program provides a graded response to observed unreliability excursions based on EDG test data. The program utilizes trigger levels based on the number of observed successes and failures out of the past 50 or 100 demands, to initiate a graded level of response. The elements of the program are t
constantly evolving as it undergoes iterative reviews by NUMARC, utilities, and NRC. A flow chart of the program as it was proposed at the time of this case study is presented in Figure 1-1.
u - - - - - - - - - - - - - - - - - - - - . - - - - - - - - - - - - - - - - - - - - - - - _ _ _ _ _
y FIGURE 1-1 i EPRI EDG RELIABILITY PROGRAM
> MONTHLY TESTING 1P SUCCESSES /
FAILURES IN PAST 50/100
. DEMANDS ,
LOW l
i MARGINAL '
l
(%
HIGH
\
PERFORM RCM -
UPGRADE OF MAINTENANCE PROGRAM 1 r 1 r
' STUDY PAST STUDY PAST ,
FAILURES FAILURES 1r 1 P TRACK cms TRACK cms 1r 1V 1V DETERMINE DETERMINE DETERMINE
, CAUSE OF CAUSE OF CAUSE OF NEW FAILURES NEW FAILURES NEW FAILURES h [ 1r F MODIFY TESTING ' F PROGRAMS OF MODIFY PROGRAM' r SYSTEM DESIGN IF COMMON TO ADDRESS CAUSE FAILURE t FAILURES ; OR IMPORTANT t ;
r m NOT CALCULATE OK EXPECTED RELIABILITY t - ,)
4 OK 3r 1-2
For plants with high reliability (i.e., neither of the trigger levels exceeded),
the only additional required actions are the review of observed failures as they occur to determine failure cause and the implementation of actions to address failures which are oeemed important due to comon cause failure potential or
~
i potentially severe consequ3nces.
Plants wnich exceed either the trigger for the 50 demand sample or the trigger for I
the 100 demand sample but not both are classified as ' marginal". Like the "high' reliability plants, these plants are required to determine the cause of any new failures. They mu:;t also perform a systematic evaluation of experienced failures to identify applicable and effective maintenance, surveillances or other changes to address these failure modes, initiate tracking of corrective maintenance as a leading indication of deteriorating reliability, and make' changes deemed appropriate by plant staff to their practices (and if necessary, to their design) to address the observed failures. Upon completion of these steps, a new expected reliability of the EDGs is calculated based on a simple procedure which accounts for the expected effectiveness of the measures taken.
- Plants which exceed both the 50 and 100 demand triggers are classified as ' low' reliability plants and are expected to perform all of the steps required of "marginal" plants plus a systentic evaluation of all dominant EDG failure modes,
, including potential failure modes, to identify further actions which could improve reliability.
The elements of the program are described in more detail in Appendix A.
1.2 PROJECT OVERVIEW '
EPRI has undertaken the assessment of the principles of this program through a case study of Crystal River Unit 3 in order to assess the applicability and cost effectiveness of the program for the industry. Crystal River was chosen as the trial application plant for the following reasnns:
- 1) The plant had, at one time, experienced EDG reliability problems, but has seen marked improvement in the last few years.
- 2) A utility sponsored EDG reliability centered maintenance (RCM) program with many elements similar to the systematic evaluation of failure modes proposed in the EPRI program had been undertaken and the results implemented in 1986. This would provide existing data on the effectiveness of systematic evaluation processes as a means to improve EDG reliability and avoid the costly and time consuming task of applying each of the program elements.
1-3 _ - _ _ _ -
l The scope of the project is to investigate the actions taken at Crystal River that were aimed at improving EDG reliability and to compare the elements of the program t undutaken to those required in the proposed EPRI program. Specific elements of l the project include the following:
o Investigation of the reliability history of the Crystal River Unit 3 diesel generators, o Investigation of the RCM program and other actions undertaken by Crystal River and assessment of the effectiveness of the implemented maintenance program changes in improving EDG reliability.
o Assessment of the similarities and differences between the reliability program elements at Crystal River and those recommended in the EPRI program.
~
o Technicai assessment of each of the EPRI program elements in light of Crystal River plant experience, o Assessment of the applicability and cost effectiveness of each of the elements of the proposed EPRI program, based on the Crystal River experience and tie findings of the investigation.
To achieve this, the case study was divided into two primary phases. The initial phase of the project involved an investigation of the Crystal River EDG history.
This included observed EDG reliability, findings and recommendations of two systematic reliability evaluations of EDG failures modes (an EDG Task Force and an ,
RCH study), and EDG operation and maintenance changes during the plant life. This phase was initiated with a plant visit by project personnel to familiarize themselves with the Crystal River EDG system design, operation and maintenance practices. Additionally, data was collected and analyzed on EDG testing and l failures. A summary analysis of the data collected in this phase of the investigation is presented in Section 2.
I
. The second phase of the project centered around the assessment of the EPRI program elements against the Crystal River experience. Each of the major EPRI program l elements was applied and evaluated including the 50 and 100 demand reliability triggers, scope and content of reliability improvement steps taken, expected reliability calculations, and important failure reviews. The results from this
! phase of the project are described in Section 3.
i 1.3 PROJECT SUtHARY AND CONCLUSIONS The Crystal River Unit 3 emergency diesel generators have experienced a significant improvement in reliability over the life of the plant. During the i
i 1-4
s .
first five years of plant operation the overall diesel generator reliability l
averaged less than 95%. During the most recent years of operation (since early
]
1982), the diesel reliability at Crystal River has averaged nearly 99%. A number of improvements have been made to plant surveillance and maintenance p'.ectices including some recommendations from the EDG Task Force and the RCH study. During that time no major design modifications were made to the diesel generator system.
This experience at Crystal River indicates that well planned changes in testing and maintenance programs can be effective in improving diesel reliability.
Each of the key elements of the proposed EPRI diesel generator reliability program were evaluated in light of the Crystal River experience. Since implementation of the reliability centered maintenance program at Crystal River, the EDGt have gone longer without a failure than at any time in the plant history. This is indicative of the potential benefit of the systematic evaluation of EDG dot inant failure modes. In addition to demonstrating the effectiveness of the syst(matic
. evaluation of EDG failure modes, this Crystal River case study demonstrates the value of other EPRI program elements including the proposed reliability triggers, the expected reliability calculation, and important failure reviews. However, some minor adjustments to the EPRI program are reconcended as a result of this investigation.
l l
l l
i I
(
l5
% 0 i
l l
l Section 2 CRYSTAL RIVER EXPERIENCE The initial phase of this project focused on the investigation of the design features and operating experience of the Crystal River EDGs and on maintenance program practices during the life of the plant.
Crystal River Unit 3 is an 825 MWe Babcock & Wilcox PWR with two identical 2750kW Fairbanks Horse emergency diesel generators. Each diesel has its own independent support systems to provide lubrication oil, fuel oil, cooling, and starting air.
The diesel assemblies are laated in adjacent, but separate rooms in the plant.
. In the event of a loss of off-site power, either of these diesels is capable of providing adequate AC power to achieve safe shutdown of the plant. The two
- diesels are normally in a standby mode and can be started by any one of a number of signals.
Routine testing of the diesels is performed in accordance with Technical
. Specifications at least once every 31 days. Current practice is to perform the routine testing of the diesel trains on a staggered basis, with one start test every two weeks. These monthly tests are normally ' slow start' tests (i.e., the diesels are accelerated only to idle speed, rather than to rated speed). However, a ' fast start' test is performed on each diesel train every six months (prior to early 1987 all starts were ' fast starts'). Prior to running a diesel for other than emergency conditions, the engine is pre lubed to minimize component wear i during engine startup. Additional test demands of the diesels also occur for various reasons, including post maintenance testing to ensure operability and loss of off site power tests.
2.1 CRYSTAL RIVER EMERGENCY DIESEL GENERATOR EXPERIENCE A review of the Crystal River Licensee Event Reports (LERs) and Nonconforming l
Operations Reports (NCORs) revealed a total of 25 failures of the EDGs to start or l load and run since plant startup in March, 1977. A brief description of each of these failures, taken from the appropriate LER or NCOR, is provided in Table 2 1 ;
i j
l 21 l
. s including the date Gf failure, cause of failure, and type of failure (i.e.,
, failure to start (FS) or failure to load and run (FL/R)).
The data from the LERs and NCORs is sumar.ized in the following table:
CRYSTAL RIVER EDG FAILURE SUM ARY
~
EDG 'A' EDG 'B' Failure Failure to EDG Failure Failure to EDG Plant to Start Load /Run Total to Start Load /Run Total Total 1977 1 0 1 3 0 3 4 1978 0 0 0 3 0, 3 3 1979 0 1 1 0 3 3 4 1980 0 1 1 1 1 2 3 1981 0 1 1 3 1 4 5 1982 1 1 2 0 0 0 2 1983 0 1 1 0 1 1 2 1984 0 0 0 0 0 0 0 1985 1 0 1 0 0 0 1 1986 1 0 1 0 0 0 1 1987 0 0 0 0 0 0 0 4 5 9 10 6 16 25 A graphic representation of this same data is provided in the form of a timeline in Figure 2-1. From this timeline and the table above it is clear that the majority of EDG failures (21 out of 25) occurred in the first 5 years of operation (3/77 to 3/82). Since early 1982, when an EDG task force was formed to address EDG reliability problems, far fewer failures have been observed (4 failures in 5 years). Since the implementation of the RCM program recossendations in early 1986, no failures have been observed.
2'
u .
Table 2-1 Crystal River EDG Failure History
~ LER/
NCOR Event No. Date Occurrence /Cause EDG FS FL/R 77-55 6/2/77 EDG A failed to start 'A' X due to loose injector holddown nuts; restored in 2 hrs.77-93E 7/26/77 EDG 'B' failed to rtart "B' X due to operator fa lure to reset trips; restored ..
in 45 minutes.77-128 9/28/77 EDG *n' failed to start 'B' X due to low lube oil pressure; restored in 7 hrs. .77-158 12/27/77 EDG 'B' failed to start 'B' X due to foreign matter in governor; repaired in 11:35 hrs.
78 01 1/3/78 EDG 'B' failed to 'B' X start, foreign matter in servo booster of governor; repaired in 5:30 hrs.
78 060 11/17/78 EDG 'B' failed to start 'B' XX(2) on two cor.secutive fast start demandt - cause unknown (returned to service immediately, restarts successful).
79 057 6/6/79 Both EDGs failed to 'A' XX(2) hold a load due to '
& *B' procedural deficiencies in SP-755; repair within 5 & 6.5 hrs, respectively.
79 069 7/24/79 EDG 'B' failed during 'B' X test due to fire in exhaust manifold; repaired within 40:30 hrs.
i M
4 88 Table 2 1 -
(continued)
Crystal River EDG Failure History
~
LER/
NCOR Event No. Date Occurrence /Cause EDG FS FL/R 79 108 12/21/79 EDG 'B' inoperable due 'B' X to failure of standby cooling pump bearings (DJP 4); repaired in 6:40 hrs.
80 030 7/31/80 F 'O' failed to run 'B' X due to separation of the '-
turbocharger discharge duct work due to sheared bolts en the mounting brackets. Repair within 94:30 hrs.
80 032 8/5/80 EDG 'A' failed while running 'A' X during loading due to a over current relay which was out of calibration. No repair time reported.
80 046 10/16/80 EDG 'B' failed to start 'B' X due to unknown causes.
Troubleshooting took 14 hrs.
81 024 4/14/81 EDG 'A' failed during test "A" X due to high crankcase pressure alarm caused by a plugged oil separator in the air ejector train Repaired within 41:30 hrs.
81 030 5/26/81 EDG 'B' shutdown froe "B' X and X the ready condition
- during routine testing.
Subsequent restart attempt resulted in failure cause undetermined.
2-4
e .
Table 2-1 (continued)
Crystal River EDG Failure History
- LER/
NCOR Event No. Date Occurrence /Cause EDG FS FL/R !
6/16/81 EDG 'B' failed to start 'B' X when offsite power was lost due to a maladjusted timing relay in the start-ing circuit. Repaired within 19 hrs.
81 077 12/4/81 EDG 'B' failed to start 'B'.. X due to a grounded control circuit. Repaired within 7 hrs.
, 1/25/82 EDG 'A' failed to excite 'A' X and maintain output voltage.
No cause was found. Repaired -
within 6 hrs.
82 012 2/26/82 EDG 'A' failed to start 'A' X in a post maintenance test due to restrictions in the air start lines, filters and strainers. Repaired within 0:30 hrs.
83 051 11/2/83 EDG 'B' failed to run due 'B' X to worn insulation on control power wire.
NCOR -
83 321 11/10/83 EDG 'A' failed to load due 'A' X to improperly ' racked in' breaker.
NCOR -
85 0168 8/15/85 EDG 'A' failed to run due to 'A' X dead bus timer failure.
i NCOR -
i 86 0010 1/15/86 EDG 'A' failed to start 'A' X within 10 seconds. Cause is unknown. Restarted successfully.
25
E FIGURE 2-1 CRYSTAL RIVER EDG FAILURE HISTORY TIMELINE 5
.- 1 :
.h.
I ; e -
- s; 55 5 #
- 5 $a
- a a ~
.I j_5 5 a, a
, . . . .! 8.
A , r b s
.q.
E k Y
- 1977 1978 1979 1980 1981 I
E " Y
' Y-aa a
.% * .E *
. M 1982 1983 1984
- 1985 1986
~
1987
r . The EDG performance at Crystal River has improved dramatically in the last 5 years as compared to the fi st five years of operation. A review of failure causes found that there were no significant recurrent failure modes which had been eliminated, but, rather, that the improvement in EDG reliability had been the result of the elimination of a number of apparently independent failure modes. The most significant improvement in EDG reliability appears to have occurred in the early 1982 timeframe, just after the EDG Task Force. Prir.r to 1982, the diesel generators had consistently experienced three to five failures per year in testing. Since 1982, Crystal River has never experienced e re than two EDG failures in a single year. Further, since 1984, no more than one failure has been observed in a single year. In addition to the improvement in observed EDG reliability, discussions with plant personnel indicated a 'much higher level of confidence in EDG reliability on the part of the operating staff. 2.2 SUPHARY OF CRYSTAL RIVER EDG IMPROVEMENT ACTIONS In 1982, Crystal River acknowledged the observed low EDG reliability and formed a task force to investigate the EDG failure history, evaluate possible improvements and recomend changes to the design, operation and maintenance of the EDGs. Upon completion of their analysis, the task force developed a number of recomendations:
- 1) Increased training on proper maintenance of Fairbanks Morse diesels;
- 2) Dedication of an EDG Engineer to support problem identification and resolution;
- 3) Thorough review of EDG service manual and site procedures;
- 4) Initiation of daily visual inspections by Operations to identify any potential maintenance.needs;
- 5) EDG design changes were proposed in the following areas:
o Elimination of timina relays and incorooration of an automatic 11mino device to record start time. This change was intended to minimize the start failures related to apparent EDG failures to reach rated speed in required time due to manual timing inaccuracies. o Installation of screens on the turbo charaer inlets tg_grevent inoestion of foreion matter. While this change was not intended to explicitly address any past failures, it did serve to minimize the potential for turbocharger damage from foreign objects. 2-7
o Modification to the lube oil system to orovide lube oil flow sooner durina enoine startue. The intent of the change was to provide improved lubrication. Expected side benef ds were to minimize the potential for exhaust manifold fires and a reduction ir. the amount of smoke discharged to the diesel room during startup. o conversion of the air intake system charner to out the scavenaer
. flow nath in series with the ".urbo charae air flow. This -
modification was intended to 'ower the exhaust manifold j temperature by 300'F and was expected to reduce engine wear. f a Modification to fuel oil lines and injector nozzles to install j aasketless nozzles. This change is only beneficial to engines
- that operate continuously.
j o Lnsta11ation of a third oil rina on the too and botton nistons. i "his was intended to provide improved 01' control and reduce the amount of lube oil released to the exhaust system, thereby ~ reducing the risk of fires. ) All of the first four recossendations were implemented. of the six recossended ) - design changes only the first two were completely installed. The third was l partially installed, along with a lube oil heater modification to increase the lube oil temperature and improve the performance of the lube oil system during l i fast starts. The other three recossended design changes were never implemented, h I j This task force approach to resolving EDG reliability problems is consistent with the EPRI reliability program element termed ' Systematic Evaluation of Past Failure ! Modes' for the marginal performer and ' Study Past Failures' for the low performer. Several other changes to operating and maintenance practices have been implemented since 1982 which build on these basic recosmondations. First, since 1983 the I l diesels have been routinely barred after each operation. This practice, which consists of turning the engine over after the engine has been shutdown for a short f Prevent lube oil accumulation in the upper cylinders, . ' ten a t-mai. -:rs recmc ' t the manual barring of the engine was quite difficult. Beginning in 1983, manual barring was routinely performed after each operation of the diesels. In 1985, a design modification was installed to allow the use of the airstart system i r barring. A second change was in the training of plant maintenance personnel m a training diesel purchased by Florida Power Corporation in 1983. This allows hand: en training of plant mechanics on concepts of diesel operation and maintenance, t.8
i A third change came in the troubleshooting approach used by plant personnel in resolving identified problems. Beginning in 1983, a more structured problem solving approach was implemented utilizing Kepner Tregoe techniques. More recently. MORT analyses have also been used in resolving each plant NCOR/LER. These techniques provide higher assurance that problem root causes are identified i . and that problem close out is achieved. l
- A fourth operating practice change which may have impacted the observed EDG
! reliability involves the scheduling of EDG PMs. According to plant personnel, l since early 1M4, EDG PMs have been scheduled to be performed just prior to routine testing. This change was made to reduce the nusber of EDG start tests by j combining post maintenance tests with scheduled surveillance tests. Prior to l 1984, there had been no coordination of these activities. This practice, while ! effective in eliminailng vanecessary EDG start demands, could increase the l apparent EDG reliability by correcting failures before testing identified them, j - However, discussions with plant personnel indicated that LERs or NCORs would have i been written to identify this type of finding as a EDG failure, if any had - j occurred. i ! In 1985, Crystal River performed a Reliability Centered Maintenance (RCM) I evaluation of the diesel generators and their support, systems. The investigation performed as part of this study, combined with plant personnel input, resulted in a number of recomended changes to the Preventive Maintenance (PM) program to l l improve diesel reliability (1). These changes were implemented as part of-the Crystal River preventive maintenance program in early 1986. f The Crystal River RCM program resulted in the addition of a number of inspection tasks to identify early degradation of the governor, control circuitry, and the f i fuel oil and lube oil systems. I J Considered together, the 1982 Task Force and the 1985 RCM study incorporate a l
- number of the. principles of the systematic dominant failure mode evaluation process required of low reliability plants in the EPRI proposed program, however, they do not include all of the evaluation and analysis steps that would be required. Most notably, the Crystal River studies do not expiteitly consider postulated failure modes; i.e., those that could occur, but have not been experienced. The studies address only the past failure history of the Crystal River diesels in a systematic way, although some plant changes do address these potential failures, l 29
One significant feature of the Crystal River PM prograa is the performance of , extensive vibration and oil sample diagnostic measurements. The results of these analyses are utilized to trigger various condition directed tasks such as lubrication changes for the governor, generator bearings and fan drive gears. Condition directed PM activities such as these are very desirable forms of
. preventive maintenance because they seek to identify early degradation of plant equipment while minimizing unnecessary equipment tsardowns.
At present, the Crystal River EDG preventive maintenance program consists mainly of the following types of tasks: o vendor recomended inspections and teardowns o condition directed lubrication changes o RCM rscomended inspections of EDG components for general condition and for yibration induced and maintenance induced leakage and component looseness. Overall, these changes have resulted in a dramatic improvement in EDG reliability, with the most substantial improvement occurring shortly after the EDG Task Force completed its evaluation and increased accountability for EDG reliability was applied by Florida Power Corp. Since the implementation of the EDG RCM program recoamendations in early 1986, no EDG failures have been observed. Additionally, since that time, the EDGs have gone longer without a failure than at any other time in plant history (greater than 150 demands). It is difficult to assess the quantitative benefit of the RCM program due to the relatively reliable EDG . performance observed in the period just prior to the implementation of the RCH program recomendations (19821985). Based on the experience to date, the EDG Task Force and the RCM study, coupled with the existing Crystal River maintenance ' program, appear to have been effective in not only preventing the occurrence of tha dominant failure modes identified, but also in improving overall EDG l reliability. l I 1 2-10
b .. t i , t i 4
^
Section 3 APPLICATION OF EPRI PROGRAM ELEMENTS The actions taken at Crystal River to improve EDG reliability have many l similarities to the actions described in the EPRI proposed program. These actions I include the following: o f,ddressing experienced failure modes through procedural, operational j . or design changes. - i o EDG performance review by a plant team (task force and experience-based RCM). o laplementation of appitcable and effective PM tasks to address the ; identified dominant failure modes. , , l j The EPRI reliability program has several elements which were not part of the I Crystal River program. Each of the EPRI program elements not previously performed l at Crystal River was applied in an after the fact manner as part of this project. { These elements include: i
- 1) Application of EDG reliability triggers as a means to identify l potential unacceptable performance. ;
t
- 2) Review of important failures imediately after their occurrence to f identify key failure modes observed with potentially severe ;
consequences to ensure they are addressed. ;
- 3) Tracking of EDG corrective maintenance (CM) history.
- 4) Calculation of EDG unavailabilities, f I
- 5) Consideration of potential dominant failure modes not observed in i plant operating experience.
- 6) Calculation of expected reliability based on maintenance program i improvements. l
- 7) Documentation of all elements of the reliability program.
The following se1tions provide a discussion of the areas of difference between the Crystal River EDG reliability program and the EPRI proposed program. It should be i noted that, the incorporation of these elements in the EPRI program is due to the 31 __; !
. ,, -l I
fact that, in addition to being a reliability improvement program, the program is intended to be a regulatory and management tool. The fact that these elements were not performed at Crystal River is not an indication that the Crystal River actions were inadequate. 3.1 APPLICATION OF RELIABILITY TARGETS i One key element of the EPRI diesel reliability program is target reliabilities and the use of triggers based on the number of observed failures in the last 50 and 100 demand samples to initiate actions aimed at improving diesel generator maintenance practices. Tne EPRI proposed program suggests the use of reliability l triggers of 5 or more failures out of 50 demands and 8 or more failures out of 100 { demands as levels for action. These triggers were compared to the Crystal River j operating experience to determine whether some action would have been called for. i Precise data on Cr.'stti River EDG testing successes and failures was not readily ) available for yeart erfor to 1983. Based on the EDG test data reported in the RCH l program report (2), an average number of days between EDG tests was calculated (5.8 days between test). This data indicates somewhat more frequent testing than observed since 1983 (approximately 7 days between tests). This is l consistent with plant personnel statements that the frequency of testing was !
. changed in early 1984 due to improved scheduling of maintenance activities.
A susmary of the Crystal River EDG failure history in teres of the estimated number of demands between failures is presented in Table 31. This same data is plotted in Figures 3 1 and 3 2 in terms of the number of failures observed in a moving window of 50 and 100 demands, respectively. From these plots it can be seen that Crystal River would have exceeded the 50 demand trigger of 5 or more failures in 50 demands on four occasions (once each in 1977 and 1979, and twice in 1981). The 100 demand tr1<gger cf 8 or more failures in 100 demands would have beenexceededtwice(in1981). The 100 demand trigger exceedences wre coincident j with the last two 50 demand trigger exceedence and wre preceded by a 50 demand ; I trigger exceedence. It should be noted that Florida Power Corp. formed the task force to investigate EDG reliability problems in early 1982, shortly after the two coincident target exceedences. l In the EPRI proposed program, the exceedence of one trigger requires a review of f past failures and corrective actions to ensure EDG reliability is improved. The [
= .
. TABLE 3-1 Crystal River EDG Demand Between Failure History
, Estimated f Demands Since Failure f Q. gig Last Failure I 6/2/77 ...
2 7/26/77 6 4 3 9/28/77 11 4 2/27/77 16 l 5 1/3/78 1 ! 6,7 11/17/78 (2) 54,1 4,9 6/6/79 (2) 35,1 1 10 7/24/79 8 , 11 12/11/19 25 12 7/31/40 38 13 8/5/80 1 i 14 10/16/80 12 15 4/14/81 31 16,17 5/26/81 (2) 7,1 . 18 6/16/81 4 19 12/4/81 29 f i 20 1/25/82 9 l 21 2/26/82 . 6 22 11/7/83 107 l ' 23 ' 11/10/83 1 24 8/15/85 112 25 2/12/86 31 i 4 33
FICURE 3-1 CRYSTAL RIVER UNIT 3 NUMBER OF EDG FAILURES IN PAST 50 DEMANDS PER YEAR , PROPOSED EPRI TRIGGDt 4 P , e FAlt.tJRES 3 ' M PAST : So DOWdDS ! l i 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 vcui .
~~~=-~~.---.---mm- , _ _ . . . , _ _ _ __ _
a CRYSTAL I ER UNIT 3 NUMBER OF EDG FAILURES IN PAST 100 DEMANDS PER YEAR
, PROPOSED EPRI TRIGGER
( 9 0F , S DOWOS ~ 3 3 1 o . 3 1979 1980 1981 1982 1983 , 1984 1985 1986 1987 YEAR l _______________-.-------4-. . + - - - - - ~ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ' - - ' - - - - - - ' " - * '- - ' - ~ " " ' ^ - - - ' - - - - - ~ - - - - - - - - - - - - - - - - - - - - - - - - - ~ ~ ~
- exceedence of both triggers coincidentally requires the performance of a RCH upgrade of the plant PM program. The application of these triggers to the Crystal River data indicates that the 50 demand trigger can serve as an early warning to the 100 demand trigger, with less costly cchsequences since a full RCH program is only required when boL1 triggers are txceeded coincidentally.
It is concluded from this application, that the reliability triggers pros,osed in the EPRI program can be effective in providing graded response to diesel performance that needs improvement and that the actions required by the program are consistent with the actions taken by Crystal River. 3.2 'MPORTANT FAILURE REVIEW Crystal River plant established a task force in 1982 to review the observed EDG unreliability problem. The review performed by this task force was similar tc. although more comprehensive than, the EDG failure review recomended in the EPRI reliability program. However, since that task force comp 1eted its mission, the important failurn review aspect of the EPRI program has not been specifically performed at Crystal River. This consists of a review o' the EDG history to identify potentially important failures. As a test of this program element, this case study reviewed the Crystal River EDG failure history to identify important failures which have occurred during the plant 1tfe. Important failures are thor.e failures which have the potential to have a major impact on plant safety on operabilli,y. Examples of important failures include consoon cause fktlures which could prevent more than one diese from effectively operating and failures whose consequences are potentially severe (i.e. result in long EDG repair times). The Crystal River EDG failure history contained one case of apparent common cause fatlure due to an inadequate operating procedure. On June 6, 1979, during routine testing of the diesels, both diesels failed to carry a load due to procedural deficiencies in the surveillance test procedure. However, this failure would in,t have cccurred in the case of a real demand, as the steps of the surveillance procedure would not have been involved. This procedure has been revised to provide proper direction during surveillance testing. l A second potentially important failure involved a fire in the exhaust manifold of EDG 'B' on July 24, 1979. In this case, the fire only resulted in the premature 36
s . l- shutdown of the diesel during a routine test. However, in the event of a real demand, this failure could have lead to more severe consequences. This failure j
) mode has been addressed through barring after testing to reduce the amount of lube oil transferred to the exhaust manifold during startup. ,
- In both cases, the important failure modes were dealt with imediately by plant ,
personnel and corrective action implemented. This action was appropriate and ; 1 confirms the need to review t perating experience to ensure that important failure ' modes are addressed effectively, j
'i !
J 3.3 TRACKING 0F EDG CORRECTIVE MAINTENANCE HISTORY j The EPRI EDG reliability program directs the tracking of EDG corrective maintenance (CM) histories as a means to it ati'y potential problem areas and I provide a potential means for anessing the impact of implemented changes. I , . Crystal River CM data was reviewed in order to evaluate the potential ! l effectiveness of CM tracking in these two respects. . [ Because data for only th'e 1981 1987 period was readily available, it was not ! I possible to trend CM activity levels during the 1977-1980 period, during which EDG l reliability was significantly lower than it is today. The period reviewed does, ! however, include the interim during which a number of maintenance program changes I were made by maintenance following the EDG Task Force (1982 1984), and does i include the implementation of the RCM recommendations (1986). ! For the purposes of this evaluation, the EDG systems were divided into the . following sub systems: l o Engine (including Turbocharger and Blower) I-o Generator l ( o Cooling t j t o Fuel l o Lubrication j i i o Intake and Exhaust f o Starting Air ! t o I & C (including the governor) [ i 3-7 !
1 1 In order to effectively assess the CM history, it was necessary to adopt a consistent set of screening criteria to identify only those maintenance items which indicate a potential failure of the EDG safety functions (e.g., to start, l . load and run within its required time interval). Maintenance items that were l rejected include the following: o items relating to, or done in support of, design change work, o preventive maintenance tasks, o work performed to investigate possible degradation (e.g., due to a vendor recom.endation or NRC requirement) but for which no degradation was observed, o failures of components (e.g., local gauges) that;would not affect EDG start, load, and run capability. Those failures that were included as safety function failures were then classified as incipient or catastrophic failures. Incipient failures are those failures which were judged to be likely to degrade EDG performance, including such items as out-of calibration control equipment (which might have hi,,;.?*c ZDG operability if the controls worn allowed to fall significantly out of calibration) and fuel oil and lube oil leaks (that could cause a fire if oil leaked onto het components or could hinder EDG operation if the leak worsened significantly). Catastrophic
. failures resulted in the complete loss of an important EDG component or the inoperability of the EDG, However, not all catastrophic failures would necessarily prevent EDG operation.
Figure 3 3 presents a graph of the total number of safety functional failures for all EDG systems. As can be seen, the annual number of such failures since lge4 has stabilized at a lower level than was experienced in previous years. It is during this period that many of the current PM tasks were implemented (e.g., the
~
oil and vibration analysis tasks, the RCM recommended PM tasks). The scheduling i of PM tasks was also changed during this period to coincide with surveillance testing. Hence, it appears that the most recent PM improvements and scheduling changes may have reduced the number of safety functional failures. Figures 3 4 and 3 5 present the number of cms related to safety function failures on a sub system basis. A number of these subsystems show an improving trend over time, particularly the cooling, fuel oil, starting air, and generator subsystems. The engine and intake / exhaust subsystems have had relatively few failures throughout the reviewed time period. The lubrication and 1&C subsystems appear to
.. An
s . have shown little or no improvement. The I&C subsystem, in fact, may be showing an increasing number of catastrophic failures over the last few years.
~
. As discussed above, the tracking of cms is intended to provide a means for .
identifying potential problem areas as indicated by increasing CM requirements. This approach allows proactive steps to be taken by the plant in the event significant trends are identified. In general, the results of the CM revivw show j that there has been improvement in the EDG CM occurrence rate as a result of the l various programmatic improvements made in the maintenance program. This ! improvement manifests itself not only through an overall reduction in cms. but i also through a reduction in the number of catastrophic failures in all subsystems, ! with the exception of Inc. The fact that I&C cms have shown a moderate increase l would make these failures candidates for evaluation by the plant, j I Several observations can be rade concerning the apparent effectiveness of the RCM j activities performed to date. The RCM recommendations sought to eliminate past ! causes of EDG failure. Those few catastrophic failures that have occurred sin'ce l 1985 have been failures for which no PM tasks have been established. Hence, they are either failure modes for which the plant elected to not develop PM tasks or f had not been previously considered. No catastrophic failures have occurred for , which PM tasks have been implemented. Hence, it appears that no failures hate t occurred that the RCM program sought to prevent. Because the vast majority of cms during the period of 1985 to 1987 were generated f due to incipient failures, it is difficult to assess whether the RCM program has l been effective in dealing with these failures. One compilcating factor is that ! the implementation of the RCM indicated inspection tasks might increase the number > of reported incipient failures, since plant personnel would be devoting more time i to inspecting and discovering incipient failures. The proposed EPRI program suggests that cms relating to EDG failures be tracked. l A review of the limited Crystal Rivtr data found that this measure did not j indicate discernible trends on a subsystem or overall basis. However, because of i the relatively short time since impleaentation, little data exists for the time ; period since the implementation of RCN snd also, no data was reviewed for the time : period during the low EDG reliability (16771981). This makes it difficult to ! draw strong conclusions on the effectiveness of tracking the total number of cms. l l n-n _i
FIGURE 3-3 CRYSTAL RIVER EDG CM HISTORY TOTAL SAFETY FUNCTIONAL FAILURES s-- 5 CATASTROPHIC E INCIPIENT me-- l e
"MME ""e"!?n" is- -
s- - e 1 I i i ! i I
.?
YEAR l I _ _ .
_ __ . _ _ _ . _ _ . . _ . - . _ _ _._.-_________-_.______.__m__ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ , _ _ . _ . . _ _ . _ _ _ . _ _ . , _ _
. l l l
,; g, ,! =
l 1 I I 2 s 3
+ = a r
,a le x 1 . ;
I h I E 1 El ,, > h i l i i ! g
. i 4 i . . . i eii... i 6 . . . . i i iei. 4 s
!ig l i l!l li
=
W
- d' h
i ! 4 ; I ! E t i' i
'll $ l 2.
1
, 8.
1 W, l li k' I p o - P suo : ; i ! l .. I ' I I 38 ' E l
, m, >
l 1 ll 5 e i5 5 5 5 5 5 e .
- 4 i e ie 5 6 5 5 5 5 5
- d B 8..
h II I Is [.s h l t 3 11 !
1 i . i
. al . , 8 .l I j i I
'I .
5 i il ! llI il i .. I ., . i i i E r e i 2
- a 1 '
E I m i i g
. W i.g! 1lg 5.1 ~2 111 b
4 E
.g
, u gi m I e 6 .l I 6. i
.. .. t, i ll if I f
!l El ~
.e E'3 I l',
I's !j II ' .;. E' .. i
}l l I
,i2 . ; ; ; i i .-
ill! illi 3-12 _ - - - - - - - - - - _ _ _ _ _ _ _ - . _ i
s . This investigation does indicate that there is f.ome benefit to the tracking of cms related to safety function failures as part of the EPRI program.
, 3.4 CALCULATION 0F EDG UNAVAILABILITIES i The EPRI proposed reliability program includes the calculation of EDG maintenance l
unavailabilities for all plants. This dat4 is already kept by utilities and j reported regularly to INP0. Crystal River EDG maintenance unavailability data was ! reviewed for 1986 and 1987. In 1984, a total of 86 out of service hours were reported for the diesel generators. This translates to a unit average
- unavailability of less than 0.55 (86/2x8760). In 1987, no out of service hours
- were reported.
l This data indicates a very low unavailability contributio from EDGs being out of-l service during a period where EDG reliability was high. Without data from past ] , years when EDG reliability was lower, it is not possible to draw conclusions ! regarding potential relationships between EDG reliability and unavailability.- J 3.5 COMPARIS0N Of SYSTEMATIC EVALUATION PROGRAM CONTENT For low reliability plants, the EPRI proposed process for improving EDG ! reliability utilizes a systematic engineering approach to identify the dominant failure modes of the diesels. These dominant failure modes are made up of both l observed and potenti.a1 failure modes (i.e., both those that have occurred and those which are relatively likely to occur). Upon completion of the evaluation, actions recossended to address those failure modes are considered for l implementation by plant staff. The decision on how many or what changes are j necessary to meet the plant reliability target is left to the discretion of the ! plant staff, with the recossendations from the systematic evaluation process serving as one input to the decision making process. In the case of Crystal River, the plant elected to implement all of the.PM program changes recommended 7 from the systematic evaluation perfonud in their RCM program. l The RCM program undertaken at Crystal River has many features cossen with the requirements proposed in the EPRI diesel reliability program. For example, the Crystal River program included a thorough review of the plant EDG testing and failure history, a review of existing preventive maintenance actions, and identification of applicable and effective preventive maintenance tasks which would address the observed failures. The early investigations in support of the Crystal River RCM program also included an investigation of failures which have 3-13
, occurred on Fairbanks Morse diesels at other plants, but only in the context of how Crystal River experience compared to other plants. No specific actions were i recomended as part of the RCM program which addressed any failures beyond those
. observed in the plant operating experience.
The EPRI proposed program includes the evaluation of both observed and potential dominant failure modes. This is accomplished through a systematic evaluation of EDG system functions and potential functional failures. Other EDG reliability studies (Lid) have also considered potential failures and found failure modes a R ver C r h en ee ob ve R p 1 ation In this case, including potential failures in the systematic evaluation process l would have been conservative. ! 3.6 EXPECTED RELIABILITY APPLICATION ) Another element of the EPRI diesel reliability program is the calculation of ' expected reliability. The calculation of expected reliability involves an i estimation of the impacts of specific plant maintenance, operational, or design changes on the performance of the EDGs. The benefit of expected reliability in an industry diesel generator reliability program is that it provides the NRC with
- some method to assess the effectiveness of utility improvements until these improvements can be demonstrated in terms of EDG testing performance. The benefit expected reliability gives to utilities is that it provides a means to show that improvements have been made without having to initiate accelerated testing to provide data.
The EPRI proposed program guidelines provide general guidance on a method of calculating expected reliability. The basis for the calculation of expected reliability is the assignment of individual confidences for the elimination of the obser. M failure modes. That is, a subjective judgment is made as to the expected impact of the imprevements on each failure mode. This judgment is made based upon I the available information at the time combined with good engineering judgment. Since this estimation is based on existing knowledge, and the amount o* quality of information available can change after the estimation, some uncertainties are accepted. The intent of the expected reliability is solely to provide a best judgment as to the reliability improvement which could be expected. ______.____-____A.AA_
e . This guidance was applied to the failure history of the Crystal River EDGs. Each observed failure was categorized as having been completely eliminated (via design change), affected with high, moderate, or low confidence, or unaffected by the
. improvements which have been made during the plant life. The results of this review are as follows:
Number of Confidence Effective Failures in Eliminating Number of Failures I Addressed Failure Mode "Eliminated' 0 DESIGN CHANGES 7 HIGH (90%) 6.3 8 MODERATE (50%) 4,0 2 1.0W (10%) 0.2 8 NO CONFIDENCE --- Total Failures ' Eliminated" - 10.5 Of the 8 failures assigned "No Confidence", 7 were due to unknown failure causes. The estimated total number of EDG demands since plant startup is approximately 660. The expected reliability is therefore: Expected Reliability = ,0.978 660 l This value compares favorably with the calculated EDG reliability between 1983 l and 1987 of 0.986. This test application of the expected reliability calculation demonstrates the potential viabliity of this method in providing a i measure of the effectiveness of maintenance program changes. l - ! 3.7 DOCUMENTATION INPUTS AND REQUIREMENTS ! Based on the relative ease of data gathering at Crystal River, it appears that the input required to develop the information and documentation involved in the EPRI proposed reliability program is readily available. For recent years, a number of key data requirements are already gathered and interpreted for other uses (i.e., INPO reporting). Other data, such as CM histories, are already computerized at Crystal River providing easy access. Thus, it appears that the information management systems required for this program are already in existence and easily accessed. 3-15
The EPRI program includes the reduction, interpretation and documentation of a number of data inputs. The formal documentation of these data appears to be relatively straightforward, given the accessibility of data. Discussions with Crystal River plant personnel confirmed that the accessibility of da.a was good and that the documentation req'airements would not be excessive. e S 6 _ _ _.. _ O. _ b O
d . j f c
?
{ l j . Section 4 j SLM4ARY AND CONCLUSIONS
, f 4.1 SLM4ARY OF CRYSTAL RIVER EXPERIENCE f
- I i The EDG reliability history at Crystal River Unit 3 demonstrates the effectiveness !
l of implementing relatively minor plant change in improving EDG reliability.
' l During the course of the past eight years, EDG unreliability at Crystal River has !
l decreased over 805 from approximately 5.5% in 1980 1 to less than 15 in 1986 7. j Further, since the implementation of the EDG RCM program in early 1986, the EDGs l 5 have experienced their longest period between failures in plant history (greater ! s
- than 28 months, greater than 150 dt3 ands). ;
) ' J Probably the most telling indicator o' the EDG reliability improvement observed at ' i Crystal River is the expressed confidince of the operating staff in EDG ! i reliability. During the plant visit, a number of operating personnel expressed ; their increased level of confidence in their EDGs as compared to the low level of l confidence described in past years. I i L During the operating Ilfe of the plant, no major hardware changes were made which j could account for such a dramatic improvement in EDG performance. These observed , improvements are therefore attributable to changes made to improve procedural l direction and maintenance practices in combir.ation with some minor design changes. f This finding is in complete support of under131ng principles of the EPRI [
. reliability program (i.e., reliability growth is expected through improvements in operating practices resulting from investigaticns of observed and potential l
failure modes), i 4.2 EPRI PROGRAM APPLICATION ( The EDG Task Force and the RCM program implemented at Crystal River have many [ similarities to the actions identified in the EPRI proposed reliability program f for a plant whir.h experiences high unreliability. These includes a detailed review of FCG failure history and implementation of preventive maintenance tasks ! whiin could detect or prevent specific failure modes. Some differences do exist f 4-1 !
i in the scope of failure modes considered, should the systecatic evaluation of . dominant EDG failure modes be required, in that the Crystal River program ! addressed observed failures and the EPRI program would require addressing all l dominant failure modes, both potential and observed. '
. Other elements of the EPRI proposed program were also evaluated and found to be applicable including reliability triggers based on 50 and 100 demand histories, EDG failure reviews, expected reliability calculations and important failure reviews. A review of Crystal River CM history was also performed to determine if cms tracked with the observed reliability improvement. For the data reviewed, some potential trends were identified. Useful insights were gained from the CM history review and, for this reason, the tracking of cms as part of the EPRI reliability program is recossnended.
Based on this case study, the following elements of the EPRI proposed program have been evaluated and appear useful: o A systematic approach for the identification and evaluation of dominant EDG failure modes (in this case the EDG Task Force and the
' RCM study) as a means to identify improvements to an EDG preventive maintenance program.
o Reliability triggers based on observed EDG failures out of past 50 and 100 demands, o Review of EDG performance by a plant team (i.e., task force) to identify potential improvements. o Review of past failures to identify potentially important failures. o Calculation of expected reliability. Insufficient data was available to conclusively evaluate the following: o The effectiveness of tracking .the overall number of cms as a measure i of EDG reliability improvements. However, a refinement to the recommended tracking method was identified based on cms related to safety function failures. . o The importance of maintenance unavailabilities could not be assessed { due to lack of data. ! o The importance of including the consideration of potential failure j modes in the systematic evaluation process (i.e., those not observed
- in plant op rating history) could not be assessed due to the
! relatively srart time since implementation of the RCH recomendatic.is. 4-2
The EPRI proposed program provides flexibility which allows the plant to define the specific improvements to implement. Explicit compliance with the analysis outputs is not required (i.e., implementation of each and every recommended change
. is not required). The program acknowledges that angineering judgment plays a role in the identification and selection of changes. Further, because of uncertainties in the data available to make judgments, it is acknowledged that the expected reliability calculation cannot be used after the fact to judge the appropriateness of these judgments.
Overall, the experience at Crystal River is juc' ped to provide support to the applicability and cost effectiveness of the principles and elements incorporated in the EPRI proposed Emergency Diesel Generator Rt11ab111ty Program Guidelines. Of equal importance, this case study indicates the value of a flexible, graded reliability program. More extensive programs requ"ring broad programmatic changes i
~
are not justified, based on the Crystal River experience. Further, less flexible i programs requiring plants to make specific programmitic changes, would be overly l i restrictive and might not allow the application of plant insights as effectivily. ' Finally, in this assessment of the EPRI program, t.o activities were identified that would cause a utility to incur significant additional cost, beyond those that the utility deemed necessary to address observed unreliability problems. I , i ; l l t I l 43
s . Sectiotr 5 REFERENCES
- 1. Guidelines and Technical Bases for NUMARC Initiatives Addressing Station Blackout at Light Water Reactors', NUMRC 87 00, October 19, 1987.
- 2. Draft Reg. Guide 1.155. ' Station Blackout *, USNRC, September 25, 1987.
~
- 3. ' Reliability Centered Maintenece Program For the Emergency Diesel Generator Units, Crystal River Unit 3, Sub'ask 2.6 Task Report on Feasibility of Reliability Centered Maintenance *, AT Engineering Systems, Inc., February.
. 1936.
- 4. 'A Reliability Program for Emergency Diesel Generators at Nuclear Power :
Plants', NUREG CR 5078, April, 1988.
- 5. "Trial Application of Reliability lechnology To Emergency Diesel Generators at !
. Trojan'. A 3282, Brookhaven National Laboratory, April 15, 1986. l l
I t I i 51
Appendix A
' EPRI PROPOSED EDG RELIABILITY PROGRAM GUIDELINES (As of October 1988)
A.1 INTRODUCTION During the past several years, considerable effort has been expended by both the NRC and the nuclear power industry on the investigation of the risks due to station blackout. The term ' station blackout' refers to the total loss of AC power to the essential and non essentiti nitchgear buses in a nuclear power plant. An event of this type involves the concurrent loss of both offsite power supplies and on site emergency AC power sources (i.e. emergency diesel
. generators).
The NRC proposed resolution to the station blackout issue is based on risk analyses. Most of the NRC's technical background work on the blackout issue is set forth in NUREG 1032. While recognizing that, on the whole, emergency diesel generator (EDG) reliability is acceptably high, there is concern that some plants
. have marginal EDG performance. Generic Issue B 56 ' Emergency Diesel Generator Reliability', is the NRC program addressing matters related to EDG reliability.
Both industry and the NRC have concluded that there needs to be a somewhat more formalized industry wide approach to assuring EDG reliability than presently exists. NUMARC 87 00, ' Guidelines and Technical Bases for NUMARC Initiatives Addressing station els ekout At Light Water Reactors * (October it,1987 draft), identiftes major elemnts of an EDG reliability program. A recently issued (September 15, 1987) NRC sponsored study for Sandia National Laboratories, ' Diesel Generator Reliability Program Review Guidelines', also offers views on the makeup of an EDG reliability program. EPRI and ERIN Engineering and Research, Inc. have sought to determine what kinds of activities might help a nuclear plant achieve high EDG reliability. The purpose of this effort was to determine the desirable elements of an EDG reliability program and evaluate a systematic approach to the identification of A-1
e dominant EDG failure modes as a means to improve EDG reliability. Sections !! and ;
!!! present an outline of this proposed program.
I A.2 EDG RELIABILITY PROGRAM NUMARC 87 00 identifies five elements of an EDG reliability program. These ! elements are the same as those identified by the NRC in Regulatory Guide 1.155: (1) Establishment of individual EDG tarcet reliability levels consistent with the plant category and coping duration. 4 (2) Surveillance testina and reliability monitorina nrearaan designed to
) track EDG performance and also support maintenance activities.
4 t (3) A maintenance nrocram which ensures that the target EDG reliability is being achtrived and which also provides a capability for failure
] analysis and root cause investigations, j
~ (4) An information and data collection system capability which services i the elements of the reliabiitty program, and which monitors achieved-l EDG reliability levels against target values.
i (5) Identified reinonsibilities for the major program elements and a j management oversight program for reviewing re lability levels being y i achieved and assuring that the program is functioning properly, l j , Each of these five elements is considered in relation to applying the systematic l dominant failure mode evaluation processes to EDGs. The following sections i describe the insights gained in this investigation as they relate to each of these five elements. l A.2.1 Reliability Tarnets i The desirability of having some type of reliability target is obvious, plants j must have something to measure their EOG perfomance against. The NUMARC 87 00 ) document outlines a procedure for determining the required coping capability of l each plant. Several key plant specific features influence this determination: l j o Site weather characteristics 1 l o Offsite power supply configuration
- o Onsite AC power supply configuration l o EDG reliability 1
I t e
In general, plants are allowed, based on past reliability experience, to classify themselves into one of two EDG reliability goal categories. These are: 0.95 per demand, or 0.975 per demand (based on the average nuclear unit EDG reliability).
. These overall targats (0.95 or 0.975) are to be used as long tern EDG reliability targets.
In assessing EDG reliability against these targets, care must be taken in evaluating the data. In the relatively small sample sizes (20 or so demands), it is difficult to draw conclusions regarding EDG performance. That is, one failure out of 20 results in a short tern reliability of less than 0.975, but it does not necessarily mean that the EDG system is suddenly unreliable. Likewise, no falleres in 20 demands does not mean that the EDGs are 1005 reliable. This is due to the statistital uncertainties associated with relatively small sample sizes. I i Very large sample sizes can also have undesirable aspects. If the data used in ! the evaluation reaches too far into the past, it may present a picture that is no l longer valid. For example, if aging is beginning to cause a rise in failure
- I rates, the calculated reliability from a large sample size (i.e. years of demands) might be more favorable than currently exists. Likewise, if the EDG reliability
- has been substantially improved, a large sample size eight indicate a n11 ability that is lower than curnntly exists. It is therefore important to assess EDG reliability in light of system changes made during the sample period.
One important consideration is whether to use individual EDG reliabilities er j average unit EDG reliability as the tracking measure and a criterion for requiring ( improvement. Currently, utilities prefer using individual EDG re11 abilities ! because of requinments to perform accelerated testing of poor perferning EDGs. ! It is also recognized that some units have dissimilar diesels with different { , functions and service histories (such as SWR plants with diffemnt EDGs for NK5 l and other station systems). In such cases, performance of one diesel may not be very useful in evaluating the other diesels. I On the other hand, use of individual EDG n11 abilities as an indicator requires either small sample sizes or data that nach far into the past. The use of either of these data sets presents drawbacks due to the nlatively low fnquency of EDG testing (once per month). In order to assess the performance of an individual EDG over the last 100 demands, several years of data must be used. It is possible that at least some of the older data would be invalidated by major maintenance activities such as overhauls or substantial changes in the maintenance program. A3
Sit.e most plants have identical EDGs eithin each unit, serviced under identical procedures and supported by the same support systems, it is reasonable to assume that the average unit EDG reliability wouldyrovide a fair representation of unit
. EDG reliability. Also, changes in EDG reliability programs will most likely be j made to all EDGs rather than to just a single poor performer. The use of unit average EDG reliability also allows the use of larger sample sizes with more recent data. There are anywhere from 2 to 4 times as many starts on a unit basis as there are on an EDG basis.
For these reasons, the average nuclear unit EDG reliability would be a good performers measure for comparison against target values. Individual EDG reliabilities would be tracked, but only to allow the flagging of individual poor performances. For regulatory purposes, average unit EDG reliability would be reported and compared against target values. Individual EDG reliabilities would be tracked internally by utilities. The determination of appropriate sample site is a very complex problem. On one hand, small sample sizes are desired to allow tracking of trends which might identify future unreliability excursions. On the other hand, one is driven toward larger sample sizes to avoid the high false alarm rate associated with the uncertainties in small sample sizes. i The use of an average unit EDG reliability overcomes some of this problem by allowing the use of a larger pool of data. However, the use of small sample sizes ; (20 demands) using average unit EDG performance still poses the risk of false al arus. For this reason, it is recoassended that only the average unit reliability j for the last 50 and 100 demands be used for regulatory purposes in the evaluation . l of EDG performance. Tracking of smaller sample sizes and individual EDGs should be performed by utilities only as internal measures of their reliability programs. i l Developing a technical basis for reliability targets is essential to the l evaluation of EDG reliability targets. The required EDG reliability target of 0.95 or 0.975 is well defined in NUMARC 87 00 ar.d Regulatory Guide 1.155. There ! is, however, insufficient technical basis for comparison of EDG test data to these reliability targets. Before EDG performance can be measured effectively, more l detailed technical justification for the target reliabilities based on 50 and 100 ! demands should N developed. A4
1 Uncertainty in these small sample sizes makes direct translation of overall target re11 abilities impractical. That is, to require a reliability target of only two failures in 50 demands for a 0.95 reliability plant would be overly restrictive J , and would result in a high false alarm ratt. Likewise, raising the target , reliability for 50 demands to five failures or less to reduce the false alars rate might decrease the significance of an excursion to the point where the target is meaningless. Thus, in association with technical bases for the 50 and 100 demand
- targets, guidance on interpretation of data versus the targets is needed. .
Due to the large uncertainties and difficulties in establishing meaningful target i reliabilities based on these small sample sizes, it is, therefore, important that the overall reliability program base actions on the sample sizes with the least
~
I uncertainty. For this reason, no specific actions are recoeveended based solely on an observed unreliability trend for a 20 demand sample size. The 50 and 100 demand bases should be used to trigger program actions. l { J A.2.2 Testina and Monitorina Procrams - 4 1
- All plants currently have EDG surveillance and testing programs in place.
l However, the content of these programs varies from plant to plant. It would seem that one desirable outcome of resolving the diesel generator reliability 1: sue i . would be to bring greater plant to plant uniformity to the guidelines for EDG j testing and monitoring. l 1 NUMARC 87 00 makes note of this need and includes very useful guidance. Of j particular importance are the definitions of ' demand,' ' failure,' and "success.' l } For example, if excessive primping of an EDG is performed before a test demand, e ] this could invalidate the usefulness of the test in indicating the real ,
) reliability of the EDG since in an unplanned demand, the prisping would not have l l occurred. Likewise, defeating interlocks and shutdown cirm :ts not necessary for l l testing could invalidate a demand. The definitions of ' success' and "failure" (
j given in NSAC 104 are generally accepted by the industry, but should be l I implemented industry wide to ensure consistency in reporting. ( l l
, A.2.3 Maintenance Procrans l 1 A maintenance program aimed at improving EDG reliability can help plants with '
marginal or low average unit EDG reliabilities. For plants with acceptably high l EDG reliabilities such a program should not be necessary. With this the case, it i l i i A5 l
would seem desirable to develop a graded program structure that is suitable for a ' range of plant EDG reliabilities. ! l
. The graded response program would require a varying level of response by the ;
utility, depending on the observed EDG reliability over the last 50 and 100 ' demands. Proposed targets for EDG reliability are as follows:
, Acceptable Number of Failures in Last NUMARC 87 00 Target Reliability 50 Demands 100 Demands 0.95 <5 <8 i ) 0.975 <3 <5 Plants whose observed EDG reliability has boon within the acceptable range for both the last 50 and 100 demands would be considered to have an acceptably high reliability. Plants whose observed EDG performancs has exceeded the acceptable number of failures in the last 50 or 100 demar3ds would be considered to have i marginal performance. Plants exceeding the acceptable performance standards for both the last 50 and 100 demands would be considered low reliability plants. The kinds of maintenance programs associated with each of these performance i
designations (high, marginal, and low) are described in the next sections. Hiah Reliability Plants l l 1 For plants where EDG reliability is acceptably high for the last 50 and 100
. demands, the existing EDG maintenance program can be assumed to be adequate.
These plants would continue with the programs they have in place while continuing to track EDG re11 abilities through the testing and monitoring program. , Additionally, an analysis would be made of each EDG failure to determine the [ following information: i I (1) the comporent causing the failure (2) the failure mode of the component (i.e. broken, siscalibrated, etc.) :
- (3) the possible causes of a component's malfunction (i.e. age,
! installation, human error, etc.) l l l A6
i I I (4) the nature of the failure (i.e., time dependent or demand related). i !
! The failure analysis need not be a detalle+ root cause analysis, but should be of !
sufficient detail to identify possible failure causes for inclusion in a plant l l failure cause data base. These data would be collected for each failure, even for
' l l plants having acceptably high average reliabilities. A qualitative evaluation of
( l the entire data base would be made on a regular basis to identify potential l l generic problems. For a plant with a target EDG reliability of 0.95 and three j (DGs, this would require no more than seven failure analyses per year, if the target reliability was met. For most plants, it would mean one or two feilures , l analyses per year. t 1 i i Failure modes that have the potential to cause the failure of multiple (DGs (i.e. ! I comon cause failures) would be identified immediately and addressed in the EDG !
, maintenance program. Comon cause failures are of'particular importance in EDG systems due to the similar nature of the subsystems. In general, the EDG systems j are virtually identical to each other and are maintained using identical j procedures. This situation is susceptible to common cause failures, i
l As observed in the review of the plant specific data, maintenance unavailability i can be an important contributor to the overall unavailability of EDGs. For this reason, it is recomended that maintenance unavailability information that is l collected as part of the INp0 performance parameter program be used internally by I utt11 ties to assess the effectiveness of their EDG maintenance activities, i i Itareinal Reliability Plants l { for plants with marginal EDs performance, (i.e. less than the target reliability for the last 50 or 100 demands), a review of past failures is recommended. This review would include evaluation of failure data on both an (DG and plant basis with the intent to identify possible commonalities among the failures. For example, if one EDG were experiencing a particularly high failure rate in its lube etl systes, some further investigation of that subsystem would be warranted. If a 1arge number of of scalibrations were observed in the EDG instrumentation, a review { of calibration techniques and equipment would be required. The failure data review could be based on the engineering judgment of a team familiar with the EDG systems and their maintenance. A key ingredient in the success of such a program would be the availability of data on past failures, e
l This is why the collection of failure acde data is desirable for all plants. , The .'; M data review should specifically examine recent failures that have ce ' W 11 ant to be classified as marginal. If any two or more of these ht ua d related causes (i.e., poor maintenance practices, same component
~
9.. N ., etc.), corrective action should be taken. Additionally, the tracking af corrective maintenance actions (cms) should be initiated. A review of EDG CHs might give a clue to a generic problem or to some . manner of degradation. It might also indicate how maintenance can be improved. The tracking of CM) would be for a utility's internal purposes only. Low Reliability Plants , For plants with low average EDG reliability (i.e. below target values for past 50 and 100 demands), a program to systematically evaluate dominant EDG failure modes
- is advisable. A number of potential analysis methods are available including:
o Reliability Centered Maintenance Analysis. o Failure modes and effects analysis coupled with a logic tree analysis. o Detailed fault tree or GO analysis.
! o Other systematic methodologies such as expert review sessions.
In order to illustrate the type of steps required, this report describes how this j would be accomplished utilizing a reliability centered maintenance approach. Reitability-Centered Maintenance (RCM) is a systematic methodology for determining l what maintenance tasts are necessary, prioritized on safety needs, reliability and
; cost. With RCM, maintenance is based on failure experience and fatture importance. The key features of the process ares (1) define the plant systems i
(2) identify its functions: (3) identify functional failuress (4) determine dominant failure modes .or functional failures (5) determine criticality of testnant failure modest and, (6) identify preventive maintenance tasks. Pilot applications to other nuclear plant systems are also described. The key output of an RCM evaluation is a set of Preventive Maintenance tasks, l 1 whether condition directed or time directed. The intent is to provide confidence l that previously experienced or anticipated failure modes will be avoided, or reduced in frequency. for those failure modas which are not evident and for which no applicable and effective PM tasks exist, the plant would undertake s. failure l A8
finding task. The products of an RCM program would include a set of appropriate PM tasks and an understanding of dominant failure modes gnuped by their disposition (i.e., managed by PM tasks or accepted as infrequent or tolerable).
. The RCM program would also include a consideration of important failure modes which have not yet occurred but which could be addressed by preventive maintenance or testing.
If an RCM program has been implemented to improve EDG reliability, any failures should be investigated to determine whether they should be managed or accepted. Where a manageable failure occurs, the RCM program should be re evaluated to determine what changes in the program are necessary to prevent recurrence of the failure. If the failure is classified as an accepted fatture, careful consideration should be given to the expected failure frequency for that mode to ensure it will not adversely affect the overall EDG reitability. If the failure is classified as new (i.e., not previously identified), then an evaluation would be performed to determine possible changes to the reliability program. . If, after trial operation under an RCM based maintenance program, EDG reliability does not improve, the PM program and RCM analyses would be re evaluated to ensure that the scope is adequate and that the specified PMs are applicable and effectin in preventing the observed failures. One measure of the effectiveness of an RCM program is the reduc.tton in the number of EUG system cms. An effective RCM program should not only improve overall EDG reliability, but should also eliminati other EDG failure modes (i.e., catastrophic, degraded and incipient). Each CM can be classified into one of the three fail 9re classifications: accepted, managed or new. Tracking the number of failures in each of classification can also provide an indication of the effectiveness of an RCM program. The number of managed failures should decrease with time, the number of accepted failures should remain constant at a relatively low level, and the number of new failures should be relatively small. While other systematic methodologies could be uttitzed to accompitsh the idtntification and evaluation of dominant EDG failure modes, it is anticipated that the steps required would be similar for those approaches. _ -__ __ v
i i A.2.4 Data Collection and Monitorina Versus Tarcatn . I l Testing, monitoring and evaluation of EDG failures are essential elements of an
. EDG reliability program. For the purposes of regulatory reporting, the unit average EDG reliability should be tracked versus the reliability target. It b
. envisioned that these data would be provided as a running average of the las'4 50 ;
and 100 demands, and reported every quarter. ! l Plants which are classified as low or marginal on the basis of their quarterly l reporting of EDG reliability would evaluate their past performance through a systematic dominant failure mode evaluation program or evaluation of past failures l as described above. Utility management and the NRC would.need an objective measure of the expected value of changes to the reliability program as a result of I these evaluations. Procedures for calculating such a measure for unit average l I EDG reliability improvement, based on maintenance program changes, are given in l Appendix B. 4 l This method is presented for conceptual purposes and would require additional j technical bases in order to be usable. However, several important concepts are , ! presented in this method. The first is reliability growth. This method directly ! applies the concept of reliability growth. Review of EDG data from plants with - significant operating expertence indicates that the effects of reliability growth ! l can be significant. The second concept presented is that of confidence in the i effectiveness of the reliability program change. The confidence table given in ; l Appendix B uses preliminary estimates. In order to be effective, some ! j justification must be provided for the values used. With these justifications, j the concept will be much stronger. l l l J If a utility repeatedly is unsuccessful in improving EDG reliability, more strict ( requirements could be needed. ! ! The concept of calculating expected reliability provides a measure for program ! l taprovements and benefits both the utility and the regulator. For the utility. ! this concept provides a means to (1) account for improvements in their EDG l reliability progrant (2) gain a measure of the effectiveness of their reliability l program improvements; and (3) address and improve EDG performance. The latter l would take place over three quarters (i.e., one quarter to identify the problem. l l one quarter to develop a program and one quarter to yield results). For the j j regulator, the calculated measure of reliability program improvements proviries a l 1 , j A 10 l
documented, rev wvable measure of the response by a utility to identify and implement EDG reliability improvements.
. For internal monitoring purposes, utilities would t ack the ultability of each
- EDG and the unit average EDG reliability over the last 50 and 100 demands. This internal monitoring would be performed regularly and upon each failure to ensure
; timely identification of unreliability trends. The results of this tracking would l be compared to the technically based target for 50 and 100 demands.
j A.2.5 Identified Reinonsibilities and,Manacement oversiaht i I j The general area of management responsibilities and oversight in an EDG reliability program were not evaluated specifically in this investigation. ' i However, several of the program elements set forth above do support hanagement involverant. First, the development of the identified monitoring and tracking paramesers will provide a means to inform appropriate levels of sanagement of EDG l l performance. Additionally, review of the increased internal tracking perform'ed by I the utility would provide etrly warning of potential problems, before reaching the
- regulatory reporting level.
i [ Second, the calculation of the expected reliability improvements provides a means , for oversight of the reliability program improvements and a quantitative measure ! of their impact. l Third, the annual evaluation of the failure cause database would provide ! ! additional information to appropriate levels of management regarding the observed f I l failures, ! i l - A3 CCWCLtJS10NS A graded response EDG reliability program that takes advantage of reliability . growth concepts should allow nuclear plants to meet EDG reliability targets. For ! j plants with acceptable levels of performance, little change would be required. l l They would be asked to keep records on the internal monitoring and tracking of l ) failures, failure causes, and saintenance unavailability. ( t i f ! For marginal plants, a failure cause review would be recoamended to assess and l l correct any coamonalities in the observed failures. Additionally, suginal plants [ l would be asked to track and review E00 cms as a means of augmenting the failure ( i t ! . A Il
. o.
cause review. For unacceptable performance plants, a systematic analysts of the EDG system and maintenance program would be recotmended.
- For the purposes of regulatory reporting, an average of the individual EDG reliabilities for the nuclear unit during the last 50 and 100 dems. ads is recommended for indicating plant performance. The observed performance would be compared to a technically based target value developed frem the required 0.95 or 0.975 reliability deterstned in NUMARC 87 00.
For the purposes of management oversight and internal tracking, data would be cc11ected and evaluated on unit average and individual EDG reliabilities in the previous 50 and 100 demands. The internal reporti'ig interval should be regular and upon every failure to provide timely indication of potential reliability problems. Any plant which has implemented the reliability program actions associatad wi,th marginal or unacceptable plant EDG performance would be expected to continue those actions untti unit average EDG reliability exceeded the 50 and 100 demand timet reliabilities. Once an acceptable level of EDG reliability is demonstratr p.$., plant meets 50 and 100 denund targets), the uttitty could, if it desired, discontinue the actions that were neded because of poor performance. Plants that have taken actions in response to unacceptable or marginal EDG perforsence would evaluate an expected EDG reliablitty based on their recent performance statistics and the expected effects of maintenance or operational changes resulting from the reliab111ty program. This expected EDG reliability should demonstrate that reliability growth is expected and EDG performance cai be expected to equal or exceed the reliability target in the future, n_nn
.. +
i I l l i i APPENDIX 8
. METHODS FOR CALCULATING EXPECTED EDG RELIABILITY The purposo of this appendix is to describe methods, based on reliability growth principles, for quantitatively extrapolating the effectiveness of maintenance program changos in reducing system unreliability (due to the elimination, or a reduction in the frequency of specific failure modes). This quantification is influenced by several factors: -
- the number and frequency of accepted failures; the number and frequency of failures managed by the maintenanca program; and
- the impact of the maintenance program changes on the frequency of failures.
A description of the methods follows: ittL1: Classification of Failures For each failure, determine whether the failure :hould be classified as "accepted' or "managed'. Acespted failures are those which are of sufficiently low frequency that they are not addressed by the mainte. nance prcgram and are therefore allowed. l l l Managed failures are those which should be prevented (or decreased in frequency) l by a maintenance program. For plants with full systematic evaluations of dominant EDG failure modes, both potential and observed failure modes should be classified f l in this manner. For plants which have only studiwd past failures, each observed fa'. lure should be classified. l l Sten 2: Failure Impact Assessment For each failure, assess the impact of improvements in the maintenance program on the identified failure mode. Changes in the maintenance program can influence failure modes by decreasing their frequency of occurrence or eliminating the failure ude altogether. 8-1 _ _ _ _ _ _ _ . _ _ . .__
i .- An example of decreasing the frequency of failure would be to increase the procedural emphasis placed on restoring the equipment configuration at the end of a CM or PM task. This change could result in a lower frequency of human error of leaving systems in non-standar d configurations. The impact on frequency would be assessed by evaluating the expected impact of the changes. An example of a change that would eliminate a fall'ure mode, would be a procedure change to avoid an activity that is known to cause a failure. TABLE B 1 Individual Confidence Factor For Failure Modas
~
Confidsace In Impact of Influencing Failure Change on l Mode Reliability (Rg) F Confidence 0 ! (At ipted Failure) Low 0.25 Moderate 0.50 High 0.90 Design Change 1.0 For accepted failures, no impact is expected from maintencnce program changes, thus a value of 0 is used for Rj. For redesign to eliminate a failure mode, a value of 1.0 is used.
$ ten 3: Calculate Expected Reliability Two possible nefi,ed; have been developed for the calculation of exper.ed reliability. Ot.. is based en observed failures in the EDG system. The second method would be used in conjunction with an RCM evaluation. A brief description of each of these methods follows:
-_ b_A________._ _ _ _ _ _ - _ _ _ _ _ _ _ . . . _ _ _ _ _ _ . _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _
~-
Observed Failure Method This method calculates expected reliability using the individual confidence factors for each of the observed failure modes. The general form of the equation. is as follows: l N I1 h RE
= 1- I
[ (1 - Rg) 1 (0 ) i =1 Where RE is the expected reliability, D is the number of demands in the sample evaluated (i.e., in past year), N is the total number of failures observed, and Rg is the individual confidence in managing the ith failure from Table B-1. For example, assume that during the last year 100 demands were put on the EDGs at a unit. In those 100 demands, six failures were observed. The six failures were dispositioned as follows: Failure Diseosition By 1 Accepted 0 2 Minor procedural 0.50 modification, moderate confidence in eliminating Failure Disoosition By 3 Design change removed 1.0 failure mode 4 New maintenance procedure. 0.25 low confidence in any l impact 1 - , 5 New PM testing, high 0.90 ' confidence in improvement 1 6 Accepted 0 r ( I l B-3 <
For this case, the equation becomes:
/1)
RE- 1-1 l (1-0)+(1 .50)+(1-1)+(1 .25)+(1 .90)+(1 0) (100/
/1 )
= 1-1 l3.35 (100/
- 1-0.0335
- 0.966 ..
For a plant with a target reliability of 0.95, that expected reliability would be acceptable. For a plant with a target reliability of 0.975, that expected reliability would be unacceptable and further enhancements in their maintenance program would be necessary. T . g.4
l t i REVIEW OF CURRENT PRACTICES C0 4CER414G DI ES EL G ENERATOR R E_l ASILITY i I i B--56 Task Force l l .
- \
ENCI.DSURE 5, Ref. 11-2-8B [ NilNARC/EPRI/NRC Heeting SilB.IECT: EDG RELIABILITY I l 1 DEVONRUE
)
)
PURPOSE ESTABLISH A SENSE OF COMMON APPROACHES USED IN THE INDUSTRY TO MAINTAIN CURRENT LEVELS OF RELI ABILITY l l ESTABLISH WHICH ELEMENTS IN NUREG/CR-S078 ARE l NOT COMMONLY USED IN THE INDUSTRY l 9 DEVONRUE
3ACKGROUND The industry EDG reliability is high The probability of an "outlier" EDG with low reliability in a given year repeating as an "outlier" EDG with low reliability in the following year is the same probabliity as for any other non-outlier EDG for the year. No statistically significant correlation between EDG reliability and the following variables was found: (a) age of the EDG (b) sized of the EDG (c) Manufacturer of the EDG (d) single unit facility versus multi-unit facility DEVONRUE
't e s . t E, .f L' i,
.E' .' I (I.. :
T I 7 :
~
.L' &
E.'i.
* }'**l f
t i 2 i
, l f lI J-l.'I *.
l I3 f h l8
- l l3 l I ,
L ' ' ' I tl_ a l e s e a s i s e , . gg: ae ss: aae *
. . . . . . . . . . . . . . , e une f a
.
- it t
,3 W
- e
! D
.. ; .fI 's ; a 2, ! .T';
. Z O
fiI,g:#
- i
'i ' i >
I !. ! r I, LLJ
~ .} i cl i.
'l 1 O
.I sl 1
, , , , , , c,,,. J . . .. .. 'i -
p, e
.c ' :,
1 9
i 8 f" ..{
.s .!
- L.'3 : .I I P
,6 ,, &. l8 *
'I g !
F J,*
; 'I sj i !8 .
18 (J.
- : . l r il '
I i.s I , . . i sj
- l
- sgaa e3 3 I s & f .*
, , a e : , e e a e g e a s : a a e i nw. l'
' e e a " *"i s i ' *
. . g ,, {.,,,
.: , .' I
g ,1 :: :
f
- t .g .
1s I k
-L,,t . I: &, ',..e *!
g *
' ' ,{ 8 g
- b. e g
. I 18 s e - 5-
=
'.LJ I r
's l j.
is
,g j 9 -
g 8 ... . . } gg
'I . , gaaeeaea a : -
9.., e e e e a ia p e a ' s s ' no.
..........ii...
i l
~'
l ACTIONS GENERALLY COVEREJ l4 STATION PROCEDUR 4
- VENDOR RECOMMENDATIONS INCORPORATED IN MAINTENANCE PROCEDURES
- VALID START / FAILURE DEFINITIONS CONSISTENT WITH REGULATORY GUIDE 1.108
- EDG TESTING COVERED IN SURVEILLANCE PROCEDURES
- PROBLEM CLOSE0UT FOR NCR RELATED PROBLEMS DEVONRUE
l f . ( ACTIONS GENERALLY COVERED l 04 AN INFORMAL BASIS i
- COMMON CAUSE FAILURE ANALYSIS l
(PERFORMED DUE TO "GOOD" ENGINEERING PRACTICE)
- INDICATION OF REPAIR FREQUENCY I (WORK ORDERS REVIEWED BY COGNIZANT ENGINEER) j - DEDICATED MAINTENANCE CREWS ; DEVONRUE l .. _ ._ _.
ACTIOhS NOT GENERALLY COVERE) BY STATIOh PROCEDURES 1 - DEFINITION OF EDG SYSTEM BOUNDARY
- SETTING OF EDG RELI ABILITY TARGETS (IN PROCESS DUE TO ST ATION BLACKOUT)
- PERFORMANCE OF A FAILURE MODES AND EFFECTS
! ANALYSIS (FMEA)
~
l j - DEDICATED MANAGER OVERSEEING EDG RELI ABILITY l DEV0NRUE
m e
%LltiIdAh pg i
%10p Mok> TEED BW- u -- 2 -ff(
NRC DIESEL GENERATOR REGULATORY REQUIREMENTS ENCLOSURE 6, Ref. 11-2-88 Three levels are established: NUMARC/EPRI/NRC Meeting
SUBJECT:
EDG RELIABILITY e One failure in 20 demands - nominal caution leve'..
- Two failures in 20 demands - warning level.
- Three or more failures in 20 demands - reportirg to NRC level.
Three corresponding action levels are established:
- Caution level. Due to the certainty of one failure, it is not serious. The licensee is required to perform a root cause analysis of the failure, keep careful records of the underlying causes and what utility actions were taken to correct the failure and prevent reoccurrence. Caution level actions are not reported routinely to the NRC.
- Warning level. Two failures may, or may not, have significance depending upon the results of the failure analysis. The actions requirad are to duplicate the caution level actions but closeout of the actions is required within one month of the second failure .
event. Further, the licensee shall ensure that potential common cause failures are not involved. (Fuel oil aging and common fuel oil system and maintenance personnel caused failures are examples of common cause failures.) There is less regulatory concern for completely random and different failures where the problems are corrected and where the reoccurrence is unlikely to due to the actions taken. Diesel basic reliabilities should be between 95% and 100%. Statistically, the warning level is expected to occur every few years with 97% underlying D/G reliability. Warning levels are not reportable to the NRC unless the root cause cannot be determined or where comon cause failures are involved.
- Reporting level. Three failures in 20 demands is a reporting level of failures requiring a prompt NRC notification and a followup written report. A cumulative probability of 0.7 exists for 3 or more failures in 20 demands over a ten year period, for a diesel generator unit with a .97 underlying reliability. Thus, the evaluation of the failures by the licensee needs to be done partly to help the NRC evaluate the diesel generator reliability per se and the licensee's reliability program for differentiation between i false alarms and real safety issues.
The prompt notification should include a description the most recent failure location and apparent cause along with the previous failure causes and correctivs actions and a discussion of any relationship between the failures. The written report shall document the following:
- a. A sumary of the warning level analysis made for the previous two failures, h CN 50 __
O' W C 1 4,A&b -d J y,# 4 4 to-2# muu . .at. c. ar a mAu
.c .
s
- b. A summary of the third failure equivalent to the warning level analysis include common mode failures.
- c. An analysis and estimate of the current diesel generator reliability in view of the 3rd failure in 20 demands. Common causes, common failures and evidence of inadequate fixes to previous D/G failures should be examined as indication of a reliability program deficiency leading to poor D/G reliability. More random failures with clearly diverse causes may be an indication of a statistical outlier condition only.
- d. A summary of proposed corrective measures, additional maintenance, special tasks or other licensee actions to be taken to insure thtt the D/G reliability is returned to the desired reliability level or to confirm the statistical outlier condition of the diesel.
1 l . i l 1 ) J i l
a
~
PNijNPAF DieselGenerators SAMPLE OF MONTHLY TEST PARAMETERS Requirements Parameter Required Optional Engine Cooling Water Water presssure to engine X Water temperature to and from engine X Water pressure to and from engine water cooler X Water temperature to and from engine water cooler X (could be radiator) Water pressure to and from raw water cooler X Water temperature to and from raw water cooler X Water pressure to and from turbocharger X Water temperature to and from turbocharger X Water pressure to and from turbocharger after cooler X Jacket water pump pressure (to and from) X
PNL/ Diesel Generators _ Comparison of Current and Proposed Management Programs Current Proposed Program Element Program Program Periodic engine overhauls are required for inspection Yes No Testing program is intended to identify incipient No Yes failures Testing program is chiefly designed to predict No Yes future operability Fast-start induced aging and wear are minimized No Yes Current diesel-generator reliability and station No Yes blackout issues were a program consideration Aging of fuel and lube oil were considered in the No Yes program development Statistical variations and problems are likely to Yes No affect the management program
- o. o .
ENCLOSURE 7, Ref. 11-2-88 NUMARC/EPRI/NRC Meeting SUBJECTS EDG RELIABILITY FINAL DRAFT NUREG/CR-5078 VOLUME 2 A RELIABILITY PROGRAM FOR CMERGENCY DIESEL GENERATORS AT NUCLEAR POWER PLANTS MAINENANCE, BURVEILLANCE AND CONDITION MONITORING
~
Authors: William Henderson, (Trident) David Burghardt, (Trident) Larry Kripps, (EI)
, Ernest Lofgren, (SAIC)
Revision 84 November 4, 1988 i l l
- s. . .
A85 TRACT Whereas Volume 1 provided a top level definition of a reliability program for Emerger.cy Diesel Generators (EDGs), Volume 2 focuses on application of reliability program concepts at a lower level, suitable for in-plant use and use by NRC on site inspectors. In particular, this volume providss in depth treatment of three areas that were given only top-level treatment in Volume 1; surveillance, condition monitoring, and EDG maintenance. For each of these areas, examples and recommendations are provided based on actual EDG operating experience, and on the opinion of diesel generator experts. This information complements the more general guidance provided in Volume 1. Volume 2 also includes a discussion of EDG interactions with other systems (Instrument Air, Service Water, de Power), since these support systems have resulted in interactions that have adversely affected EDG reliability. This volume is a supplement to, and should be used in conjunction with, Volume 1 for an overall evaluation of an EDG reliability program. l iii i ..
- o. o . -
TABLE OF CONTENTS P.AEt ABSTRACT ........................... iii CONTENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . v FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi TABLES ............................ vi FOREWORD ........................... vii ACKNOWLEDGEMENTS ....................... ix
- 1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . 1-1
- 2. RELIABILITY PROGRAM ELEMENTS (
SUMMARY
) . . . . . . . . . . 2 3. EDG MAINTENANCE EXPERIENCE . . . . . . . . . . . . . . . . 31
- 4. EDG SURVEILLANCE EXPERIENCE ............... 4-1
- 5. EDG CONDITION HONITORING CONSIDERATIONS . . . . . . . . . 5-1
- 6. EDG INTERACTIONS WITH OTHER SYSTEMS AND AGING EFFECTS ;
THAT IMPACT EDG RELIABILITY
. . . . . . . . . . . . . . . 6-1 6.1 EDG Interactions with Other Systems . . . . . . . . 6-1 6.1.1 EDC Interaction with Instrument Air System . . . . . . . . . . . . . . 6-1 6.1.2 EDG Interaction with Service Water System ............... 68 6.1.3 EDG Interaction with de Power System . . 6 10 !
6.2 E f fe ct s o f Ag i n g . . . . . . . . . . . . . . . . . . 6 10 : 1 y I
LIST OF FIGURES Ei21 Figure 6.1.1-1 Generic IA/EDG Interaction Mechanism Potential, Specific for Overheating of EDG Controls . . . 6-3 Figure 6.1.1-2 IA/EDG Interaction for Coolant Overheating during a Surveillance Test . . . . . . . . . . 6-4 Figure 6.1.1-3 IA/EDG Interaction for Coolant Overheating . . . 6-5 Figure 6.1.1-4a IA/EDG Interaction for Cooling Water Design Deficiency . . . . . . . . . . . . . . . . . . 6-6 Figure 6.1.1-4b IA/EDG Interaction for Cooling Water Design Deficiency . . . . . . . . . . . . . . . . . . 66 Figure 6.1.1-5 IA/EDG Interaction for loss of IA Pressure . . . 67 Figure 6.2-1. Change in Failure Rate Over Time ......... 6 11-LIST OF TA8LES l E121 Table 1-1. Diesel Generator Reliability Program Review Items .................. 1-2 Table 3-1. EDG Subsystems and Components ........... 32 Table 3-2. Failure Analysis Chart . . . . . . . . . . . . . . . 3-4 Table 3 3. Generic Trouble Shooting Chart for Emergency Diesels ............... 3-7 Table 4 1. EDG Surveillance Recommendations . . . . . . . . . . 42 1 , Table 5 1. Static Checks of the Engine by Engine System . . . . 52 I Table 5 2. Dynamic Engine Measurements ............ 55 I 1 l i vi ! l
ACKNOWLEDGMENTS The authors wish to express their thanks to Aleck Serkiz of the United States Nuclear Regulatory Comission. Division of Research for his strong leadership and guidance during preparation of this Volume. l l I l IX
- 1. INTRODUCTION U.S. Nuclear Regulatory Comission (NRC) regulations and guidelines contained la 10 CFR Part 50, Section 50.63, "Loss of All Alternating Current Power," and Regulatory Guide 1.155, "Station Blackout," establish the need for an emergency diesel generator (EDG) reliability program to achieve and maintain a selected (or target) level of EDG reliability (i.e., 0.95 or higher). Regulatory Guide 1.155 specifically states in Section C.l.2, "Reliability Program":
The reliable operation of onsite emergency ac power sources should be ensured by a reliability program designed to maintain and monitor the reliability level of each power source over time for assurance that the selected reliability levels are being achieved. An EDG reliability program would typically be composed of the following elements or activities (or their equivalent):
- 1. Individual EDG reliability target levels ...
- 2. Surveillance testing and reliability monitoring programs designed to track EDG performance and to support maintenance activities. -
- 3. A maintenance program that ensures that the target EDG reliability is being achieved and that provides a capability for failure analysis and root cause investigations.
- 4. An information and data collection system that services the elements of the reliability program and that enitors achieved EDG reliability levels against target values.
- 5. Identified responsibilities for the major program elements and a management oversight program for reviewing reliability levels being achieved and ensuring that the program is functioning properly.
Consistent with these guidelines and to support the resolution of Generic Safety Issue B-56, "Diesel Reliability," NUREG/CR 5078 Volume 1. "A Reliability Program for Emergency Diesel Generators at Nuclear Power Plants
- Program Structure," was prepared. This report provides technical guide-lines for NRC staff use in the development of positions for evaluating EDG reliability programs. Table 1 1 from NUREG/CR 5078 Volume 1, identifies the principal review items or elements of an EDG reliability program that must be present for the program to be effective.
The primary purpose of Volume 2 to NUREG/CR-5078 is to provide additional guidance, especially for use by NRC onsite inspectors and plant personnel, on three EDG reliability program elements listed in Table 1 1. The elements i are EDG maintenance, surveillance, and condition monitoring.* For each of i l
- Condition monitoring, along with reliability monitoring, make up the EDG performance monitoring element in Table 1 1.
1-1
TABLE 1-1 DIESEL SENERATOR RELIA 8ILITY PROGRAM REVIEW ITEMS A. EDG Reliability Target Ensure that the reliability target for the diesel generator has been established and that calculational measures have been defined that can be evaluated and compared to the target. B. EDG Surveillance Needs Ensure that the diesel generator equipment boundary has been defined and that the diesel generator reliability program has specified a task for analyzing the surveillance needs of this equipment C. EDG Performance Monitoring Ensure that the reliability program specifies a task to monitor diesel generator performance, using both statistical trending and engineering data, to spot dearadations in performance. , D. EDG Maintenance Program Ensure that the diesel generator maintenance program has a reliability focus that includes preventive maintenance, prioritization of maintenance actions and spare parts considerations. E. EDG Failure Analysis and Root Cause Investigation Ensure that there is a task to systematically reduce identified diesel generator problems to correctable causes. F. Problem Closeout , Ensure that the diesel generator reliability program redires a formal t problem closeout procedure and that this )rocedure involves both (1) establishing criteria for problem closeout w1en a reliability problem is detected, and (2) providing for any special monitoring activity to ensure that the criteria have been satisfied >y the corrective action. G. Data System Ensure that a data gathering, storsge, and retrieval system with sufficient capabilities to support all features of the reliability program is in place or will be implemented as part of the diesel generator reliability program. H. Responsibilities and Management Controls Ensure that there are clear line responsibilities and management controls in place that identify responsible individuals for implementing and operating the diesel generator reliability program, and ensure that these individuals are qualified to perform the functions for which they are responsible. 1-2
these elements, examples and recomendations of EDG surveillance, and condition monitoring are provided based onmaintenance, actual EDG operating experiences and the opinion of diesel generator experts. This information complements the more general guidance that is already provided in NUREG/CR 5078, Volume 1, and establishes an enhanced basis for review by NRC onsite inspectors and plant personnel with limited EDG experience. This report also includes a discussion on EDG interactions with nther systems (i.e., instrument air, service water, and de power) since these support systems can affect and have adversely affected EDG reliability. While this report is a supplement to and should be used in conjunction with NUREG/CR-5078, Volume 1, for the overall evaluation of an EDG reliability program, it independently contains information useful to NRC onsite inspectors and plant personnel in reviewing an EDG reliability program. Section 2 is a summary description of all the EDG reliability program elements in Table 1-1 with appropriate references to NUREG/CR-5078, Volume 1, where additional information can be found. Sections 3, 4, and 5 contain the supplemental guidance for evaluation of the adequacy of the maintenance, surveillance, and condition monitoring elements of an EDG reliability program, respectively. Section 6 discusses EDG interactions with other systems and the effects of aging on EDG maintenance, surveillance, and condition monitoring. . It should be noted that Appendix E to NUREG/CR 5078, Volume 1, contains detailed guidelines on failure analysis and root-cause investigation, including several examples that illustrate the absolute need for a rigorous and systematic approach to identifying failure causes and an awareness by maintenance personnel of the importance of their role in the process. Also, it should be noted that additional valuable data on EDG aging and how it affects EDG reliability are contained in NUREG/CR 4590, "Aging of Nuclear Station Diesel Generators: Evaluation of Operating and Expert Experience." NUREG/CR-4590, in addition to NUREG/CR 5078, Volume 1, and this report (Volume 2), provide useful guidance when evaluating an EDG reliability program, i l l l l l l l 13 1 i
- 2. RELIABILITY PROGRAM ELEMENTS (
SUMMARY
) This section presents a sumary of the necessary elements of an EDG reliability program that are listed as review items in Table 1-1. The elements in Table 11 represent the essential considerations in the development of a reliability program designed to sustain the reliability levels needed for EDGs at nuclear power plants. Each element, which is sumarized below, is described in detail in Appendices A through H to NUREG/CR 5078, Volume 1. EDG Reliability Taraet The reliability target for individual diesel generators has been established in Regulatory Guide 1.155 to be 0.95 or 0.975, depending on the plant specific emergency ac power system. This target reliability is to be interpreted in the following way: e The target is an average value over a specified base time or number of demands, o The number of demands are to include actual demands for the - diesel generator systems' function and demand tests of the system that involve an attempted start and run. e Both failuret to start and failures to run are to be included in the calculation of diesel generator reliability. e Diesel generator failures that are recovered with a success-ful start and load within 5 minutes are not to be counted as failures. Appendix A to Volume 1 of this document presents a more detailed discussion of the diesel generator reliability target and how that target is to be estimated, including an EDG failure evaluation criterion for judging the acceptability of EDG failure j historios. l EDG Survei11ance Needs l l Surveillance is defined to include all failure detection and in-l plant reliability information gathering activities. The surveil-l lance strategy for the diesel generators should be a result of an l analysis of diesel generator surveillance needs. This analysis j should be systematically perforined and the resultant surveillance needs periodically evaluated. The dynamic nature of the surveil-1 lance plan, with respect to the EDG's performance, helps to ( ensure a reliability focus to the surveillance activities. l A first step in establishing EDG surveillance needs is to define i the EDG equipment boundary. The diesel generator subsystems and equipment exclusively employed to produce emergency ac power and l 21 l l t
whose sole function 1. related to diesel generator opera! ;y should be included in the EDG boundary. Analysis is rer,uired to ensure that surveillance of diesel generators addresses a minimum set of criteria for acceptable surveillance. The analysis should result in a documented surveillance plan. The surveillance plan should specify the diesel generator surveillance and the rationale for the specified surveillance. The considerations that should be addressed to provide acceptable diesel generator surveillance ar;.
- 1. All critical failure modes of the diesel are covered by the surveillance. Critical failure modes are likely failure modes that would fail the diesel generator function of providing emergency ac power.
- 2. The analysis should identify engineering conditions that are precursors to critical failure modes and suggest surveillance methods (e.g., condition monitoring) to detect those conditions in a timely fashion.
- 3. The analysis should identify likely standby diesel generator 4
aging mechanisms and identify surveillance to detect these. -
- 4. The analysis should emphasize consideration of comon cause failure mechanisms that could fail more than one diesel generator at a site and identify surveillance to protect against these failures.
- 5. Diesel generator repair outages can result from off-normal conditions or failures that are caused by stress on the diesel from starting and running. Failure can als'o result from mechanisms that operate on the diesel generator while it is in standby. Diesel generator demand test periods
, should be set by balancing the effects of these two failure t causes failure modes related to demand stress and those related (to standby stres;). The analysis should contain these considerations.
- 6. A surveillance plan should be prepared that defines the types of surveillance to be employed, the surveillance intervals for each type, and other considerations such as test staggering. Justification based on engineering, human, or reliability considerations should be given as to why the surveillance types and intervals were chosen and why they are sufficient to achieve the reliability target.
Appendix B to Volume 1 of this document presents a more detailed discussion of the assessment of diesel generator surveillance needs. 22
EDG Performance Monitqdp.g Performance monitoring of a diesel generator includes monitoring physical conditions that are precursors to failure or correlated to degradations in performance. Examples include lube oil temperature, manifold temperature, and starting air moisture. Performance monitoring also includes statistical trending of failures and outages that may show detectable degradations in performance. While surveillance provides a "snapshot" of diesel generats operability, perfomance monitoring provides the memory" portion of the problem detection task of a diesel generator reliability program. Appendix C to Volume 1 of this document presents a more detailed discussion of diesel generator performance monitoring. EDG Maintenarce Proaram The maintenance policy for the diesel generators should be docu-morted and clearly exhibit a reliability focus. The maintenance policy should include procedures for preventive maintenance, triggered by observed conditions and/or regularly scheduled, and . a description of the spare parts policy. The maintenance policy should also establish the basis for maintenance actions and their priority. This involves the identification of those conditions or precursors to catastrophic failure that (1) are detectable, (2) are potentially severe in terms of diesel failure, i.e., lead to catastrophic diesel generator functional failure, (3) require long out of-service times for repair if the condition proceeds to catastrophic failure, and (4) are relatively likely to occur. A well formulated, aggressive, preventive maintenance policy will increase EDG reliability. However, as part of the program one needs to demonstrate that the extra equipment down time for ; preventive maintenance will not have a not negative impact on the EDG availability. , A more detailed presentation of the issues to address in this element is given in Appendix 0 to Volume 1 of this document. : EDG Failure Annivsis and Root _Cause investiaation < The diesel generator reliability program should contain a struc-tured approach for systematically reducing identified diesel generator problems to correctable causes. This structured approach involves the following steps:
- 1. Use a failure cause analysis to determine the proximate cause of the failure. The proximate cause is ex description of the piecepart failure cause, e.g. pressed , "relayasxx a failed to transfer due to corroded contacts."
23
4 1
- 2. Compare the proximate cause to past failures or ccnditions on the same and other EDGs to determine if the problem appears to have a systematic root cause, e.g., corroded contacts could be caused by an environmental mechanism.
- 3. If no systematic root cause is indicated, continue EDG operations as usual, including EDG performance monitoring.
If a systematic root cause is indicated, begin a structured root-cause investigation.
- 4. Determine if the problem is generic or plant specific by reviewir.g Nuclear Plant Reliability Data System (NPRDS) and ,
other data and analyses for similar problem symptoms, or ' through contact with other utilities or industry groups.
- 5. If the detected reliability problem is generic, contact other plants that have had the problem to determine what corrective actions, if any, have proved effective. If an effective corrective action has been devised, implement it and proceed to the problem closecut portion of the EDG reliasility program. If not, proceed to the next step.
- 6. If the detected reliability problem is plant specific, determine if the cause i', related to the system's unique design or to operational aspects such as test or main-tenance. This can be done by special monitoring during test, review of operational procedures, or engineering design i
review.
- 7. If the reliability problem is determined to be design related, determine the particular design deficiency.(through special condition monitoring,perhaps), and redesign or specify other correctivn action.
t
- 8. If the reliability problem is related to fadity operations, identify and correct the cpecific procedur4(s) that are the
; root cause of the problem.
- 9. When the root cause has been identif:$d and corrective action implemented, proceed to the prob ~eem closeout item of the EDG reliability program.
Appendix E to Volume 1 of this document describes the process in more detail. An EDG reliability program should be able to verify that the above or simila steps are included in the systematic problem investigation procedures. l Problem Closecut The reliability program plan should specify the procedure that
- will be used for closing out diesel generator reliability 24
problems. The problem closecut procedures should be verified to contain three elements:
- 1. Establish criteria for problem closeout that are based on the nature of the reliability problem detected.
- 2. Provide for any monitoring activity, and s peci fy closeout procedures to ensure that the criteria have seen satisfied.
- 3. Provide clearly defined responsibility levels and actions for problem clo3eout.
The problem closecut criterk should be numerically based and be casable of measurement. TM diesel generator reliability program su>mittal should specify any special problem closecut procedures that will be empicyed tn provide assurance that corrective actions will be effective. ' Appendix F to Volume 1 of this documW yesents a more detailed discussion of this element, r Data system
~
The reliability program should include a description of the data gathering, storage, and retrieval system that will support the die.el generator reliability program, including the following:
- 1. Store both catastrophic diesel generator failures and diesel repair outt es from noncatastrophic failures.
- 2. Store the time of detection, times when repair was initiated and completed, and restoration time of the equipment for each diesel generator repair action.
- 3. Stora 4 description of the root cause or condition that led to repair and the method by which it was detected.
! 4. Store each attempted start and run, runtime, and any failure rate or failure probability denominator information.
- 5. Store in a retrievable way all the information identified in all the above elements.
1 In addition to the above identified operational and failure ) information, the data gathering, storage, and retrieval system > should contain operating experience information nn similar EDGs as providad through NPROS, Part 21 reports, 50.55(c) reports. Licensee Event Reports (LERs), consultants, and especially EDG manufactur ~s and their suppliers (e.g., governor vendors). EDG vendor correspondence and reconnendations and updated operation test and maintenance procedures should also be stored in support of the reliability program. Appendix G to Volume 1 of this document presents a more detailed discussion of this element, t 25
-- _ _ _ _ _ = ._.
.c . . Resnonsibi1ities and Mannaement Controls The reliability program should have clearly defined responsibili ties and management check ('ints to ensure that all items are interacting effectively to maintain the EDG reliability at, or above, target values. This element should provide a means for plant management to review the operation and effectiveness of the reliability program and for altering the program if it becomes necessary. In addition, r. means for independent audit of the effectiveness of the EDG reliability program should be incer-porated into this element. The following considerations are irportant:
- 1. A procedure and schedule fo' verifying that the EDG relia-bility targets are being met should be established.
- 2. There should be an identi'ied mechanism for altering the EDG reliability program sho'nd it become necessary.
- 3. Qualified perso.,nel who will implement and maintain the reliability program should be identified. Personnel quali- .
fications should include diesel operation, maintenance, diesel design, reliability methodology, and implementation of reliability programs.
- 4. Plant management should make an unconditional comitment to implement ind maintain an EDG reliability program.
Appendix H to Volume 1 of this document presents a more detailed discussion of this element. 2-6
- 3. EDG MAINTENANCE EXPERIENCE An essential element of a successful EDG reliability program is a maintenance program with a reliability focus that includes preventive maintenance, prioritization of maintenance actions, and a spare partt program. Section 3.4 and Appen'.4 D to NUREG/CR 5078 Volume 1, describe the general issues that need to i,e addresswo in an EDG maintenance program and how the maintenance program interacts with the other elements of an EDG reliability program. These issues are sumarized in Section 2 af this report. Other obvious keys to an effective maintenance program are trained, qualified, and experienced mechanics and an apprecistion that, while EDGs have excellent reliability if they are maintained correctly, they are a complex piece of equipment that can tolerate only a comparatively narrow band of on design performance from the individual components comprising the EDG system.
As additional guidance to NRC onsite inspectors and plant personnel in # reviewing the adequacy of an EDG maintenance program, this section describcs prevalent EDG component failures and their causes along with suggested preventive and corrective maintenance and preventive design measures to eliminate or minimize them. The section is derived from two sources; (1) NUREG/CR-4590, "Aging of Nuclear Statfon Diesel Generators: Evaluation of Operating and Expert Experience," and (2) expert opinion of Trident engineers who have significant experience in maintaining and trouble-shooting EDGs at nuclear power plants. The experience described here on common and dominant EDG component failures and causes can be used to specifically examine whether an implemented maintenance program is addressing identified prnblem areas known to affect EDG reliability. The remainder of this section describes the most frequently encountered EDG problems, as well as preventive and corrective actions, for each of the components within the subsystems shown in Table 3-1. The subsystem break-down in Table 31 is consistent with that given in NUREG/CR-4590 from which much of the material is derived. (Note: The breakdown is not consistent with the definition of diesel subsystems suggested in Volume 1 of NUREG/CR-5078.) A convenient summary of the failure symptoms and their causes is presented in a failure analysis chart in Table 3-2. The failure analysis chart can also be put into the form of a trouble shooting chart, which is presented in Table 3-3. This trouble shooting chart is generic; the operating manual f ar a given engine will have one that is specific for that engine. This chart includes symptoms, causes, and remedies for a variety of engine ralfunctions. The probitms are those that most comonly occur in engine operation. INSTRUMENT 1T!0N AND CONTROL Governor- a governor's failure to perform correctly can be mini-mized b.v following the manufacturer's recomended maintenance program. This piece of equioment needs little required main-tenance, but it must be done M ficiently or the unit will not perform. The hydraulic gov"m should have its oil replaced per specifications, washing y d r. ( lightweight lube oil, before replacing governor oil. Ah r : the oil change, all the air must be bled from the governos hydraulic system. The drive gear 31
)
~
i
.. . . l TABLE 3-1 EDG SUBSYSTEMS AND COMP 0NENTS Instrumentation and Control Governor Overspeed Governor <
Control Valves Alarms and Shutdowns Linkages , Startup System Control Air System , Wiring and Terminations ' Relays Sensors Fuel Components Injector Pumps Fuel Oil Pumps , Injectors and Nozzles
- Piping -
Oil Storage Start!.g Components Starting Air Valve Air Compressor , Controls Starting Air Motor Switchgear Components Breakers Relays Instrumentation and Control Cooling Components ! Lube Oil Pumps Heat Exchangers Radiator Piping ' Lubrication Components Pumps Filters Heat Exchangers Pi sing Lusricating 011 Intake and Exhaust Scavenging Air Blower Blower-Turbocharger Valve Linkage Turbocharger \ 3-2
TABLE 3-1(Continued) EDG SUBSYSTEMS AND COMP 0NENTS Generator Components Generator Exciter Voltage Regulator Pedestal Bearing Electrh..i Systems Switches Wiring Trans formers Controls Engine Structure Crankcase Cylinder Heads Main Bearings . Cylinder Liners , Drive Train Connecting Rod Bearings
' Valve Mechanisms Exhaust Valves l
Mechanical Systems Structural Components i I t 3-3
TABLE 3-2 FAILURE ANALYSIS CHART SYSTEM SYMPTOM CAUjil INSTRUMENTATION Governor failure Insufficient oil A140 CONTROL Dirty oil Improper oil Air in system Governor hunting Dirt in needle valve Overspeed governor fails Lack of calibration Alarm / shutdown improperly Improper calibration activated loose connections Control air valve fails Water / dirt / rust . Improper temperature Improper calibration and pressure measurement Dirt on sensor FUEL COMPONENTS Unit injector failure Leakage; clogged tip Injector / nozzle failure Leakage; clogged tip Piping fails (cracking) Vibration induced i fatigue failure ! Sludge in fuel oil Water in storage tank STARTING Air valve stays open Dirt / water / rust COMPONENTS Starting motor fails Dirthater/ rust : Solenoid valve failure Dirt / water / rust Overheating i SWITCHGEAR Relay failure Dirt, loose contacts COMPONENTS Instrumentation and loose contacts controls Pitting, corrosion t of contacts COOLING Insufficient cooling Excessive pump COMPONENTS water pressure clearances Pump failure Excessive wear and seizure i 3-4
., . o TABLE 3-2(Continued)
FAILURE ANALYS!$ CHART SYSTEM SYMPTOM CMSI COOLING Leaking heat exhanger Gaskets / seals COMPONENTS improparly installed (Continued) Insufficient cooling Clogged / dirty heat exchanger Piping fails (cracking) Vibration / insufficient suoports l LUBRICATION Insufficient lube oil Fump clearances too COMPONENTS pressure great Water in lube oil Seizure of pump Debris in lube oil
~
Clogged filters Improper maintenance Water leaking into Gasket / seal lube oil improperly installed Tube failure in cooler Piping fails (cracking) Vibration-induced fatigue failure; insufficient supports Loss of lube oil quality Water in oil Dirt in oil INTAKE AND Roots type blower fails Low /no load operation EXHAUST Oscillating of air Insufficient oil in dampers blower / turbo valve linkage dashpot Turbocharger failure Imbalance--air side fouling Bearing failure GENERATOR Generator failure Insulation breakdown COMPONENTS due to overload, overheating, or overlubrication Exciter failure Insulation breakdown Brushes worn 3-5
c TA8LE3-2(Continued) FAILURE ANALYSIS CHART SYSTEM SYMPTOM CMSI GENERATOR Voltage regulator Burned and oxidized COMPONENTS failure contacts (Continued) ELECTRICAL Switch failure Corrosion / oxidation SYSTEMS Control failure Corrosion / oxidation ENGINE STRUCTURE Loss of crankcase Piston ring blowby vacuum Leaking seals / gaskets Eductor clogged Cracked cylinder heads Uneven torquing of head during ssembly Severe detc.ation . Thermal stress Main bearing fails Engine overload Dirty / contaminated oil Improper installation Improper lube oil Liner distortion or Uneven tightening cracking during assembly , Lack of lube oil Lack of cooling DRIVE TRAIN Connecting rod bearing Engine overload failure Dirty / contaminated lube oil Oil not per specifications Improper installation VALVE MECHANISMS Burned exhaust valves Engine overload
- Injector leaking Fuel contaminants
- Improper lash adjustment 36
TABLE 3-3 GENERIC TROUBLE-SHOOTING CHART FOR EMERGENCY DIESELS SYMPTOM M REMEDY ENGINE FAILS TO Air line valves not Check valves in starting START open to engine air system Air starting pressure Start compressors to too low buildup pressure Improper air starting Check air starting timing valve timing FAILURE TO START Fuel shut off Open proper valves from fuel tan ( Service or fuel tank Refill tank empty Fuel injection system Prime fuel system air bound Clogged air filter Replace air filter Engine cranks over Starting air pressure may too slowly be too low Improper fuel metering Check pump timing pump timing loose high pressure Tighten connection fuel line connections from injection pump to injector Improper intake and Readjust valve clearances exhaust valve clearances RARD STARTING Fuel pump needs Prime system priming Air in system Prime system Incorrect fuel pump Check pump timing timing Throttle not set to Move throttle to start starting fuel position position 3-7
.o . .
l TABLE 3-3(Continued) GENERIC TROUBLE-SHOOTING CHART FOR EMERGENCY DIESELS SYMPTOM CAlljil REMEDY HARD STARTING Defective injectors Test and adjust or replace (Continued) Water in fuel system Check system for watqr Leaky intake or Check engine compression exhaust valves Improper valve Check intake and exhaust clearance valve clearances Engine too cold Circulate warm water ENGINE DOES NOT Clogged air filter Replace air filter COME UP TO SPEED ' OR DEVELOP FULL Clogged fuel filter Check and replace - POWER; ENGINE LOSES POWER Insufficient oil Check to see if booster pressure at fuel pump is operating; check injection pumps header pressure regulator Improper fuel injector Check fuel pump timing pump timing Water in fuel Drain water.from bottom of day tanks Flush fuel lines with clean fuel Centrifuge fuel Air in fuel injection Bleed system to injectors system j Low compression Check cylinder compression pressure Governor not function- Check governor ing properly adjustments i Defective injectors Test injectors and adjust or replace cylinders misfiring Check for air in system ! 3-8
TABLE 3-3 (Continued) GENERIC TROUBLE SHOOTING CHART FOR ENERGENCY DIESELS SYMPTOM 2$1 REMEDY ENGINES DOES NOT Cylinders misfiring Check for water in COME UP TO SPEED (Continued) system OR DEVELOP FULL POWER; ENGINE Phock for valve clearances LOSES POWER (Continued) Faulty injector , Loose high-pressure fuel line connection Collapsed or clogged Check exhaust back muffler pressure Poor quality fuel Change fuel . ENGINE RUNNING Improperly balanced Check metering pump fuel UNEVENLY fuel injection system rack settings Defective injector Test injectors and i adjust or replace Malfunctioning Adjust compensating hydraulic governor needle valva Air leaks in booster Check for air leakage in or transfer line fuel system DETONATION Early injection timing Check and adjust timing Defective fuel Test injectors injectors Poor quality fuel Change fuel Engine overload Remove some of the engine load ENGINE OVERHEATS Lack of coolant in Check radiator or system expansion tank Radiator or coolers Clean and replace dirty 3-9
.o - o TABLE 3-3(Continued)
SENERIC TROUBLE-SHOOTING CHART FOR EMERGENCY DIESELS SYMPTOM Gallii REMEDY ENGINE OVERHEATS Lack of lubricating Check oil sump (Continued) oil causing friction Worn water pump Replace pump Engine overload Remove excessive load Late fuel injection Check injection fuel timing pump timing Faulty thermostat or Replace thermostat or thermostatic valve adjust valve EXCESSIVE External engine oil Locate and replace (UBRICATING JIL leaks gaskets, etc. - COSUMPTIO'a Excessive valve guide Replace valve guides clearances Worn rings Check compression pressure Excessively worn Check compression cylinder liners pressure Excessive main and Check bearing clearances connecting rod bearing , clearances Excessively high oil Unclog or clean dirty temperature oil cooler LOW LUBRICATING Dirty or clogged oil Check and replace OIL PRESSURE filters Fuel oil dilution Locate internal fuel leaks and change lube oil
- Faulty oil pressure Check and reset regulator Worn lube oil pump Replace pump 3-10
TABLE 3-3(Continued) 4-S DERIC TR008LE-SH00 TING CHART FOR SERGENCY DIESELS SYMPTOM El11 RLtiLD1 LOW LUBRICATING Too light an oil for Change to heavier grade
, OIL PRESSURE high temperature oil (Continued) operation Excessively worn main Renew bearings and connecting rod bearings BLACK SMOKE Engine overload Reduce load Malfunctioning Test, repair, or replace injectors I
Improper fuel Check injector timing injection timing - Unbalanced fuel Check rack settings for injection pumps each pump Dirty oil filter Replace filter low engine Run cylinder compression compression pressure test BLUE SMOKE Excessive lube oil in combustion space Worn valve guides Replace guides or seats Worn oil control rings Replace rings Worn cylinder liner Replace liner i Excessive main and Replace bearing connecting rod bearing clearances WHITE SMOKE Improper fuel Check for air fuel ir, fuel Combustion / cylinders system Misfiring Check for water in fuel system Check valve clearances i Faulty injector i 3-11
clearances should be checked during any major overhaul. Note that any major repair or maintenance should be performed by the manufacturer's representative. Controls to adjust include s drop, load limit, speed regulating and the needle valve, peed and regulating control oil flow in the governor's compensating system. The needle valve is used for bleeding the air from the system and is susceptible to any dirt entering the hydraulic system, which would block flow through the valve and cause hunt-ing of the engine. Periodically, the oil level should be checked. Finally, on some units there is an electric governor that works in conjunction with the hydraulic governor. The electrical (electronic) components should be calibrated periodi-cally. If the electrical auxiliary control is nullified, the engine fuel controls can move to the maximum fuel position, resulting in severe overload operation of the EDG. Oversoeed Governor--the typical failure symptom of the overspeed governor is that the engine overspeeds on startup. After every major overhaul, the overspeed governor should be checked and adjust 2d as necessary to the proper tripout speed, or more frequently if the manufacturer so recommends. They are not designed for precise steady state operation, and the speed - increase to check this governor should be quick, not g rar'u al . The negative effect of vibration on all engine components regarding calibration must be borne in mind constantly. Another condition that causes the unit to overspeed is poor maintenance of the main injection pump racks, which stick due to dirt, grime, or slight misalignment and thus prevent the rack from moving back as operating speed is reached. Alarms and Shutdowns--the engine will fail to operate if any of the following alarms should activate: high temperature of cool-ing water from (1) the engine jacket water discharge from engine, (2) the air cooler water discharge, and (3) the lube oil discharge from the engine; and low pressure of (1) the jacket water, (2) the air coolant water, (3) lube oil, and (4) too little crankcase vacuum. The alarms and sensors should be checked, calibrated, and adjusted during every overhaul of the engines. Linkaaes--linkages fail to perform properly due to two primary causes: one, maintenance and lubrication, causing binding or failure to move as required; and two, the linkage joints between component parts of the linkage and what it controls become worn and this creates excessive play in the linkage. Start-uo System -the failure of controls during start up are primarily electrical, centered on relays not performing correctly. Relays fail if they do not activate at the correct on and off values or if the contacts are pitted, resulting eventually in physical failure of the unit. The units should be checked and calibrated periodically, noting that if they are located in a place subjected to engine vibration that the vibration can cause the settings to move off calibration. 3-12
.o . .
Additionally, a cold and damp environment hastens physical i deterioration of the relays. I control Air System -failure in the control air system is brought about by dirt and/or water in the system. There should be adequate dehydration and filtering of the control air. Periodic checking of the dehydrator and the filters is necessary. Wirina and Terminations the connections of wires are susceptible to engine vibration. They should periodically be checked for tightness. Vibration may also cause the wire to flex and break, so the wire integrity must also be noted at the same time. A loose connection or broken wire will indicate a false signal, perhaps causing the engine to shut down.
! Re' avs--relays fall if they do not activate at the correct on and l off values or if the contacts are corroded, dirty or pitted, t resulting eventually in physical failure of the unit. The units should be checked and calibrated for the on values for pressure differential relays and for pressure /off and temperature limits on alarms. Vibration can cause the settings to move off i calibration. -
' Sonsors--temperature and pressure sensors will either not work at aL1 or move off calibration. The sensors should be periodically
; recalibrated, particularly in the case of pressure sensors.
i Additionally, cieck the cleanliness of thermocouples and pressure transmitters, as dirt will lower the reading. This is one of the 4 most important areas of maintenance and is essential if useful diagnostic programs are to be developed. Rigorous calibration { and connector checking programs must be developed and adopted. Effective maintenance programs depend on reliable data and are i not truly possible without such data. FUEL COMP 0NENTS j Iniector Pumos (Unit "niectorsh these combination pumps and injectors most comon' y fail to perform due to leakage or a plugged injector tip. The leakage is caused by scoring of the barrel and plunger, which must be re laced and the unit l , overhauled. Pluggedorificesinthenozzgetipcan cause high I atomization, possible detonation and fuel impingement on the j piston crown, with resulting burning of the crown. If there is water in the fuel, the tip will easily be blown off because of l the high surface tension of the water in the tip at high ! injection pressures. High pressure water also erodes and j enlarges the fine spray holes in the nozzle tip. l Fuel 011 Pumes- these transfer pumps fail in two ways, producing l insufficient pressure or seizure of the gears. These positive j displacement pumps depend on proper, close clearances and good ! lubrication for their successful operation. As the clearances
- become great, the pressure buildup across the pump decreases.
Water in the fuel oil can accelerate wear dramatically. Checking i 3 13 l i
.. . o i
the clearances during repair is essential to the operation of ! these pumps. Any foreign matter entering the pump will scuff the pump housings and reduce oil flow as well as pressure. Igjectors and Nozzles--failure of the injectcrs to perform is l often due to leakage or plugged nozzle tips. Leakage may be into ' the injector body or into the cylinder. Leakage into the ! injector body occurs because of wear between the nortle body and : the needle valve in the nozzle. This leakage is caused by wear l and scoring between these two parts, often due to improper t filtering of the fuel. Should this occur, replace the nozzle i assembly. Leakage into the cylinder is most often due to low i injector opening pressure, which can be corrected by proper i adjustment of the injector on an injector test stand. This type ! of leakage can also be caused by poor seating of the needle valve : on its seat in the nozzle. If this is the case, the nozzle assembly should be sent out to be serviced or a new assembly installed. Plugged orifices in the nozzle tip can cause high , atomization, possible detonation and fuel impingement on the piston crown, with resulting burning of the crown. Poor filter ( maintenance with resulting dirt in the fuel will cause plugging i of tips. - ! Pinina--piping failure, cracking of piping lines, is brought about primarily from engine vibration. This may be remedied by increasing or reducing the number of hangers supporting the piping and making sure the hangers are securely fastened to the pipes and to the engine. The more the piping is isolated from the engine by expansion joints, the less tie engine vibration is transmitted to tis piping. Oil Storane- improper oil storage may result in water and/or sludge in the fuel oil. Periodic draining of water and sludge from the storage tanks is required, as well as filters located in the lines from the storage tanks to engine fuel day tanks. 011 can also age so as to thicken and have other problems. Sampling and testing of oil is required. STARTING COMPONENTS Startina Air Valvo the valve fails to operate most frequently by not closing comp'etely or sticking open. Dirt or water in the air starting system will cause this to happen. The water transports the dirt and metal particles and creates rust. Air Comorassor- the two primary failure modes are not building sufficient discharge pressure and/or insufficient capacity requirements. Leaking discharge valves are the primary source of both failures. The valve leakage may be age related as the valve and valve seats wear over time or it sty be due to carbon build-up on the valve seats. The carbon build up is created as a result of lube oil in the air, which in turn occurs from not changing lube oil as specified or from a lack of oil scrapper ring maintenance. The effect of both allows the oil on the 3-14
.
- o cylinder walls to vaporize into the air and oxidize on the hot discharge valve seat surface. Additionally, the failure to produce sufficient capacity may be caused by dirty intercoolers, which should be cleaned periodically.
Controls the most common control failure in the starting air system is solenoid valve failure. The valve may stick because of dirt and/or water as above items fail, but additionally is susceptible to overheating and coil failure. Startina Air Motor- the motor fails to operate, is stuck, most commonly due to dirt or water (forms rust) in the system. The rust and dirt from the air system piping will cause internal wear of the motor and vanes and loss of motor efficiency.
$WITCHGEAR COMP 0NENTS Breakers breakers fati to cpen at correct load levels if the contacts are pitted, welded, dirty or corroded, resulting eventually in physical failure of the unit. Maintain cleanliness of the unit and tightness of connections.
Belavs- relays fail if they do not activate at the correct on and off values or if the contacts are pitted, welded, dirty, or corroded, resulting eventually in physical failure of the unit. Maintain cleanliness of the relays and tightness of connections. Tests as recommended. Instrumentation and Control--all electric and electronic instrumentation is sensitive to vibration, causing loosened connections, and to adverse invironmental effects, such as high humidity, water, and dirt. These environmental effects will cause overheating and accelerated pitting of contacts. Periodic checking of the calibration of these units will minimize these problems. COOLING COMPONENTS Egti--pumps fail by not producing sufficient head and flow to cool the diesel engine under load. In some instances the pump may fail totally. The insufficient flow failure results from allowing a deteriorated condition to persist; for instance, excessive clearances cause the pump to not be able(impellertocasing, to produce the flow bearings) required. The deteriorated condition may be age related due to normal wear or may be caused by excessive vibration. If the inlet and discharge piping is not adequately supported, excessive vibrations will be transmitted to the pump. If the pump produces insufficient flow, or a heat exchanger is dirty, tie water inlet temperature may exceed the maximum safe operating temperature of the pump and accelerate wear or cause seizure. Clogged inlet strainers or air in the suction lines will reduce the flow of the pump. 3-15
Heat Exchanaers--heat exchangers fail by leaking or by insufficient heat transfer. Periodic inspections of the tubes are required to ensure cleanliness and proper installation of seals and gaskets during maintenance and repair operations. Vibration may cause loosening of gasketing if heat exchanger piping is not properly supported. Ensufficient expansion joints in the piping to and from the heat exchanger will also cause excessive vibration to be transmitted. Care must be taken when heat exchanger tubes are bored out to remove scale, as the tube wall may be cut in the process. Radiator -radiators fail by leaking or by not producing sufft-cient heat transfer. Periodic inspections of the inner radiator surfaces during maintenance and repair are required to assure proper cleanliness. Outer radiator surfaces should be washed out and blown out to assure proper heat transfer. Insufficient expansion joints in the piping to and from the radiator will cause excess vibration to be transmitted and possible radiator leakage. Pioina- piping fails by cracking or loosening at joints. Vibration is the problem and the cure is increasing or decreasing the number of hangers for support. Insufficient expansion joints in the piping will also cause vibutional problems. LUBRICATION COMPONENTS Lube Oil Pumos -Lubricating oil pumps fail in two ways: producing insufficient pressure or seizure of the gears. These positive displacement pumps depend on procer, close clearances and good lubrication for their successful operation. As the clearances become great, the pressure buildup across the pump decreases. Water in the lube oil can accelerate wear dramatically. Checking of clearances during repair is essential to the operation of these pumps. Debris in the oil ducts after an overhaul, such as from a broken light bulb or forgotten cotter pin, scuffs pump housings and reduces oil flow as well as pressure. Filters filters will fail, clog, or deteriorate, as a result of improper maintenance. The most frequent cause is a carelessly installed 'O' ring. During engine operation, the differential pressure across the filter must be checked and be within the manufacturer's guidelines. Clogging of the filter causes the pressure to increase, and this increased pressure can cause the filter material to fail. Heat Exchancers- the most frequent heat exchanger failure is leaking cooling water into the lube oil or lube oil into the cooling water. This will be detected by oil in the water reservoir or by a false increase in the lube supply. The cause is improper installation of the gaskets and seals during maintenance and repair or by failure of the tubes in the cooler. 3 16
[.. . . i
' Pinine -leaking due to cracks or at gasketed joints is frequently caused by engine vibration. Adding or subtracting sufficient hangers and supports is necessary. Leaks at expansion joints may i
be encountered resulting from improper procedures in securing the expansion joint. The more the piping is isolated from the engine by expansion joints, the less the engine vibration is
! transmitted to the piping, j
Lubricatina oil -contamination of the oil is a frequently encountered problem with water being a typical contaminant. It can enter t to system due to a cracked liner, an improperly installed liner seal, improper storage, or in a heat exchanger as 4 noted above. Dirt as discussed under filters can be another d contaminant that increases the wear of engine parts. Properly i performed lobe oil tests will detect th's problem before it affects other components. Water is extremely detrimental in that it not only reduces the load capability of the oil, but some oil j additives are destroyed or nullified by water. 1 i INTME M0 EXHMST i Seavennina Air Blower (Roots type).the blower fails most - ' frequently because the lobes rub on the housing or each other. This is caused by thermal stresses, causing excessive expansion, and is consenly brought about by sustained low load or no load
- operation. Poor attachment of the lobes to the shaft will allow
! improper lobe movement and hence rubbing. Excessive strain on l the blower housing will distort the housing and cause scuffing, i Most frequently this strain is created by lead inputs from rigidly attached inlet and outlet ducts that do not isolate the precision blower assembly from thermal expansion of the ducting or the heavy weight of the ducting, i Blower Turbocharner Va've Linkaan--this valve controls the air ! dampars between the b'ower and the turbocharger and may stick open or oscillate, opening and closing the damper. The t cause is lack of oil in the das'npot attached to the valve. ypical I Turbocharner the turbocharger will most frequently fati because l' of imba'ance, causing extreme vibration. This is usually caused by dirt accumulation on the air side, although there have been I cases where a piece of metal, such as from a failed compression ring, will pass through the exhaust strainer and damage the turbine blades. If an engine is run at low load or no load for sustained periods, raw oil, fuel, or lube oil can be deposited on the turbine blades. When the engine load is suddenly increased. l the accompanying high gas temperatures carbonize some of the oil
- deposited on the turbine blades causing an imbalance. This l condition can exist if there are lon cooldown periods. Turbo-chargers of the type where the rotor s supported by one central bearing are more susceptible to bearing failure than ones where both ends of the shaft are supported. Lube oil quality and pressure are extremely important in this situation to prevent failure.
3 17
+. . .
l GENERATOR COMP 0NENTS Generator- generator wiring is sensitive to high humidity or water, which act to break down the wire insulation. Dirt accumulation, sometimes the result of overlubrication, will result in arcing and breakdown of insulation values. Dirt will also plug cooling holes, causing generator overheating. Operation at severe overload will cause insulation breakdown and generator failure. It is possible on some systems for overload to occur on startup due to the lack of restrictions on the load sequencing of the EDG. Exciter--the exciter may be a small de generator or a rectified ac generator. In the former case the brushes must be checked for c fit, tension, and size and the comunutator for cleanliness. I, both cases of ac or de generator, the guidelines for a generator, noted above, should be observed. Voltaae Reaulator -the voltage regulator controls the generator voltage by changing the excitation of the generator fields. In the Tirrill regulator, the field resistance of the exciter is short circuited temporarily when the bus bar voltage drops. This - is accomplished by contacts that are vibrating continuously with the time they are closed depending on the bus bar voltage. Contact burning and oxidation will lead to failure. Pedestal Bearino. pedestal bearings support the generator and one of the bearings must be insulated from the bedplate to pravent shaft (stray) currents. The insulation must be tested periodically. ELECTRICAL SYSTEMS Switches- switches fail to open if the contacts are severely pitted as a result of arcing. Additional failures are associated with loose connections to the switch. Wirina failure of wiring occurs due to insulation deterioration with time which may be accelerated by adverse environmental factors. Periodic inspection of the wiring should include insulation resistance testing and visual inspection of wire torsinations. Transformers--overheating of transformers is one of primary causes for failure. Overheating results primarily from dirt accumulation, preventing the required heat dissipation. Air passages in dry transformers and the windings should be periodically cleaned. For liquid filled transformers the liquid should be checked for the presence of moisture and sludge. The accumulation of sludge on tie coils will cause overheating. Controis -all electric and electronic instrumentation is sensitive to vibration, causing loosened connections, and to adverse environmental effects, such as high humidity, water and 3 18
i dirt. These environmental effects will cause overheating and i accelerated pitting of contacts. Periodic . checking and , calibration of these units will minimize these problems. ENGINE STRUCTURE Crankcast -the most comon mode of failure is loss of vacuum in the crankcase. This is most fre the piston compression rings, seals,quently and/orcaused gasketsbyleaking, blowbyor of the ( 4 eductor located in the exhaust line clogging with carbon at the exhaust line juncture. 3 Cylinder Heads- the most frequent failure is cracked heads. These are caused by uneven torquing of the head during assembly . i of the engine, severe detonation, or severe thermal stresses associated with dramatic cooling water tem >erature change or pump i failure. The cooling passages should >e checked for scale deposit formation during overhaul periods. Severe detonation can i occur during fast load starts. Main Bearinas--loss of load support ability or bearing failure. This is caused b loss or deterioration of the lubricating oil -
, film between the aring surfaces. The most frequent causes are i engine overload, loss of lubricating oil pressure, dirty oil j
(water or dirt in the oil), fuel oil dilution, misalignment of l the bedplate, misalignment of the bearings, and improper l maintenance procedures during bearing installation. An improper !
- maintenance procedure is a comon problem because of the size and j weight of the parts and the inherent, difficult accessibility of f the bearings. When major main bearing failures have oMurred, ;
block bearing saddles and caps must be checked for excessive ! i distortion. ' i tvlinder Liners the cylinder liner fails by cracking or i } distorting. Distortion can be caused by uneven tightening of the !
- head during assembly causing out of roundness of the liner. This !
in turn can cause liner scuffing by the piston. Cracking is ! brou ht about b too great a frictional force between the piston , i and iner. Fat ure causes normally relate to lack of lubricating oil or cooling water, or to fast starting and loading the diesel, ' The cracks will frequently appear where the greatest residual i j stress in the liner 1s- around the intake and exhaust ports. ! Water treatment is important in corrosion related cracking. The treatment must not only prevent scale but corrosion as well in ; J this region of intense and cyclic thermal stresses. Some ' cylinder liner assemblies incorporate 'O' ring coolant seals. l Care must be taken to ensure that water treatment chemicals do L i not destroy the seals and that they are not used longer than ' i' recomended, as there is a finite material life to the seals ! whether they are in service or on a shelf. l i
- i r
ll 3-19
CRIVE TRAIN Connectina Rod Bearinas -bearing failure primarily occurs because of the lack of sufficient lubrication. Causes are due to overload, lack of sufficient lube oil pressure, contaminated lube oil, or not using proper oil per manufacturer's recomendations. Use of properly filtered and chemically maintained lube oil suit-able for severe, heavy duty, diesel application and with which the engine manufacturer has had good field experience is essential. VALVE MECHMISMS Exhaust Valves exhaust valves will suffer severe thermal stress (burn) due to high exhaust temperatures. The failure causes typically relate to excessive cylinder load or injector leaking, both of which cause high exhaust temperature or carbon particles trapped between the exhaust valve and the exhaust valve seat, not allowing it to close proserly. Improper opening and closure is caused by the valve lasi not being correctly adjusted. Loose valve seat inserts can cause valve failure as well.
~
MECHMICAL SYSTEMS Based on NUREG/CR 4590 the number of failures associated with this system are very small and not included for further analysis. STRUCTURAL COMPONENTS Based on NUREG/CR 4590 the number of failures associated with this system are very small and not included for further analysis, I i j ) l 1 1 i I 3-20
.. s
- 4. EDG SURVEILLANCE EXPERIENCE 4
EDG surveillance includes all failure detection and in plant reliability information gathering activities. Section 3.2 and Appendix B to NUREG/CR-5078, Volume 1, generally describe the considerations that must be addressed - in establishing an acceptable EDG surveillance plan. This section describes specific surveillance needs that have been identified by diesel generator l experts to help prevent frequently occurring or particularly damaging EDG l ! failures. While the recomended surveillance activities described here should be useful to an NRC onsite inspector and plant personnel when reviewing an EDG reliability plan, they are not intended to represent all i
- necessary and sufficient surveillance needs. l i
The surveillance needs for individual (DG components or subsystems are ( ) described under separate headings below. Table 4 1 summarizes the EDG ' surveillance recommendations. Connectina Rod and Main Bearina surveillance Bearing failure cannot be avoided in all circumstances, but it ) can be detected in its incipient stages. The surveillance 4 strategy in this instance is to determine the causes of failure, - 3 such as overload. The peak pressure from each cylinder should be monitored during operation of the engine so that excessive 1 j pressure due to overload or imbalance can be detected. If the
, overload is due to an improperly operating injector, it can be i pulled when the engine is stopped, and a properly cleaned and i tested one put in its place. For severe cylinder overload, the connecting rod bearing should be removed and inspected at the
- next overhaul period. The removal of bearings for inspection can
- do more harm than good if the bearing is functioning well since
- the removal can damage the bearing so that it will fail in the
- future. It is well recognized that overnaintenance or unnecessary maintenance can increase failure rates.
l The bearing to test when the engin6 is overhauled, if no individual bearing has indicated a need, is the last one, nearest
- the generator. This one will be subject to the greatest wear i under normal circumstances. Any generator misalignment will i affect this one first. The bearing should be rolled out and measured for wear and distortion gage readings taken.
Is there a way to determine a hot bearing when the engine is operating? In some cases, yes. The short and transient nature of EDG operation at nuclear power plants makes this method less reliable than when the EDGs are under steady state operation. Diesel engineers will often check the temperature of the crankcase doors during the running of the engine. The oil from ! the connecting rod and main bearings in most engines is thrown from the bearings and splashes against the door. If the tempera-ture of one door is hotter than that of other doors, it means the bearing is running hotter. It may not fail, but should be
- checked at the next opportunity. The nuclear power plant i operators are not diesel operating engineers, so the sense of i
l 41
e
'l TA8tt 4-1 (DG $URVftLLANCE t[ComutuoATIONS EDG Cassponent/Sidssystem h Jed Serwelilante Carmecting Sed and testa teoring 1. Cyllader peak pressere or cyltader noen effectlee pressure.
- 2. Crankcase deer (or victatty) temperature.
- 3. 8teesersments of last mela engine beering neerest genereter.
T. Lake oil seg1tng and analyste during engine operetten. Air Starting Velve 1. Temperature of str starting Ifne neer air start values. Torbecherger I. Torbacharger rpm versus loed.
- 2. Tertecterger discherye pressure eersas load.
- 3. Seertas temperatures adiere evallable.
(shaust Valve 1. Feel ell pg rock settlags taken ehlle engine is operating. b 2. Cyllnahre enheest tesquerature. Lube oli pum , I. Liee ett pisy discharge pressere and temperature.
- 2. Engine Isod.
- 3. Lies oil wiscoelty adjusted to pony talet condittens.
Governor 1. Static c9eck of the governor ett level.
- 2. Check gewerner linhage to feel rocks for esse of setten.
Liee 041 temeter 1. Duck heeter element for cleanliness during asjer engine overtieel. Coollag tester Systems 1. semesure unter inhtbitor concentratten level during static chschs of the engine.
- 2. Chuck jachet unter electric fester element for scale storlag overhol periods.
- 3. Check electrolytte protectlen systema chsring overheel periods.
4 Stonitor tausperature change across heat rC,. during operetten versus Iced for fouling of heat exchanger.
a, 4 g touch can be replaced by a temperature measurement. There are at least two choices, a hand held contact thermometer or locating a thermocouple on each door or in a place where the oil splashes. If there is a temperature difference between locations With one being hotter than the others, it can be an indication of a bearLng that is showing problems due to dirt, water, excess pressure, or lack of lubricating oil. Since this is a qualitative measurement, the baseline information must be established for each plant and engine. Because any additional instrumentation must be calibrated and maintained, the portable contact thermometer is the better choice if such a measurement is deemed desirable. Bearings are very susceptible to dirty and contaminated lubricating oil. Both act to reduce the load carrying ability of the oil, which, if it occurs, will cause a bearing to fail. Lube oil analysis should be performed on a running engine, with the sample drawn from the centerline of the lube oil header. The sampling connection from the bottom of the header will tend to have more contaminants than the oil in general. Taking samples from the crankcase does not provide a satisfactory indication of the oil reaching the bearings. It should be noted that lube oil - will build up contaminants such as carbon, fuel oil, dirt, and metal particles with time. This is expected. The analysis will help in tracking wear of key parts. In sumary, the following parameters are useful in a condition monitoring program:
- 1. Cylinder peak pressure or cylinder mean effective pressure.
- 2. Crankcase door (or vicinity) temperature.
- 3. Measurements of last engine bearing nearest generator.
- 4. Lube oil sampling and analysis during engine operation.
Air startine Valve Surveillance When an air starting valve begins to malfunction, it is usually caused by dirt building up around the valve seat, preventing the valve from completely closing. This condition will get worse and may prevent the engine from starting. Diesel engineers will observe the air supp y line near the air valve to notice unusual conditions. All tie lines are hot because of their location on the engine, but if the combustion gases are leaking into the air start line, the line will smoke in some instances and certainly be significantly hotter than other air starting lines in any case. Measuring the temperature of the air start piping near the air starting valve becomes a diagnostic aid. If there is a demonstrable temperature difference between the readings, it is an indication of an air start valve not closing completely. The baseline temperatures will vary from engine to engine, and there can be variations betveen locations on a given engine due to the 4-3
piping layout. It will be necessary to denote the air starting line measurement point unless thermocouples are attached. A hand held contact thermometer would be a better choice since relative measurements are required, and it would not necessitate a permanent sensor attachuent. Parameters useful in a condition monitoring program:
- 1. Temperature of air starting line near air start valves.
Turbocharoer surveillance The turbocharger is one of the more sensitive pieces of equipment on the diesel engine and is one with a history for failing according to the data collected on aging of nuclear power plants. The failure can be caused by bearing failure due to inadequate lubrication or by imbalance of the rotor. The central bearing models are difficult to monitor because the shaft and bearing are enclosed and the bearing temperature is not accessible. On shaft end bearing models, the bearing temperatures can be monitored. In addition, the rpm can be monitored on both models, although . most easily on the shaft end bearing models. Why monitor the rpm? A record of the turbocharger rpm versus load should be established. As the turbocharger gets dirty on the air side, the rpm will decrease because of the dirt, and this can be detected. Also, the discharge pressure and temperature from the turbo-charger should be noted for the same load conditions. On a shaft end bearing model a tachometer may be used to detersine the r On the more coamionly encountered central bearing model,pm.a magnetic pickup could be installed on the rotor housing to denote the rpm. In either case, this gives an indication of turbe-charger performance, a deterioration of which will indicate a need for physical inspection. Parameters useful in a condition monitoring program:
- 1. Turbocharger rps versus load.
- 2. Turbocharger discharge pressure versus load.
- 3. Bearing temperatures where available.
O hgust Valve surveillance Burned exhaust valves allow combustion gases to leak into the exhaust manifold with a resulting high exhaust temperature for that cylinder as an indicator. Most engines have the exhaust manifold temperature measured as well as the temperature enterir.g the manifold from each cylinder. Another reason for a high exhaust temperature, which may cause burned exhaust valves, is a fuel oil pump rack setting that is too high. During engine o peration the settings of the fuel oil pumps should be noted. Tiey should all be the same, or nearly so. 44
i i Parameters useful in a condition monitoring program: j
, 1. Fuel oil pump rack settings taken while the engine is operating.
- 2. Cylinder exhaust temperature, tube oi19~ survei11ance I Adequate lube oil pressure is essential to the safe operation of I the diesel engine, so the monitoring of the lube oil pump's per-
, formance is equally important. he 's performance is a function of four variables, the lube et ressure and temperature ; leaving the pump, the viscosity of the of entering the pump, and ; the engine load. Measuring the discharg pressure from the puen is not sufficient for trending pump per ormance, thou h it is e , ma or indicator, as it is affected by the other varia les. The ! oi viscosity will vary with age, for the same temperature conditions, because of contaminants, and is determined from the ( t i lube oil sample. ; parameters necessary for determining lube oil pump performance: l ( j 1. Lube oil pump discharge pressure and temperature. l l 2. Engine load. l l 3. Lube oil viscosity adjusted to pump inlet conditions. ! Gavarner survei11ance ( Correct governor control is essential to the starting and operat-
- ing of the diesel engine and there is little that can be done
. operationally to mon' tor the hydraulic governors found on (
- engines. There are two static checks that can be made on a j routine basis while the engine is not running. These checks
- increase the likelihood of correct governor act< on. One, check the oil level in the governor. If it needs oil, edd it, but this is an indication of an oil leak at the shaft seal from the governor. Two, check that the linkage from the goverr.cr to the
, fuel racks is not binding, that it operates smoothly. This i allows the governor to control the racks, preventing overspeeding l vf the unit on startup. j parameters useful in condition monitoring of the governor: ; l
- 1. Static check of the governor oil level.
I j 2. Check governor linkage to fuel racks for ease of motion, f tube 0f1 Heater survei11 ante { .i i Heated lubricating oil is circulated through the diesel engines t in the keep warm condition. The heater is usually an electric l l l 45 ( l
6 immersion heater that can have carbon butidup occurrirg on the
, heater element due to local overheating of the lube oil. Tne heater will function well under these circumstances, but the carbon may flake off, contaminating the lube oil. The heater element should be checked for cleanliness during overhaul periods. This is a static check. , Parameter useful in condition monitoring programs:
- 1. Check heater element for cleanliness during major engine overhaul.
l Coolina Water systee Survalliance i There are several elements to the cooling water system that can , be checked during static conditions and during the operationa' mode. During static conditions the water inhibitor (potassium l chromate, antifreeze) level can be checked. Any decrease will ' i indicate a leak in the cooling water system. During overhaul 1 periods, inspect the jacket water heater element (if electric) for scale buildup. If the heat exchanger between the engine i cooling water and the primary coolant uses salt water, check the - zine plates to ensure protection from electrolytic action. If , l l other systems are used for electrolytic protection, check those : during overhaul periods so as not to remove the heat exchanger ! ! from operational readiness. The heat exchanger cleanliness may f ! be detennined by monitoring the temperature change across the heat exchanger for given load conditions, a decrease in delta i 1 temperature indicating fouling of the heat exchanger. This may be a problem during warm weather whether or not salt water or fresh water is used as the primary coolant since plant and crustacean life abounds in the warm environment of the heat exchanger. Parameters useful in a conditicn monitoring program:
- 1. Measure water inhibitor concentration level during static j checks of the engine.
I 2. Check jacket water electric heater element for scale during overhaul periods.
- 3. Check electrolytic protection systems during overhaul periods.
- 4. Monitor temperature change acress heat exchanger during i operation versus load for fouling of heat exchanger, i
l I. i 4 46
l i t I ! 5. ED6 CO*CITION NOMITORING CONSIDERATIONS This section presents engineering condition measurements that are ;
! recommended by diesel generator experts as needed for *. rending purposes to l detect degraded diesel performance. Also presented are some er.gineering 1
considerat'ons for performing trending analysis. The information here i supplements the general discussion of condition monitoring that is contained a in Section 3.3 and Appendix C to NUREG/CR 5078 Volume 1, and should be
! useful to NRC onsite inspectors and plant personnel. Note that the Appendix C discussion in NUREG/CR 5078 treats condition monitoring as one part of the performance monitoring element, the other part being reliability monitoring, which is not addressed hora, j i Conditlan Monitorina Paranators with Their Frecuencias Obtaining information for condition monitoring may be done l directly, for example, direct measurement of the moisture content ;
)' of the air start system, or direct observation of corrosion or i burning of electrical contactst or it may be done indirectly; for t example, measurement of metallic particles in the lubrication system as an indicator of bearing or cylindtr wear, or measure- ! rent of acoustic vibrations as an indicator of crankshaft align. ( )! ment problems or bearing wear. To be effective, the condition ! I monitorin program must be applied to engineering conditions that l are: (1) haracterized by a measurable, precuisor condition that : 2 is known to be related to an important EDG failure mode; and (2) l conveniently and practically measured withiut incurring an i inappropriately large EDG outage time. ! i Table 51 indicates static measurements and their frequencies 1 that can be made as part of a condition monitoring program. Table 5 2 presents dynamic checks. Checks that may be unusual, and utay not appear on a plant's present list, are asterisked for attention and consideration for adoption. All the condition monitoring suggestions from the surveillance section of this ) report are included in the dynamic and static checks. A l su ested frequency for static checks is noted in the parentheses ! fo owing the parameter, e.g., each shift, daily, weekly. In 1 addition to the checks listed in Tables 51 and 5 2, all measure-j ments taken during an engine overhaul as per manufacturer's
- recommendations should be included. It is important that the j utility obtain'and maintain these data from the vendor.
! In addition to the checklist in Tables 51 and 5 2, the following ! items were noted in the surveillance section and should be in-
- cluded in the overhaul static check data.
l l l 51 1
TABLE 5-1 STATIC CHECKS OF THE ENGINE BY ENGINE SYSTEM ENGINE STRUCTURE
- 1. Check the lube oil level in the crankcase (shift).
2.* Check head gaskets and water manifold gaskets to and from engine cylinders for water leakage (daily). DRIVE TRAIN--nr special checks STARTING COMP 0NENTS
- 1. Barring device disengaged (shift).
- 2. Check startint air compressor for overheating (shift).
- 3. Blow down air receiver (shift).
- 4. Check air compressor lube oil level (shift).
5.* Blow down air drier (shift).
- 6. Check air receiver pressure (shift).
7.* Check air motor lubricator oil level (shift). VALVE MECHANISMS--no special checks FUEL COMPONENTS
- 1. Check day tank fuel oil level (shift).
2.* Duplex strainer!111ter handle not in mid position; flow through only one and nott which one in use (shift).
- 3. Fuel oil filter differential pressure (shift).
- 4. Fuel oil storag6 tank level (daily).
INTAKE AND EXHAUST--no special checks LUBRICATION COMPONENTS
- 1. Lube oil temperature and pressure in kee;, warm system (shift).
- 2. If appropriate for given system, check lube oil filter ar.d strainer differential pressures (shift).
52
TABLE 5-1 (Continued) STATIC CHECKS OF THE ENGINE BY ENGINE SYSTEM 3.* If appropriate for given sy5 tem, check piping system for leaks (d ail.v) .
- 4. Lube oil level (shift).
COOLING COMPONENTS
- 1. Check jacket water temperature and pressure (shift).
- 2. Check jacket water / cooling water expansien tank level (shift).
- 3. Check for leaks at piping connections and radiator (daily).
4.* Check inhibitor level / antifreeze level (per manufasturer). GENERATOR COMPONENTS--no special requirements
~
SW:TCHGEAR COMPONENTS Check the following for the circuit breakers and/or motor cor.tro11ers on a week 1v basis:
- 1. Appropriate ones are racked in.
- 2. Remote / manual controls in remote.
- 3. Control fuses installed.
- 4. Power to break verified.
- 5. Auto / manual switches in auto.
- 6. Aligned to appropriate power source.
- 7. Fault indicators.
INSTRUMENTAT!0N AND CONTROL
- 1. Check governor lube cil level (shift).
- 2. Check alarm indicator lights (weekly).
- 3. Check governor setting. wto or manual (shift).
I
- 4. Check that the remote /%1 start switch is in the remote position (shift).
- 5. Check that The auto / manual start nitch is in the auto position (shift).
5-3
.s o TABLE 5-1 (Continued)
STATIC CHECKS OF THE ENGINE BY FJNi!NE SYSTEM 6.* Check that the governor lead limit is set to the maxt=u: Ic:d - position (shift).
- 7. If the plant has a separate control air system:
! a. Check compressor for overheating (shift).
- b. Blow down air receiver (shift).
- c. Check air compressor lube oil level (shift).
- d. Check air receiver pressure (shift).
MECHANICAL SYSTEMS--A visual inspection of all hancers and supports (weekly). ELECTRICAL SYSTEMS -no special check. - STRUCTURAL COMPONENTS 1.* Hamer test the engine mount holddown bolts (6 months). MISCELLA Y G
- 1. Ctack annunciator circuit (shift).
- 2. Check EDG fire suppression system (daily).
5-4 ;
.w ,
l TABLE 5-2 DYNAMIC ENGINE MEASUREMENTS Measured Parameur Generator Load (kW) Volts Frequoney Amps Kilovars Alternator winding temperature Alternator bearing temperature EDG vibration (mils) Engine rpm -
- Turbocharger rpm
- Time required for starting
- Fuel rack position for each cylinder. As a minimum the rack position for #1 cylinder.
- Governor load position Blower inlet pressure or vacuum Blower inlet temperature Blower discharge pressure Blower discharge temperature Intake manifold pressure Intake manifold temperature Crankcase pressure Fuel oil header pressure Fuel oil filter pressures in and out Lube oil temperature (if necessary) l l
5-5
.e .
TABLE 5-2 (Continued) DYNAMIC ENGINE MEASUREMENTS
~
Measured Parametar (Continued) Lube oil pressure to engine at the header Lube oil pressures entering and leaving filter Lube oil pressures entering and leaving strainer Turbocharger back pressure Lube oil pressures entering and leaving turbocharger Lube oil temperature from turbocharger Lube oil cooler temperatures entering and leaving Lube oil temperatures to and from the engine Jacket water pressures to and from the engine - Jacket water temperatures to and from the engine Jacket water cooler temperatures in and out Jacket water cooler pressures in and out Air intake cooler water pressures to and from cooler Air intake cooler water temperatures to and from cooler Water pressure to and from turbocharger Water temperature to and from turbocharger Exhaust temperature for each cylinder
- Minimum exhaust temperature and cylinder number
- Maximum exhaust temperature and cylinder number Combined exhaust temperature and pressure to turbocharger Exhaust temperature and pressure from each turbocharger
- Lube oil sample from header during loaded condition
- Crankcase door or vicinity temperatures
- Air start line temperature near each air start valve l 5-6
j .. TABLE 5-2(Continued) DYNAMIC ENGINE MEASUREMENTS During EDG tests at full load for :::h cylinder, cht:in th: fellcwing - measurements:
- 1. The cylinder firing pressures
- 2. All fuel rack settings at this time
- 3. All exhaust temperatures at this time 5-7
Useful Overhaul Sunnlemental Data
- 1. Measurements of last engine bearing nearest generator.
- 2. Lube oil heater element cleanliness.
- 3. Jacket water heater element cleanliness.
- 4. Check electrolytic protection systems.
- 5. Deflection measurements of the crankshaft should be made to ensure proper alignment with the generator and the support bearings.
- 6.
- Calibration of alarms and sensors.
- 7.
- Check for scale on cooling passages in heads.
Some of the most useful types of measurements on an EDG are those obtained while the equipment is running. In this way it can be determined how well the unit is functioning without disassembly. Furthermore, it may be possible to determine the wear as - translated into decreasing performance from trending the data. Table 5-2 includes all the measurements found in Table C-4 of NUREG/CR-5078, Volume 1, as well as additional readings that may be worth taking (noted with an asterisk). Trend Analysis The purpose of trend analysis is to plot a given parameter with time, such as exhaust temperature for a given cylinder. A change in temperature for constant load conditions will indicate that something is awry, and further investigation is called for. Trend analysis is part of condition-based maintenance, where maintenance ar.tions are determined in part by the operating characteristics of the machinery. Condition-based maintenance has been developed in a scientific manner in the past two decades. It is true that maintenance engineers have for genera-tions employed the human senses of touch, sight, smell, and hear- ' ing in assessing the onset of mechanical faults; it is, however, the more recent development of a wide range of condition monitor-ing equipment which, by significantly supplementing the human senses, has taken condition monitoring to the point where it now is accepted as an important part of preventive maintenance. Whatever approach to condition monitoring is chosen, it will be necessary to set limits at which some defined action must be taken. These limits might be defined as absolute values for a given parameter or function or as a change from a periodically recorded level. Condition checking, being a single measurement (as opposed to a change from a previous level), must be compared to an absolute limit. Condition checks, by their very nature, are usually required in situations where limits can be set on design grounds, for example, the minimum oil pressure required. 58 i
- o '
O l l i A more significant problem arises when prediction is required and ! lead time is critical. If extrapolation is to be used, such as l in the wearout portion of a component life-cycle, then the deterioration / time relationship needs to be defined with some precision. It seems probable, particularly in wearout cases, that the relationship will be curviline&r in form, and hence good definition is essential if effective extrapolation is to be undertaken. It is important to use as accurate data as possible in trending since the initial slope changes will greatly impact the prediction of wearout. The use of data obtained under transient conditions would be suspect and not a good choice on which to make an extrapolation. Hence, sensor calibration and a steady and known load condition are essential for accurate and useful trend analysis. Trend analysis is effective with changes that are gradual, creat-ing a long-term predictive system. However, equally important are short-term condition monitoring systems used to predict and avoid unexpected malfunctions that arise suddenly and would turn into serious problems quickly (for instance, seizure of a piston if not detected and corrected). An example of long term wear would be measuring the liner bore and plotting these data with . time (normally run-hours). This is normally predictive and can be extrapolated as to when the liner will no longer be useful. An example of a short-term wear problem would be piston ring wear, which can change suddenly and create an engine operating problem in a short time. There are several short-term parameters that should be observed as the engine is running, exhaust gas teeperature being the most important. An example from an engine manufacturer will illustrate the point. An engine equipped with an exhaust gas temperature monitor was run for several hours under full load, and then the load was reduced to a low value prior to stopping the engine. When this happened, suddenly one cylinder shewed a temperature increase of several hundred degrees. Upon shutdown of the engine and subsequent inspection, it was determined that at full load one of the fuel racks had loosened and stayed at full stroke as the load was reduced. With less air provided by the turbocharger, the temperature of that cylinder reached a very high value. Exhaust gas temperature monitoring of each cylinder provides a cumulative view of the whole fuel injection ~ system, combustion process, and valve phasing. It becomes an importan+. parameter to monitor for trending and for short-term failure analysis. Two other parameters that are important are bearing temperatures, which are difficult to measure directly, and crankcase over-aressure. On large two stroke cycle engines it is possible to lavo a thermocouple inserted directly into the bearing cap to constantly measure the temperature at that point. A rapid change in crankcase pressure could indicate piston ring failure (allow-ing blowby) as the most probable cause. Broken rings can cause 59
4 g O g scuffing and possible seizure of the piston, causing engine failure, so detection of this condition using a condition monitoring program is important. The items listed in Tables 51 and 5-2, static and dynamic checks on the diesel engine, fall under the c:t:gery of condition ' checks. These conditions should be checked against established values, where the values are established either on an engineering perfomance basis or from past measurements when the conditions were known to be stable. If the conditions move from the previously established values, or begin to trend toward engineering performance levels that are known to be associated with degraded performance, corrective action is indicated. i Measurements made during overhaul on bearings and other : components should be recorded and tracked with time. These items are the long-term condition monitoring features. These items will, in most circumstances, change gradually with time unless an unusual engine operating condition, severe overload, or inade-quate lubrication on startup is experienced. For the dynamic measurements, the readings should be taken consistently at the same load setting and after the engine has reached steady operating conditions. This is determined by wait- - ing for no further variations in temperature or pressure at a given load. Normally an hour at full load would be sufficient , time to reach steady or quasi-steady state, and the readings , would no longer vary with time to any appreciable degree. On any component a differential change in value will give an indication i of the unit's operating performance and hence condition. When these data are plotted, it can be detemined if problems are developing or not. For instance, if the lube oil temperatures
- entering and leaving the cooler increase, it could be evidence of heat exchanger fouling.
For the engine itself the last three items mentioned, cylinder firing pressure, fuel rack settings, and exhaust temperatures, will provide a wealth of information about the condition of each I cylinder, its injector, valve timing, and piston seals. By tracking this performance with time (trending), gradual deteri-l oration can be detected. For short-tem monitoring for sudden failures or rapid deterioration of engine perfomance leading to failure, the para-meters must be monitored continuously to be most effective. This i is particularly the case with exhaust gas temperatures and may be an item that should be addressed in the condition monitoring i program, Crankcase pressure is easily measured, but bearing temperature on medium speed engines is not as readily determined, except through indirect measurements as previously described. t 5 10 t
- 6. EDG INTERACTIONS WITH OTHER SYSTEMS AND AGING EFFECTS THAT IMPACT EDG RELIABILITY Section 6.1 discusses EDG interactions with instrument air, service water, and de power systems since these interactions can adversely affect EDG reli-ability. Section 6.2 is a limited sunmary of the effects of aging on EDG components. It is included as a separate section since aging effects are important to understand for all three of the EDG reliability program elements: maintenance, surveillance, and condition monitoring.
6.1 EDG Interactions with other Systems This section presents a generic review of the interactions between EDGs and other support and safety systems. Risk-significant interu:tions with the instrument air, service water, and de power systems have been revealed by probabilistic risk assessments and other analyses, as well as by actual EDG failure. These interactions must be reviewed when establishing a relia-bility program for EDGs to ensure that there are no design or operational characteristics related to systems interactions that would prevent the EDGs from achieving their target reliability. Using the experience on past adverse interactions presented here, each plant should investigate systems interactions that involve their EDGs since these interactions are plant-specific. EDG interactions with the instrument air, service water, and de power systems are discussed under separate headings in the following three sub-sections. 6.1.1 EDG interactions with Instrument Air System The instrument air (IA) system in a nuclear power plant provides compressed air of high quality for the operation of pneumatic instrumentation and controls. At most nuclear plants, the IA system is not a safety grade system. The IA system, however, is ubiquitous in that it reaches out to many systems and components, some of which are safety grade. This section will examine IA/EDG interfaces, and actual and potential failures documented for IA/EDG interaction (from LERs). Three principal IA/EDG interfaces are identified: e Air operated valves for component (EDG) cooling water, e Air operatad dampers for EDG room cooling, o Air-operated dampers for EDG exhaust. The component cooling water (CCW) system is a safety grade system at all plants. Certain of the CCW system valves are air-operated, including those which, at most plants, control CCW flow for EDG cooling. Dampers for EDG room cooling and for EDG exhaust are air operated at most plants. It is clear from the foregoing that EDGs depend upon IA for startup and for contin ct operation, with dependencies exclusively related to providing cooling of EDG components. 61
Several of the many LERs discussed in Reference n (BNL/SAIC IA Report] concern IA/EDG interactions. These IA/EDG interaction events are discussed below and are presented pictorially as flow diagrams in Figures 6.1.1-1 through 6.1.1-5. Figure 6.1.1-1 is a flow diagram of a potenital generic IA/EDG interaction mechanism, specific for overheating of EDG controls. This potential occurrence has been found at several plants. This occurrence is describable, in generic terms, as a top event that leads simultaneously to a failure path and a success path, but with a failure path that eventually interrupts the success path because of system or component interdependencies. The specific occurrence is a LOOP event which results in loss of IA and in EDG startup. Since the EDGs need proper temperature control for continued operation, and since such temperature control depends on IA system success, eventual heatup could fail the EDGs thus leading to station blackout if offsite power is not restored before EDG failure. Figure 6.1.1-2 is a flow diagram of IA/EDG interaction for EDG coolant over-heating during a surveillance test. This event occurred at the Ft. Calhoun plant. The combination of water backup in an air accumulator and contamina-tion in a pilot valve and in an EDG radiator exhaust damper air motor led to failure of the exhaust damper to open. This failure allowed the EDG cooling water temperature to increase, resulting in an EDG trip on overheating. The contamination entered the components through the IA system, and the water entered the air accumulator from the fire system. Another event representing IA/EDG interaction for EDG coolant overheating, also at Ft. Calhoun, is shown in Figure 6.1.1-3. The root cause of this event, IA system contamination, led to sticking of a pilot valve and eventual EDG coolant overheating with consequent EDG shutdown. EDG failure resulting from a design deficiency which failed to account for IA/EDG interaction is shown in Figures 6.1.1 4a (for Haddam Neck) and 6.1.1-4b (for Maine Yankee). Here, loss of IA resulted in failure of the air-operated valves controlling EDG cooling water, thus causing EDG failure on overheating. In Figure 6.1.1-5, the combined effects of two sources of IA contamination resulted in a manual shutdown of an EDG. This event occurred at Brunswick
- 2. EDG IA system contamination clogged an EDG IA filter. At the same time, crud had accumulated on the seat of an EDG IA filter drain valve. These effects combined to reduce EDG IA pressure below the normal value of 100 psig. As a result, the EDG was declared inoperable.
The events described by these five figures have certain comonalities. e EDGs overheat because loss of IA pressure (total or partial) causes air-operated EDG related valves or dampers to fail, o EDGs overheat because contaminants in the IA system prevent proper functioning of air operated EDG-related valves, dampers, or filters through blockage or piecepart damage. 62 1
l l l l Lees of Offsite Power Fallure Path h sw PaGi 1r q> ! l 14ss ofIA EDG Startup ir 1r Closure of EDG Vent Offsite Power Ea rcf IA-operated Dampers Substitution ar 1P EDG Room Hemtup Continued Station Operation 1P EDG Control Heetup u Failure of EDG Controls Termination of Success Path Disabling of EDGs Interivption by Failure Path 4 4 less of Offsite Power E.m r iSubstitution e ip Station Blackout Figure 6.1.1. l. Generic IA/EDG Interaction Mechantem Potential, Specific for Overheating of EDG Controls 6-3
l
)
)
I Entry of Fire System Water into IA Gunm EM in Pilot Valve and in EDG Radiator Enhaust Demper Air Motor 1P Backup of Water in Air Accumulator 1r 1r 1r Failure to Open of EDG Radiator Eshaust Damper ar High Engine Cooling I Water Tanperature i l t 1r i EDG Trip ( l l l l F1gure 6.1.12. IA/EDG Interaction for Coolant Overheating Dudng a Suneillance Test 6-4
- a Residue in IA U
Sticking of EDG Damper Puot Valve ep Failure to Open Fully of Aireted EDG Exhaust Damper 1P Restrictjon of Air Mow 1hrough EDG Radiator ir BJgh EDG Coolant Temperature 1P Automatic Shutdown of EDG Mgure 6.1.13. IA/EDG Interaction for Coolant Overheating l 6-5
p - q i l l 1 i l 1 I4es ofIA d Air 0perated Control Valves for EDG Cooling Water Fall Open 0 Less of EDG Cooling d Failure of EDGs Figure 6.1.1-4a. IA/EDG Interaction for Cooling Water Design Deficiency IAss ofIA d Closing of Alt Operated Temperature Control Valves For EDG Cooling Water to Both EDGs O Loss of EDG Cooling d Failure of EDGs Figure 6.1.14b. IA/EDG Interaction for Cooling Water Design Defielency 6-6
a o e F EDG IA Contamination Crud on EDG IA Filter Drain Yalve Seat U Clogged EDG IA Filter 1r 1r 1P EDG IA Pressure Drop Below Normal Value of 100 psig gr EDG Declared Leble Figure 6.1.1-5. IA/EDG Interaction for Loss of IA Pressure 6-7
The event described in Figure 6.1.1-1 is unique relative to the other events and is clearly the most serious. Its generic nature -- as a design deficiency l in which a top event failure may lead to a failure path which, because of ! interdependencies, will interrupt the success. path -- transcends the specific l IA/EDG manifestation discussed here. l Based on the foregoing discussion, EDG reliability can be improved by the following actions:
- 1. Reduction of IA contaminants by careful attention to I prescribed filter maintenance procedures, e.g., use of l adequately mesh-sized filters and filter replacement before air pressure reduction occurs. l
- 2. Reduction of IA moisture by careful attention to air dryer maintenance and general maintenance procedures directed ,
toward abnormal moisture entry into the IA system.
- 3. Use of PRA analysis to reveal potential design deficiencies.
6.1.2 EDG Interaction with Service Water System There are no standard service water systems for EDG cooling. Each plant will have a unique system reflecting the plant's original design philosophy. With the caveat in mind that each )lant is unique, three generic types of service water systems for EDGs can )e defined: a) Self contained with no external supply. A radiator cools the water for the engine and auxiliary systems. The radiator's cooling fan is engine driven, not relying on any external power source, b) Use of a fresh water )rimary cooling loop, supplied from an external source, suc1 as a lake. This water, through heat exchangers, cools the engine cooling water and the engine's auxiliary systems. c) Use of a salt water primary cooling loop, to a fresh water secondary loop, which then acts as (b) above. For system (a) above, no additional maintenance procedures are required as they are covered as part of the engine system maintenance. Note that section 6.1.1 indicates a case where the radiator shutters failed to operate correctly, ceusing engine overheating. For systems and (c) additional maintenance is required and may be performed by (5) personnel not associated with the EDG. Figure 6.1.2 1 illustrates case (c). The following are the key components in these systems and an indication of the problems associated with each component. Systems for case (b) include a subset of these items.
- 1) Raw Water Pumps- pumps fail by not producing sufficient head and flow to cool the diesel engine under load. In some instances the pump may fail totally. The insufficient flow 68
failure results from allowing a deteriorated condition to persist, for inst 1nce excessive clearances (impeller to casing, bearings), causing the pump not to be able to produce the flow required. The deteriorated condition is normally age related, but if suction strainers are not functioning properly, debris will accelerate wear.
- 2) Piping--piping fails primarily due to physical deterioration due to age. Depending on climate, exposed piping should be insulated to prevent freezing in the winter.
- 3) Jacket Water Heat Exchanger -this is covered under the i engine system maintenance.
- 4) Sea Water Supply Pumps- pumps fail by not producing !
sufficient head and flow to cool the diesel engine under load. The causes are the same as in item (1) plus, if suction strainers are not functioning properly, debris from the salt water source will accelerate wear. Salt water is a highly corrosive liquid and all metallic surfaces are subject to corrosion and electrolysis.
- 5) Cathodit. Protection because of the electrolytic action of the salt water the cathodic protection for all components must be maintained. This is done by periodic inspections of the components.
- 6) Strainers--strainers fail for two primary reasons, physical breakdown due to age and clogging due to not maintaining a proper cleaning schedule. The cleaning schedule may have to vary as a function of the water condition, since there are seasonal variations in water quality. In the winter freezing conditions may in some circumstances cause ice crystals to form even in salt water, causing a dramatic drop in water flow.
- 7) Heat Exchangers- the heat exchanger between the salt water and the raw water system needs special attention, particularly on the salt water side due to potential fouling and electrolytic action. Heat exchangers fail by leaking or ;
by insufficient heat transfer. Periodic inspections of the tubes are required to assure cleanliness and proper installation of seals and gaskets during maintenance and resair operations. Care must be taken when heat exchanger tuses are bored out to remove scale, as the tube wall may be cut in the process. The potential for interaction with other systems does not end here, however. No nuclear power plant will have a sea water supply system, or a raw water system just for the EDG, but rather for the entire facility. Additionally, on some plants the jacket water system is interconnected with the containment cooling water system. Since these systems are coupled, there must be a comunications link with personnel in other maintenance areas to ' i 69
ensure actions in those areas, or in the EDG area, do not have unanticipated adverse impacts. 6.1.3 IDS Interaction with DC Power System Just as with the service water system, th:re 1: r,c :t:ndard de power system and several variations exist. However, ths EDG de power system in most plants is interconnected at some point with the station de power system. The following cases will illustrate two types of de power systems. In case one, the batteries in the EDG room are used to flash the EDG generator field. The station vital power battery bus provides de power to the instru-mentation and control system. The maintenance associated with batteries, regardless of the system, has three aspects: a) making sure the liquid level is correct; b) checking that the liquid specific gravity is appropriate; c) assuring that the battery charger is functioning properly and that the terminal connections are clean. Batteries primarily fail by not holding charge due to age. . In case two, the EDG system and the station system are totally interconnected. There are no batteries in the EDG room, but rather the station 125 volt vital ac bus, through an inverter, maintains constant charge on the station batteries. These batteries are used for EDG generator field flashing on startup. Additionally, the de power created by the inverter is used for instrumentation and control. The primary failure mode associated with inverters is overheating. Adequate ventilation and cleanliness of the inverters will assure that the required heat dissipation occurs. 6.2 Effects of Acina As en EDG ages, some types of failures are more itkely to occur than others. This is where additional vigilance is needed in the preventive maintenance program and in the condition monitoring of the engine. Once the engine reaches a certain level of deterioration, it can be completely rebuilt, so, when examining engine failures, the age of the engine is determined from its last rebutiding, not from the day it was installed. The subsystem failures should be viewed in the same manner. In the study of failures related to aging, there is a failure curve, sometimes called the ' bathtub" curve, that indicates the change in failure rate with age. This is shown in Figure 6.2-
- 1. Thert is a "wear in" portion, with a high failure rate associated with many pieces of new equipment as shown by the curve. Once the machinery is broken in, tne failure rate is at its lowest and remains reasonably constant as indicated by the maturity section of the curve in Figure 6.2-1. As the
, machinery wears and reaches the end of its lifetime, the failure rate increases again as shown on the curve. The trick is to determine the time scale for these regions for each piece of equipment. An additional complication is that a change of environment or operating conditions can change the time scale and the failure scale. 6 10
7
*Q t '
I i i 4 e i 2 u
~
L
.. U I) &
S-2 a # .
! 5 Ei ng >j ;
Il la w E
- t. ,E II u y 2
.5 s
N e G
'I g y,
%h, 4
7 . - .- A study was undertaken by the NRC on the "Aging of Nuclear Station Diesel Generators: Evaluation of Operating and Expert Experience," NUREG/CR 4590, published in August 1987. This study reports an analysis of the breakdowns and failures in EDGs over a 20 year period. Through the analysis of these data and from expert opinion, the effect of aging for EDG systems was deter-mined. The study further looks at the specific types vi aving associated with each manufacturer. The remainder of this section is a very limited summary on aging effects on EDGs taken from NUREG/CR 4590. For additional experience data on the effects of aging on EDG component failures and causes and expert opinions on those EDG components most susceptible to aging effects, see NUREG/CR 4590. Like all mechanical and electrical equipment, diesel generator systems and components are subject to aging. Each system and component is affected by its operational history and environment. Engine systems deteriorate, or age, both while operating and standing idle. Since diesel generators in nuclear service are not operated for most of their installed lives, the diesel generator and certain related systems should be considered unique from the aging standpoint when compared to many other systems in a nuclear facility, or diesel generators in non nuclear service, which are used extensively. The following describes in a general way the factors affecting the engine's systems with age. . Coolina Water Corrosion inhibitors, used to control chemical attack or electrolytic action between different metals in a cooling system, become depleted and break down during operations or long periods of inactivity. Uncontrolled water chemistry results in sludge, hard water inorganic deposits (scale), and rust. Intercoolers, heat exchangers, and other engine components c4n be affected by corrosion due to poor water chemistry and scale. Rust and sludge can cause uneven heat transfer, accelerating corrosion, and thermal stresses to build up in areas of high rates of heat transfer within the engine. Lubricatina 011 When lubricating oil is used for long periods of time, the aging arocess includes effects of increased organic impurities or sacteria content. Wattr c.cntaminants and microbial growth have been found in some oil sy:tems. Wear metal and corrosion products catalyze reactions within the lube oil, adding to the aging process. Operation of the keep warm system can cause the oil to oxidize and break down. During startup and non stabilized engine operation, the buildup of lube oil contaminants is accelerated and the additives in the lube oil to combat them may not have time to react completel is running. This may accelerate engine wear.Duringy while the engine operation the contamina-tion of the lube oil with fuel oil and water will reduce its effectiveness. Periodic testing of the lube oil is required and certainly using the lube oil reconnended by the manufacturer is a necessity, i 6 12
r- W L fuel 0f1 The storage of distillate fuel oil can result in physical separation, thickening, and chemical changes due to aging processes. When the oil is stored for long periods, water can accumulate and microbial growth can fer:::. The:o factors can f cause filters to plug rapidly and cause corrosion and seizure of injection system components. Startina Air When a pneumatic system is static for long periods, water condensate and corrosion may degrade components such as valves, orifices, and controls. Starting air systems, particularly those operating in humid environments, must ensure adequate air drying capabilities to reduce the effects uf moisture introduced into the system. Intake and Exhaust Intake and exhaust systems are subject to the cumulative effects of dust, moisture, other air contaminants, thermal stresses, vibration, and corrosion. Alkali deposits (sulfur dioxide, for instance) in the exhaust system can cause rapid corrosion by attracting airborne moisture. Nonmetallic Deterioration The nonmetallic materials used in seals, gaskets, and hoses are subject to aging by oxidation and oil-induced degradation. Thermal cycling, with brief hot periods followed by long, cool perieds and frequent component removal for inspection, can affect the life of the nonmetallic sealing components. The packing or mechanical seals of engine driven pumps wear as a result of engine operation, and this is aggravated by engine vibration. Dynamic Stresses Diesel generators are subject to extreme dynamic loading, especially during fast-start conditions that place unusual demands on electrical and mechanical components. Furthermore, some engine parts, such as pistons and cylinder heads, are subject to dynamic thermal stresses. The thermal stressing is due to the unevenness of the thermoclines with the structural components. Thermal Stresses and Thermal fatfaut Some diesel generator components such as cylinder liners and exhaust manifolds are subject to thermal stresses. Bolts under-going great temperature changes can creep (time related failure) and fail or be subject to intergranular failure when exposed to corrosive atmospheres, particularly at high temperatures. Turbo-6 13
o o charger vanes, blades, and inlet vane support bolts are subject to this failure tsode as well as to low cycle thermal stress. Hydraulic Deterioration Components that are subject to aging due to hydraulic forces, such as cavitation, include cylinder heads and liners, water pumps, bearings (especially connecting rod bearings), high-sressure lines, fuel lines (fatigue due to pulsation), and the tydraulic loading of air start controls, valves, and lines. Electrical Deterioration Electrical insulation is subject to aging from oxidation, high-temperature exposura, oil contacination, dust, and vibration. Electrical contacts corrode from atmospheric contaminants and normal aging may be accelerated by the presence of moisture. Commutator and slip ring wear accelerates by dust accumulation between uses or by corrosion due to condensation during periods of inactivity. Probes, lines, contacts, connections, and diodes will all age as a result of vibration, heat, and chemical attack. While all factors contributing to aging cannot be controlled, there are three causes of agtng that plant personnel can control ano that play an
- l important part in preventing failures and improving reliability. First is the environment in which the EDG operates. The key here is to minimize the affects of dust, moisture, and chemicals on the engine systems. Second is the effect of vibration of engine components. It is not possible to eliminate vibration, of course, but it is possible to minimize its effects.
Adequate hangers and supports, tightness of bolts and fittings, all minimize the transmission of damaging vibration. Very often it is not vibration per se that is the problem, but the fact that vibrational effects amplify, causing piping to flex more than it should, that is harmful. The piping vibration adds to the stress and possible failure of the components attached to the piping, such as cooling water and lube oil pumps. Third, ensure through training and supervision, that the maintenance that is performed is done correctly. 6 14 _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ _ _ _ . ____________________}}