ML20212C626

Forwards BNL Draft NUREG, Evaluation of Reliability Technology to Improve & Maintain Emergency Diesel Generator Performance, Discussed at 861218 Meeting W/Nuclear Utility Group on Station Blackout

ML20212C626
Person / Time
Issue date:	12/29/1986
From:	Rubin A Office of Nuclear Reactor Regulation
To:	Maloney S AFFILIATION NOT ASSIGNED
References
REF-GTECI-A-44, REF-GTECI-EL, TASK-A-44, TASK-OR NUDOCS 8612310122
Download: ML20212C626 (113)

Text

.

a '. .

December 29, 1986 Mr. Stephen Maloney Devonrue 108 Lincoln Street Boston, Massachusetts 02111

Dear Steve:

As we discussed at our meeting on December 18, 1986, with the Nuclear Utility Group on Station Blackout (NUGSB0), enclosed for your information is a draft report prepared by Brookhaven National Laboratory entitled " Evaluation of Reliability Technology to Improve and Maintain Emergency Diesel Generator Performance". This report may be of use to NUGSB0 as it relates to NUMARC's station blackout initiative number 4 on diesel generator reliability.

Please note that this report is a draft and does not reflect comments from NRC's review of the report.

Sincerely,

/5 Alan M. Rubin, Sr. Task Manager Reactor Safety Issues Branch Division of Safety Review & Oversight cc: w/o enclosure:

S. Karimian, BNL DISTRIBUTION w/ enclosure P0ygge - _q;j w

DISTRIBUTION w/o enclosure:

Central File DSR0 Chron RSIB r/f T. Speis B. Sheron W. Minners P. Norian P. Barnowsky A. Rubin 8612310122 861229 Rubin c/f PDR ORO NRRC PDR OFC :RSIB:DSR0 :RSIB:DSR0  :  :  :  :  :

NAME :ARubin/ab :PNorian  :  :  :  :  :

DATE :12/29/86 :12/ /86  :  :  :  :  :

GnA 0FFICIAL RECORD COPY

N,.s .1. ,

.' GEREG/CR-BNL-NUREG - ,

/

,' DRAFT REPORT ,

EVALUATION OF RELIABILITY TECHNOLOGY TO IMPROVE AND HAINTAIN EMERGENCY DIESEL GENERATOR PERFORMANCE S. Karimian, J.C. Higgins, and J. H. Taylor Division of Engineering Technology October 1986 Brookhaven National Laboratory Department of Nuclear Energy Upton, New York 11973 l

l l

~-

be l' . >

e

, NUREG/CR-3NL-NUREG-l e * *

/

Y DRAFT REPORT EVALUATION OF RELIABILITY TECHNOLOGY TO IMPROVE AND MAINTAIN EMERGENCY DIESEL GENERATOR PERFORMANCE S. Karimian, J.C. Higgins, and J. H. Taylor .

Division of Engineering Technology Data Completed: October 1986 Data Published:

Department of Nucle'ar Energy Brookhaven National Laboratory Upton, New York 11973 i

,,-p _

a

_11 ABSTRACT

.An important factor in determining the capability of nuclear power plants (NPP) to cope with a concurrent loss of offsite and onsite emergency AC power (i.e., station blackout) is the reliability of the plants emergency diesel generators (EDG). Recommendations made regarding a proposed, resolution of the station blackout issue (i.e., unresolved safety issue A-44) include the requirement that the reliability of each EDG be maintained at or above specified acceptable reliability value.

This report documents the process for establishing an EDG reliability program for the purpose of supporting the objectives of improving and main-taining EDG reliability consistent with the resolution of the station' blackout safety issue.

The work reported herein draws heavily on work conducted within the three i

related programs performed by Brookhaven National Laboratory (BNL) for the NRC. Methods, approaches, and strategies used within the operational safety reliability research pr eject for evaluating the applicability of reliability technology to NPP operational safety have been synthesized with a program that analyzed the data and recommendations made by various groups of industries in support of EDG activities. Another program, a review of issues related to improving nuclear power plant diesel generator reliability, reviewed the data submitted by manufacturers, utilities, foreign groups, NRC and other organizations, and especially the data which accompanied the utilities response to NRC Generic Letter 84-15. The methodology considered in this report is largely based on the reliability processes developed in the operational safety reliability research project, which are further elaborated, summarized, and tailored especially for EDG's. Within this process the effectiveness of current EDG activities and industry's concerns regarding these activities are examined to specify the methods that can identify problems and determine the root cause of problems.

e . >

-111-CONTENTS Page ABSTRACT............................................................

SUMMARY

LIST OF FIGURES................~.............................'........

LIST OF TABLES......................................................

1. INIR0 DUCTION...................................................

1.1 Background................................................

1.2 Repo rt Pu rpose and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2. CURRENT PRACTICES..............................................

2.1 Utility Practices to Maintain DGs.........................

2.2 Technical Specifications and Surveillance Testing.........

2.3 NRC Activities............................................

l

3. BASIC APPROACH TO THE EDG RELIABILITY PROGRAM..................

3.1 Structure of EDG Reliability Program......................

3.2 Primary Elements of the Reliability Program for EDG.......

3.2.1 Task I. Problem Prediction and Recognition Element 1. EDG Performance Monitoring & Analysis.... l 3.2.2 Task I. Problem Prediction and Recognition Element 2. Evaluate Reliability in Design

& Ope ra t ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.2.3 Task I. Problem Prediction and Recognition Element 3. Compare Reliability to Target. . . . . . . . . . . .

3.2.4 Task II. Problea Prioritization & Correction Element 4. Emergency Diesel Generator Problem and Correction...........................

3.2.5 Task II. Problem Prioritization & Correction Element 5. Failure and Root Cause Analysis..........

I 3.2.6 Task III. Problem Closeout Element 6. Determine Corrective Action..............

3.2.7 Task III. Problem Closeout Element 7. Implement / Verify Corrective Action.......

3.3 EDG Reliability Program Process...........................

i l

t

.s s e . >

, , -iv-CONTENTS (Cont'd) 4 RELIABILITY TECHNIQUES AND ACTIVITIES FOR IMPLEMENTING RP ELEMENTS.....................................................

4.1 Reliability Techniques....................................

4.1.1 Failure Mode & Effects Analysis / Reliability Block Diagrams.'............................ ........

4.1.2 FRA Based Modeling (Fault Trees)....................

4.1.3 Performance Indicators (CUSUN Technique)............

4.1.k Data Analysis.......................................

4.1'.5 Condition Monitoring (Trending) . . . . . . . . . . . . . . . . . . . . .

4.1.6 Reliability Monitoring..............................

4.1.7 Common Cause/ Common Mode Analysis...................

4.2 Rallability Activities....................................

4.2.1 Data Base Development Activity......................

4.2.2 Operational Activities..............................

' 4.2.2.1 Surveillance Testing........................

4. 2.2.2 Walkthrough Inspection . . . . . . . . . . . . . . . . . . . . . .

4.2.2.3 Preventive / Corrective Maintenance. . . . . . . . . . .

4.2.2.4 Personnel Training..........................

4.2.2.5 Operations..................................

5. INDUSTRY C0NCERNS..............................................

5.1 Dis cus si on of Con ce rn s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5.2 Use of Reliability Program for Responding to 1

Industry's Concerns.......................................

6. ESTIMATE OF LICENSEE RESOURCES TO IMPLEMENT EDG RELIABILITY PR0 GRAM............................................

6.1 Resource Requirements for Develo Program......................... ping EDG Reliability 6.2 Resources Required to Implement & Maintain EDG Reliability Program.......................................

7. REFERENCES.....................................................

APPENDIK A. .De termination of CUSUM Parameters . . . . . . . . . . . . . . . . . . . . . . .

APPENDIX B.. Effectiveness Measures for Maintenance Activities.... .. .

APPENDIX C. . Summary Document for Trial Application . . . . . . . . . . . . . . . . . .

e -me ew- _ _ - - - . - - - . - . . ,,,,,--wmy--.--,,r-- . , . . .--- _ - ___ __,.-__,- -., _ , - _ , . _ - - _ _ , , . . . --_ - - -,._-- - -_-__y_.- - %_ _ _ , -wy--_.-- ,,,, y__,.__,.

t

-v-LIST OF FIGURES Figure No. Title Page 1.1 Operational / Reliability Interrelationship.............

3.1 Structure of Reliability Program for EDG..............

3.2.1 Work Activities "of the EDG PERFORMANCE MONITdRING AND ANALYSIS Element.................................

3.2.2 Work Activities of the EVALUATE RELIABILITY IN DESIGN AND OPERATION Element.........................

3.2.3 Work Activities,of the CCMPARE RELIABILITY TO TARGET Element.......................................

3.2.4 Work Activities of the EDG PROBLEM IDENTIFICATION Element...............................................

3.2.6 Work Activities of the DETERMINE CORRECTIVE ACTION Element..............................................

3.2.7 Work Activities of the IMPLEMENT / VERIFY CORRECTIVE ACTION Element.......................................

3.3 EDG Reliability Process..............................

4.1.1 Failure Modes and Effects Analysis...................

4.2.2.1 Process to Determine Changes in Surveillance Testing Requirement Due to the Increase in the EDG Failures..

4.2.2.3 Maintenance Logic Diagram.............................

a l a

l

.z.

-,y-.-. s-

y. ,

3< .

v.; . 4 ,

e ,, . 3/..1, 4

2. . ., , a a -vi-

~

A '.

. ',' s

% ( ,

i

\. '\ t,- '

LIST OF TABLES g t' a

' Table No.'s ,

Title ,- Page

' .) *

+ yl .y , t.

t

, , 3.2 Tha,ts[ Elements,:and Activities Necessary for an 4ffictive EDG Reliability t,

Program....................

s ,. - t , .

3.2.1 ' /1: st. of ceramete'rs that need to be monitored

/ during the EDG ' Opera tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

\ ,

4.1 Reliabt'.ity Technique and Activities to Accomplish Figure 3.1 0bj ective . . . . . . . . . . . . . . . . . . . . . .

' 4 5.2 t

'etalisbill:y Elements for EDG vs Major Concerns Fro m Indui t ry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

\'

./

f' g- I e

P 1 1 J #

g 4

\

\

5 t

'-i s I 8

's . \

i  !

\ <

r i I 4 f

1 t

i n

3 k

i s

1 g r i t i

c

\ '

t e

J i?

f

,u_ - - .....-,_ .- - - - _ - - , - , , - - - . , . - - - - - . - . . , - - - - - - . - . - - - - - - - - . _ _ . . - - - , - _ . - - , . - - - - - - _ . _ , ,

,, a

.. -y11 1

. t SUMMARi This report demonstrates that a reliability program can be developed using risk and reliability-based techniques that can be integrated within cur-rent plant operational activities to: 1) analyze problems that have affected system performances, 2) forecast the onset of potential problems, and 3) sus-gest actions that could eliminate or reduce their occurrence.

The attributes of a successful reliability program are summarized as fol-s lows: ' '

It is important to have a closed-loop program,#

i.e., as problems are detected, corrected, and closed out the component continues to be monitored to assure that problems do not reoccur. '

A systematic approach in incorporating reliability technology in the day-to-day activities of EDG operation is an important attribute. A

successful reliability program for EDG's is one that is developed.and implemented on a continuing basis, as opposed to the one that is re-surrected only when a problem comes up.

A successful reliability program should include both prognostic, as well as diagnostic reliability problem detection. w In this report, seven reliability elements were identified, as necessary',

for accomplishing the objective of the program (i.e., maintaining or improving -

reliability of the EDG) and they are:

Element 1 - Performance Monitoring and Analysis 2 - Evaluate Reliability in Design and Operation 3 - Compare Reliability to Target G

. i #

?- -

-viii-4, -

-6' -

y . ,,

4- ~ %,$ .

" l 4

4 - Problem Identification l

' S - Failure and Root Cause Analysis l si 6 - Determine Corrective Action 7 - Implement Corrective Action.

Figure 3.3 shows the overall reliability process in which these elements interact and is included in this summary. Portions, or part of each of these elements, may be currently performed by plants without reliability programs.

However, through the conduct of a comprehensive reliability program, the re-

,~, sulting effect on operational activities will be more effective performance.

Several of these elements, such as performance monitoring (reliability and conditional monitoring), evaluation of 'the reliability in design and oper-ations, and compara reliability to target are not usually performed by plants unless a systematic use of reliability technology has been adopted.

Referring to Figure 3.3, as long as reliability of the EDG is maintained above or equal to the reliability target, the plant would maintain pre-estab-lished operational activities. If measured reliability is lower than the tar-get value, or exhibiting a downward trend, then the full reliability process is implemented as illustrated in the lower loop of the figure.

The NRC Draft Regulatory Guide for Station Blackout has established a reliability target of 95% for EDG performance. This report establishes cri-teria for licensee actions based on this target value. Providing an adequate reliability program is in place, this suggested criteria could recult in less

, surveillance testing of diesel generators. This is described in more detail

[

!- in Section 4.2.2.1.

+

w w

. e

\

60NITOR EDG PERFORMANCE RELIABILITY. EVALUATE RELIABILITY IN COMPARE RELIABILITY IRELIABILITY & CONDITION 5 DESIGN & OPERATION r TO TARGET MONITORING) DATA

/m f 3

' I

! NO PROBLEM-CORRECTIVE  !

ACTION IS ' I IMPLEMENTED  !

MAINTAIN RELIABILITY k PROBLEM

& VERIFIED - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

V

, IMPLEMENT AND VERIFY PROBLEM CORRECTIVE ACTION IDENTl/ICATION

'l IW ROVE RELIABILITY CORRECTIVE 4 . k8 ACTION ,

FOUND '

l .

DETERMINE CORRECTIVE PROBLEM CAUSE IDENTiflED FAILtRE AND ROOT CAUSE ACTION -< ANALYSIS g FIGURE 3.3 EDG RELIABILITY PROCESS

.x-Current industry practices to maintain EDG performance is presented as well as industry recommendations for areas needing attention to improve EDG reliability.

o It is demonstrated how an adequate reliabilit,y, program addresses ,

thes/ areas of concern.

Resource ,. estimates' are provided for both establishing a reliability program, and for maintaining it on an on;;oing basis. The estimate for the former is 10 man months, and 2-5 days per month for the latter. However, it is pointed out that if a " systems engineer" concept is in place at a plant, then the resource requirement for program upkeep would be minimal.

- ------w -- -g - - - - - - - - - - - -

., 4,

-1

1. INTRODUCTION l.1 Background Current nuclear power pla' n ts (NPPs) utilize a large amount of AC electric

^

/

motor-driven equipment.- Alternating current (AC) electric power is also used

/

for a number of other applications important ' to plant operation and safety such as ' instrumentation, control, and battery charging. In the event that normal AC power from the offsite utility distribution network or from the NPP nain generator output is not available, then a reliable backup source of AC power is needed to power safety related equipment. With only a few exceptions, commercial NPPs in the United States use emergency diesel generator (DG) units as a backup source of AC power. These DG units consist of a diesel driven engine connected directly to an AC generator. They are safety grade and are normally arranged so that separate DGs supply each of the two, three, or four redundant electrical divisions of the NPP. The DGs are typically designed to start automatically, be at rated speed and voltage in 10 seconds, and to accept full load within one minute.

If a NPP were to lose all offsite electric power being supplied to the plant and if, coincidentially, the onsite emergency DCs also failed, there would be no AC power to operate the plant's equipment. This is termed a sta-tion blackout and has been considered an unresolved safety issue (A-44) by the U.S. Nuclear Regulatory Commission (NRC). Batteries and steam-driven equip-ment could still operate for a period of time ranging between 4 to 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> l

depending on plant specific capacity of DC power and condensate storage tanks.

If within a period of 4-8 hours AC power is not regained, the potential for i core damage is significant due to the inability to remove decay heat from the reactor. Probabilistic Risk Assessments (PRAs) detail the core melt i frequencies from various accident scenarios. Studies (NUREG/CR-3226 1 and

.. 1 7

NUREG/CR-1032 ) have shown that the contribution to overall core melt frs-quency from accident sequences related to station blackout can be signifi-

-dant. As part of the proposind technical resolution to the station blackout '

/

(A-44) issue, the NRC staff is considering new requirements to reduce the risk of core damage such as mandated minimum DG reliability, and design features to allow a plant to survive 4 to 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> without AC electric power. Since the reliability of NPP diesel generators is one of the factors which affects the risk from station blackout, the achievement of high reliability of DGs has been considered important and has received considerable attention.

Over the years, there has been concern about the reliability of indivi-dual DGs and overall onsite AC power systems at NPPs. Failures of Trans-america Delaval Inc. (TDI) DGs at preoperational NPPs contributed to these concerns, as did a number of incidents at operating NPPs involving other vendor's DGs. There was also concern expressed that the NRC DG testing re-quirements were contributing to DG wear. out and unreliability. As a result, in July 1984, the NRC issued Generic Letter 84-152 to (1) obtain information on the reliability of DGs at all opersting NPPs, (2) reduce the number of cold fast starts, and (3) obtain industry comments on NRC proposed actions to be taken to maintain and assure DG reliability. NRC also received numerous comments and recommendations relating to DG reliability from various other industry sources. All of these comments and recommendations were analyzed and summarized in NUREG/CR-45573 .

Brookhaven National Laboratory (BNL) has developed guidelines for an Operational Safety Reliability Program, NUREG/CR-4618 for any equipment in a Nuclear Power Plant. This program was utilized in a trial application (App. C) to develop a plant specific reliability program for the emergency diesel generators at the Trojan Nuclear Power Plant 5, 1 -___

a a o ..

i

-3 1.2 Report Purpose and Scope The purpose of this report is to develop a generic diesel generator reli-atfiliyt program which would ens're that DG reliability is maintained above the '

u minimum specified level necessary to minimize plant risk from station black-out event. .This generic DG reliability program is based on the Operational Safety Reliability Program and contains important and appropriate insights gained from industry experience and comments documented in Ref. 3.

The reliability program's purpose is to ensure adequate DG reliability and availability. This is accomplished by optimizing the various normal plant operational activities and by supplementing these with certain new reliability activities. This interrelationship is illustrated in Figure 1.1.

The reliability program for DGs presented here is basically designed with two levels of effort. The basic program would be used by those plants with satisfactory DG reliability above the target of 0.95 (i.e., fewer than .05 failures per demand). This level of effort would serve ::o maintain reli-ability above this minimum desired value. If DG reliability were to drop below 0.95, then additional actions would be necessary to improve the DG's reliability. These additional actions constitute the second or higher level of effort.

l l Section 2 of this report describes the current practices of the U.S.

nuclear power industry in the diesel generator area. It illustrates particu-larly the range of programs and activities that currently exist. Section 3 outlines the overall structure of the proposed EDG reliability program.

Section 4 provides more detail into the reliability techniques and activities necessary to implement the reliability program. Section 5 summarizes the cur-rent concerns of industry in the DG area and then illustrates how and where the proposed reliability program addresses these concerns. Section 6 provides

^* '

.; a l

.. _4 an estimate of the manpower necessary to implement such a reliability program at a typical NPP.

o . -

/

ESTABLISH PLANT OPERATIONAL REQUIREMENT PERFORMANCE RELIABILITY TARGETS V

ON COING PLANT OPERATIONAL ACTIVITIES OPERATIONS MAINTENANCE -

MODIFY PLANT ACTIVITIES SURVEILLANCE TESTING TO ACHIEVE PERFORMANCE TRAINING AND RELIABILITY TARGETS DESIGN SPARE PARTS - ^

(

ONGOING RELIABILITY ANALYSIS NO ACCEPTABLE RESULTS MEETING ALL TARGETS

{

YES V

OPTIMAL PLANT PERFORMANCE AND RELIABILITY Figure 1.1 Operational / Reliability Interrelationship I

t. ., -

-' I

-2. CURRENT PRACTICES 2.1 Utility Practices to Maintain DGs

! / Current NPP utilities have in place a number of progr'ans whose purpose is to maintain their diesel generators at a high level of reliability. These programs are. usually part of the utilities' normal activities and include pro-grams such as:

corrective maintenance, preventive maintenance, problem iden- -

i tification and correction programs, inspection, testing, trending, and train-ing programs. Not all plants utilize all of these programs. Also, the extent and quality of the programs vary considerably between plants. Specifically for their DGs, some plants have very extensive programs for TDI diesels manda-ted by NRC and others have routine programs that are one small portion of the

utility's overall maintenance scheme. Some discussion of the types and vari- >

ability of the different programs follows.

Regarding maintenance, most DG manufacturers have issued special mainte-nance recommendations for use in the nuclear industry. Many issue periodic j information sheets regarding diesel operation and repair.

, However, some util-l ities do not receive the periodic information sheets and some do not have up-dated Technical / Service Manuals. The actual process of ensuring receipt of i

j all updated technical information from the prime DG vendor and pertinent sub-vendors (such as Woodward Governor) can sometimes be a time-consuming venture.

The actual maintenance on DGs is either done by expert vendor representatives, by specially trained and dedicated utility maintenance workers, by routinely trained utility maintenance workers, or by subcontractors. According to the DG vendors, some utilities have vendor representatives at the site during each refueling cutage to assist in the periodic DG overhauls, while the vendors have not heard from other utilities for years. Some utilities assign one or two people overall responsibility for DG items, and as a result,. the vendor is

)

l always working with the same person. At other utilities, every time the vendor discusses DG issues, it is with a different person.

/ Specialized DG training is acknowledged as important,' but varies consid- ~!

erably. Some utilities have no special training in DGs, while others utilize the training. hrovided by vendors and subvendors for maintenance and operating personnel. Vendors not only have the expertise, but also special training l facilities with mockups, models, etc.

Inspections and operations vary also. Some inspections are just walk-throughs for general observations, whereas others record parameters with data loggers for later entry into computers for automatic trending. During DG operation most, but not all, utilities record operating parameters and trend them to determine the overall health of the DG.

Corrective action programs can take many forms. Some follow up on essen-tially all DG problems to determine and correct the root cause of the failure, while others merely correct the individual problem. One example is a preoper-ational plant where there were over 900 repair actions accumulated on three DGs before any concerns were raised about an overall endemic problem with the DGs.

Thus, one can see that the programs for maintaining DGs vary considerably between plants. Reference O shows also shows this variation in programs and activities.

There have not been detailed studies done to correlate the quality of the programs with reliabil.ity numbers, but in certain specific cases it has proven true that plants with weak programs have EDG's with low reliability.

2.2 Technical Specifications and Surveillance Testing Surveillance testing for NPP components important to safety is specified I

in the Technical Specifications, included as part of the plant's license. For plants that were issued operating licenses more than about ten years ago,

__ = - - _ __ __ ._

f.

technical specifications (TS) were custom-written for each individual plant and were not standardized. Standardized TS are now available and most new plant's TS are based on these standards. Over the next several years the Standard TS were revised and ' updated, so that even plants with technical specifications based on the Scandard do not have identical requirements. This section gives a sample of the different types of DG requirements at current NPPs as a result of their TS.

The three important parts of the TS that will be discussed here are the Limiting Conditions for Operation (LCO), the Action Statements, and the Sur-veillance Requirements for DGs. Typical LCOs for operating plants require two independent offsite power sources and two (or more) separate, independent DGs when the plant is at power. If the LCO is not met, then , the TS allow con-tinued operatica for a specified time provided that certain Action Statements are met. The typical Action Statement for the Standard TS (all revisions) is with one offsite AC source or one DG inoperable, the licensee must demonstrate 4

operability (start and load) of the remaining DGs within one hour and then every eight hours thereafter. A total of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> is permitted in this state before shutdown is required. Each DG could be tested 9 times in 3 days.With more than one DG or offsite AC source inoperable, a quicker shutdown is required. Required Action for older plant custom TS varies considerably. A few representative examples are given. Indian Point-3 TS allow one DG to be inoperable for up to 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />, provided the remaining DGs are tested daily.

Arkansas Nuclear One-Unit 1 TS allow operation for 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, with one offsite AC source inoperable, if the DGs are tested immediately. ANO-1 TS also permit operation for a cumulative time of 7 days per month, with one DG inoperable, (if the other DGs are tested immediately and daily thereafter, and if other specified equipment is available). In Generic Letter 84-152 the NRC proposed i

. . l a new DG technical specification in order to, -among other items, reduce un-necessary DG testing. The required action for one DG inoperable per these TS is to test the remaining DGs within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. Retests are not required and the* time allowed in this state 'would be site-specific. Additionally, there would be an annual cumulative outage time limit. If a specified number of DG 4

failures were - exceeded, then additional reporting and reliability actions would be required.

The TS Surveillance Requirements also vary from one set of TS to another.

A sampling of the nominal monthly test requirements is given here. Much more detailed testing is required at 18 month intervals. The early Standard TS called for DG start tests from ambient conditions every 31 days. Run time was for one hour at 50% load. Mo're recent Standard TS require a DG start test from ambient on a sliding scale (from 3 days to 31 days) based on the number of failures in past 100 starts. Test starts must be in 10 seconds, loading in 60 seconds to the DG continuous rating (essentially 100% of design), and run time is one hour. Indian Point-3 Custom TS require a monthly start test, loading with normal loads (< 100%), and a run until operating temperatures stabilize. Arkansas Nuclear One TS require a start test monthly, loading to full-rated load (= 100%), and a run until operating temperatures stabilize.

The Generic Letter 84-15 draf t TS call for a DG start test from a pre-lubed and pre-warmed state on a two-step scale (every 7 days or 31 days) based on the number of failures in the last 20 starts. Test starts must be in 10 see-onds, loaded to the continuous rating (= 100%) in a plant-specific time, and DGs run for 60 minutes. A similar start test from ambient conditions would be done every 184 days.

A point worth noting is that plants sometimes choose to test more f re-quently than required by TS. One plant, with excellent DG reliability, tests

w. -, .c .y-w,.__. w._,y -_._.,.-__-,--,,,-....m_ -__ , . - , . , . . . . - . _ _ - . . , , _ _ _ _ - . _ _ . _ _ _ . _ . _ _ _ . _ _ _ - _ _ , _ .

. ~'. I

.. its DGs weekly, although -its TS specify a monthly test. Additionally, there are some plants with unique test requirements, such as requiring DG tests if emergency core cooling equipment is inoperable or added tests of special de-sign shared or " swing" diesels.' ~

As can be seen from the above sampling, NPP Technical Specifications and hence Surveillance Testing varies considerably between plants (even plants be-longing to the same utility). In some cases the TS requirements may actually be counter productive to reliable DG operation.

2.3 NRC Activities NRC has been interested in emergency diesel generator operation and reliability for a number of years. NRC has completed internal staff analyses and has sponsored contractor studies in numerous DG-related areas. Some exam-ples of work completed in the past' are: NUREG/CR-0660 15 in 1979, several Regulatory Guides over the years (e.g., 1.6, 1.9, 1.41, 1.75, 1. 81', and 1.108), and involvement in pertinent IEEE and ANSI Standards Committees. More recently NRC has performed an in-depth study and evaluation of* Transamerica Delaval Inc. (TDI) DG problems, has performed many studies to support the Station Blackout Accident Analyses, and has issued Generic Letter 84-15.

Currently NRC is sponsoring a study of the effects of aging on DGs, a study on how to quantitatively evaluate DG Technical Specifications, and this effort to develop a detailed DG reliability program. Some of the recent items are dis-cussed in more detail below.

Station Blackout is the complete loss of AC electrical power (with the exception of battery supplied 110V AC) to all AC circuits in the plant, re-sulting in an increased likelihood of core melt due to an inability to remove i

decay heat from the reactor. This involves the loss of both off-site AC and on-site emergency AC from the Diesel Generators. NUREG-1032 7 details and i

1 J

summarizes a nur.ber of studies associated with the Station Blackout issue. It gives current industry values of DG reliability and specifies a~ desired value of 0.95 to protect against Station Blackout. The draft Regulatory Guide 8

~

discusses an acceptable means to comply with the proposed b1ackout rule (Ref.

10CFR50.63) that would require NPP's to have the capability to survive with a station blackout should one occur for a specified period of time. The draf t regulatory guide specifically addresses three areas:

1. Maintaining highly reliable AC power systems.

2. Procedures and training to restore lost AC power (onsite and offsite).

3. Plant design to ensure that NPPs can cope with a Station Blackout for at least a fixed period of time.

This Draft Regulatory Guide states that reliable operation of on-site emergency AC power sources should be ensured by a reliability program designed to monitor and maintain reliability at, or above the specified acceptable level. This report, in the following sections, provides a generic reliability program for NPP diesel generators that could be applied on a plant specific basis to satisfy this guideline.

2 In NRC Generic Letter 84-15 , NRC requested from NPP utilities reliabil-ity information and comments on DG reliability programs. Utility responses to this letter and other industry recommendations for DGs were analyzed and sum-marized in NUREG/CR-45573 . These industry comments and recommendations are included (where applicable) in this reliability program.

-11

, 3. BASIC APPROACH TO THE EDG RELIABILITY PROGRAM This section provides the methodology employed in developing a reliability program for emergency diesel generator systems at the U.S. nuclear poder plants (NPP). *

• The methodology considered is largely based on the reliability tasks, techniques, and activities developed in the Operational Safety Reliability Program (OSRR) Project", which are further elaborated, summarized, and tailor-i ed especially for EDG's. Within this process the effectiveness of current EDG i

activities will be avamined to specify the methods that can identify problems, determine their causes, and resolve the problems.

3.1 Structure of EDG Reliability Program for EDG The overall safety objective of a reliability program for a NPP is to 1

assure that for the duration of plant life an acceptably low core-melt fre-quency is maintained. (Note: Utility objective is also to achieve high plant availability.) This can be accomplished by assuring that plant reliability is maintained, with all equipment performing within established reliability goals, and with the minimizing of equipment failures.

i A reliability program must have definable characteristics to accomplish the objective stated above. The program should be able to detect problems that may exist and be capable of predicting potential problems that may occur in the future. In general the top level tasks contained in a reliability pro-gram to achieve these objectives are:

Reliability problem prediction and recognition

+

Reliability problem prioritization and correction l

+

Reliability problem close-out l

\

9

,-_-------,..,-,,,v--e__.,,

-12 1

These three top level tasks are expanded into reliability program ele-ments. The. foregoing is illustrated in Figure 3.1. The reliability program elements can then be accomplished by the use of tools such as reliability activities and techniques which are discussed in Section 4. .

Equipment reliability problems can be detected by monitoring equipment performance and comparing this performance to the pre-established reliability targets. (See Fig. 3.1 and 3.3)

The problem predication and recognition top level task can be partitioned into prognostic (predictive) and diagnostic processes. The prognostic process focus is to evaluate elements of design and operations that may be potential reliability problems, and the diagnostic process focuses on performance moni-toring and the comparison of reliability to established targets.

The problem prioritization and correction top level task is partitioned into two subtasks (elements); problem identification, and failure / root cause analysis.

The top level task of problem closeout is partitioned into two elements:

l determine and implement corrective actions, and implement / verify the resultant effectiveness.

3.2 Primary Elements of the Reliability Program for EDG The three top level tasks defined previously are expanded into seven sub-1 i tasks (elements) that are suitable to the emergency diesel generator reliabil-

. icy program. These lower tier elements are necessary to accomplish the objectives of the reliability program needed to maintain or improve the reliability of the emergency diesel generators used at nuclear power plants.

The following sub-sections describe these activities, as well as the flow of information and' the decision making process within the reliability program framework. All licensees have some, or all of the elements of the reliability i

.- =. -. . - , . c ASSWE THAT EDG

, OBJECTIVE -

RELIABILITY AND AVAILABILITY ARE WITHIN LIMITS 1

I.

TOP L'EVEL RP TASKS (OSRR PROG 1AM REF. 4)

PROBLEM PREDICTION AfD PROBLEM PRIORITIZATION

• i RECOGNIZATION AND CORRECTION PROBLEM CLOSEOUT l

. ELEMENTS OF -

! RELIABILITY ,

j PROGtAN FOR i EDG j EDG EVALUATE COWARE EDG FAILURE / ROOT DETERMINE IWLEMENT/

j PERFORMANCE RELIABILITY RELIABILITY PROBLEM CAUSE CORRECTIVE 'IERIFY NDNITORING IN DESIGN & TO IDENTIF1 CAT 10N ANALYSIS ACT10N CORRECTIVE

& ANALYSIS OPERATION TARGETS ACTION l s.

t l

j RELIABILITY DATA FMEA/ CC/CM PERSONNAL P/C S WWEILLANCE TECHNIQUES &

ACTIVITIES

--f{-- ANALYSIS FT/ET ANALYSIS ANALYSIS TRAINING MAINTENANCE ACTIVITY OPERATIONS ISEE SECTION 4) ,

1 t

1

! FIGURE 3.1 - STRUCTURE OF RELIABILITY PROGRAM FOR EDG l

1

___ _ 9

l*. .

program inherent in their plant programs. Table 3-2 shows the seven reli-ability program elements with additional detail on the activities and techni-ques which may be required to accomplish the objective of the reliability pro-gram., These seven elements are' also discussed in more detail in the following sections.

3.2.1 Task I. Problem Prediction and Recognition -

Element 1. EDG Performance Monitoring and Analysis Performance monitoring comprises both reliability monitoring (e.g., the direct monitoring of failure frequency, down time due to the surveillance testing and maintenance activity, outage race, etc.), and condition monitoring (e.g., monitoring conditions that are related to failure, such as degraded and 4

incipient failure, temperature and pressure of engine, etc.). Information available from specific plants can be utilized to detect or predict equipment reliability problems at the plant, and can provide a large data base for eval-4 uating reliability on an industry wide basis.

In general, a condition monitoring scheme can be developed by using the physical parameters recorded with the instruments presently available (e.g.,

cooling water temperature, engine oil pressure and temperature, turbo charger pressure and temperatur,, etc.). When these performance related physical parameters are coupled wit: the statistical techniques developed for the OSRR program", such as aziti-variate statistical and regression analysis . then unusual or abnormal behavior can be detected through statistical investigation of the correlation of the measured parameters. Such analysis also may provide information for root cause analysis of identified failures. Condition moni-toring also can provide a means to recognize component degradation and deter-i mine failure mechtnisms that can cause equipment deterioration. Component de-gradation may be indicated by changes in the operating parameters that are

. _ _ -- - -- =

l .

1 Table 3.2 Tasks, Elements and Activities Necessary for an Ef f active EDG Reliability Program 1

i l Task 1 Problem Prediction and Recognition Task II. Problem Prioritization and Correction i

Element 1. Performance monitoring and analysis. (3.2.1) Element 4. Problan identification (3.2.4)

• Establish monitoring data base.

• Determine equdgment importance

• Monitor equipment performance, in regard to DG operation.

• Assess test requirements.

• Assess problaa severity.

• Perform cause analysis.

Element 2. Evaluate Rollability in Design (3.2.2)

• Identify proutes cause.

and Operation Collect plant Information. Element 5. Failure and Root Cause Analysis (3.2.5) i Collect industry Information.

i

• Assess system design.

• Assess systen design.

• Assess test adequacy.

• Evaluate available data.

• Assess systen Interactions.

• Set depth of the cause analysis

• Assess common cause prevention.

• Task lit. Probian closeout Elenant 3. Compare Reliability to Target (3.2.3)

Elanent 6 Determine Corrective Action (3.2.6)

• Specify top level safety goal.

• Define rellability targets for EDG.

• Problem analysis.

• Compare performance to targets.

• Information from problem detection

• Operate tracking / alert systen.

• Information from cause ID.

• Information f rom ladustry.

• Define problem correction options.

Choose correctiva actions.

I 1 Element 7. Implement / Verify Corrective (3.2.7)

Action i

• Schedule corrective action, j

• Implement corrective action.

1

• mnitor corrective action.

• I

• Check performance against ' objective I

• If ef fective, close out.

• j

• If not ef fective, recycle.

I i

l 1

. l

, o

~

. . measured during the DG operation or by witnessing abnormalities such as vibra-tion, noise, etc. that indicate the diesel generator is operating under car-tain abnormal conditions.

As was stated previously,'the most important part of monitoring DG per-formance is to detect say abnormal performance which may be shown either by exceeding acceptable ranges of one or more measured parameters or by comparing -

it to past performance.

The interrelationship of surveillance activities and preventive mainte-1 nance programs with the reliability program developed for the EDG, must be defined before establishing reliability targets for the DG. Figure 3.2.1 shows the activities required to monitor emergency diesel generator perfor-mances, and Table 3.2.1 provides examples of parameters that may be monitored for the EDG.When parameters measured during EDG operation are trended and analyzed properly, the resdits could indicate that a component or sub-system of the EDG is not functioning per its design requirements. Reliability is monitored to assure that the reliability target of 0.95 in met. Maintaining EDG reliability at the specified acceptable level is a major factor in re-i solving the safety concerns stemming from the station blackout issue. Condi-tion monitoring can provide information for root cause analysis, assist in verifying the effectiveness of the implemented corrective action, and flag the component degradation. Performance monitoring could be used to trigger main-tenance or surveillance actions to prevent the occurrence of a catastrophic failure. Based on the above tabulation, the reliability task of performance monitoring is one of the requirements in developing a successful reliability program for EDG.

I

- - - - - - - - - " - " ' ~ ' ~ " ~ ~ ~ ~ ' -

4 e 4 9

• i e

6 #

4 52 4W zg IQ

%g

~ 52 22 5

, 28 zw

- [2 .J W2 .  :

/ 2k WE.

t w- l Eb:

': egw

• w d Em g u -

=

gg 3

. N

- ** 5 y

< Ett g j5g- WC" <

Wm3 3i' 5 253o $

-N = -

gm8 =

Ya<tI .- -E M

w 82c 5 wwg e

g. z

-b5 9 E a w 8

- 21o W

6

!! 5

• 84* '

ge

-t "3

8- E]

em, 8 E t$m $~E W 9 gd5=' 33o

• a m8:

g=2 m

_.m e kt.

5! t E

E*5 U 2 985 *

=]"= s m

s

-e N"a -

O!M C o 9 3 ta E E .E 2 5 eft a

E*C" c j$0U o-<

.o WWW 5 m "y52 E95 E Cgd*

E ,t*

8ggm j BI

• mE5 kw M

___-,-____--,______e.. - - - - , , - , - - - - - . . - , - , - - - - - - , -

1 -*

-15 .

Table 3.2.1 List of parameters that could to be monitored during the EDG operation

1. Engine oil temperature ,

2. Oil filter" inlet and outlet pressure ' ,

/ 3. Oil strainer inlet and outlet pressure

4. Cooling water system temperature (engine)

,5 . Cooling water system temperature (heat exchanger)

.~6. Engine crank case pressure

7. Engine RPM

8. Turbo charge oil pressure -

9. Turbo charge oil temperature

10. Fuel filter inlet and outlet pressure

11. Generator voltage

12. Excitation voltage

13. Generator output current i

3.2.2 Task I. Problem Prediction and Recognition Element 2. Evaluate Reliability in Design and Operation This element consists of the evaluation of the design and operation of the diesel generator to determine if conditions exist that may result in unreliable operation or deterioration of the emergency diesel generator. The lack of prelube and prewara subsystems is an example of an identified design

, deficiency that may deteriorate the EDG operation. This element may uncover potential reliability concerns before they a

manifest themselves in j deterioration of the diesel generators. When a design change has been initiated, the overall impact of that change on diesel generator operation should be determined. To assure that built-in design reliability is maintained within acceptable levels during the life of the plant, those

degraded components and hidden failures must be detected and corrected as soon as possible. The basic requirement of a reliability program is to maintain or improve component reliability. It is important' that when system performance is analyzed, and sets of data have been developed, information be compared

against a target. To do so, the system behavior due to the design and opera-t tion must be determined. Changes in the DG performance could be due to the dperational problems (e.g., t'oo many surveillance tests degrading the DG or '

/

improper maintenance practices, lack of personnel knowledge in operating or maintaining.DG, etc...) or it any be due to the improper design changes (e.g.,

the effecta of the design modification was not determined properly). All of the above indicate that without plant specific and industry wide information, and without assessing design and operational activities, the reliability pro-cess cannot be complaced. Evaluating reliability in design and operation l includes evaluacion of:

System design, Operational activities (e.g., surveillance testing, operation, maintenance, personnel training, management control, spare parts, etc...)

System interaction (e.g., in the area of DG control circuits and generator excitation)

Common cause failure prevention.

Design problems that deviate from expected design behavior such as system interactions (e.g., in DG control circuits) and, setpoint variation (e.g.,

i j

speed switch setpoints for initiating excitation field flashing circuit) should be identified. Figure 3.2.2 shows the work activities for this ele-ment. When quantifying reliability and availability in operation, the EDG un-availability due to the surveillance testing, and corrective / preventive main-tenance activities must be included.

., , _ . , __._.--__--r-- "

l l

\

EVALUATE RELIABILITY IN DESIGN A m OPERATION '

i. s. ,

1 1

4 i

I COLLECT PLANT INFORMATION COLLECT AND MONITOR I WUSTRY EVALUATE DESIGN Ale OPERATION WIDE INFORMATION FOR RELI ABILITY ADEQUACY EVALUATE SYSTEM EVALUATE OPERATIONAL EVALUATE ADEQUACY OF EVALUATE ADEQUACY OF COMNDN DESIGN ADEQUACY ACTIVITIES SYSTEM INTERACTION CAUSE FAILijRE & PREVENTION l

l lDENTIFY DESIGN DEFICIENCY, & EVALUATE EDG OPERATIONAL ACTIVITIES 1

INCORPORATE DESIGN CHANGES 1.e., SLRVEILLANCE TESTING, P/C .

REC 04 MENDED BY VENDORS, ETC.. NAlHTENANCE, TRAINING, SPARE PARTS, ETC.

FIGURE 3.2.2 WDIE ACTIVITIES OF THE EVALUATE RELIABILITY IN DESIGN AND OPERATION ELEMENT O

Some design improvements that could be considered for implementation at each plant as part of this task element are:

t

1. Prelubrication and prewarming systems

/. 2 Recommendations of NUREG/CR-066015  !

3. Design improvements contained in INPO SOER, 83-11 "

3.2.3 Task I. Problem Prediction and Recognition Element 3. Compare Reliability to Target The work defined by this task is to compare the actual reliability of the emergency diesel generator as monitored by the reliability program to the specified performance carget.This reliability target is based on requirements -

for resolving unresolved safety issue A-44, station blackout.

One of the important factors in determining the plant specific duration of station blackout is the reliability of each of the onsite emergency AC power systems. In coping with station blackout the maximum allowable i

emergency diesel generator failure rate has been estimated at 0.05 failure per demand. (Draft Regulatory Guide for Station Blackout .NRC March, 19868 ).

Failure per demand is the number of emergency diesel generator failures in the

last 100 valid demands divided by 100 (NRC generic letter 84-15)2 This value is only based on the probability of an EDG failing to start and/or run and does not consider downtime unavailability caused by surveillance testing and maintenance activities, etc.

The result of the performance monitoring element is compared against this i

target reliability value. If a reliability problem is indicated (Figure 3.3) d then the succeeding tasks in the reliability program process are performed.

1 Otherwise the monitoring of the DG performance and those activities necessary i

for maintaining- the EDG will continue. The work activities necessary to l accomplish this element are shown in Figure 3.2.3 Details of the actual process of calculating EDG reliability, and actions to be taken based on varying results are given in Section 4.2.2.1 and Figure 4.2.2.1.

1 g 4 COMPARE RELI ABILITY TO TARGET 1,

4 I

l i

ESTABLISH RELI ABILITY TARGET COWARE CURRENT PERFORMANCE (0 RAFT RG FOR STATION BLAOCOUT TO RELIABILITY TARGET NRC MARCH 86 .

i I OPERATE PERFORMANCE IMllCATOR AND ALERT SYSTEM TO IDENTIFY DEVIATION FROM TARGET l

l FIGtRE 3.2.3 WDRK ACTIVITIES OF THE COMPARE RELIABILITY TO TARGET ELEMENT ,

l i

I

~

-13 3.2.4 . Task II. Problem Prioritization and Correction Element 4. Emergency Diese1~ Generator Problem Identification

/

In order to -devise effective corrective action, the cause of the problea must be identified. The activities by which this can be accomplished is 11-lustrated in. Figure 3.2.4. To accomplish this reliability element the follow-ing information is needed; engineering judgement, the method of detection of the failed component, component operating history, sad previousperformance monitoring data. Once the problem is identified then failure and root cause analysis can be performed to determine the real cause of the component failure.

3.2.5 Task II. Problea Prioritization and Correction Element 5. Failure and Root Cause Analysis 1

This reliability program element is found to be effective in the resolu-tion of the problems and identifying necessary corrective action. One of the principal reasons for analyzing component unavailability is to characterize significant problems, and develop actions for decreasing their chances of future occurrence. Once the problea is identified then f ailure analysis or root cause analysis can determine the depth of the problem, the cause, the consequences, and in most cases, the necessary corrective action. Root cause investigation should be performed for any component failure. The problem must be analyzed and its causes identified. Root cause analysis can also identify common mode f ailures. Failure analysis gives systematic and detailed consi-deration to the actual effects of the assumed failure. Improvement in real reliability depends on proper identification of the basic problem and its root j cause, and the proper choice of corrective action that must follow.

i f

_- - - ~. .

, = -

s

=

~,,s..

- ~  %.

,n_. -

,_f .

IEDG PROBLEM IDENTiflCATION ~' \

l ,_

4 c. ,

l

- - :: c;

,.f

-r _

4' N . .

_ s. .

rKORMATION NEEDED TO - GUIDELINES TO PERFORN PcPf0HM AAALYSIO-iOR -

IDENTIFY PR00 LENS ,

, __ CMISE ANALYSIS tCip!PJW3 THE PROBLtMF. .

,, 7 un -- '. ,

c

, . c ..

~

, , m ,..

c 3

i jD -w_. ,/] %.

}

_~ ^r a, ~"',..

s-*' '

y .

_e_ _

~

I USE OF CONDITION THAT CX3eOrC!T ~~

ENGINEERING FAILED CapFONENT (PERATING ~ - -

JUDGEMENT WAS FOUND HISTORY

~

l 4

I FIGtRE 3.7.4 WDHK ACTIVITIES OF THE EDG FROBLEN IDENTIFICATION ELEDENT l

l i

I e

t l

i l

1 1

l

o '.

w In general, causes can be divided into two ma.1or categories; component causes and root causes. When the malfunctioning of one component acts as the cause of another component failure, this is catagorized as cotaponent cause failure. A root cause is an underlying or initiating reason and is ' anything -

other than a component cause. Root cause of a failure for diesel generator

~

components can usually be identified within the following areas: ,

Design, manufacturing / construction inadequacy Operating procedure inadequacy Human actions (operation and maintenance)

Maintenance activities (scheduled, forced)

Environmental stress '

Internal to component 3.2.6 Task III. Problea Closeout

)

Element 6. Determine Corrective Action Information concerning the nature of the reliability problems from pre-vious tasks, and industry wide sources should be used to define the best pos- ..

sible options for the necessary corrective action. The chosen corrective action must be based on its perceived performance versus the objectives.

Figure 3.2.6 shows the work activities of this task. This task is usually performed by utilities that do not have an active reliability program for the EDC. The effectiveness of all efforts to improve reliability depends on the identification of the true cause of the problem and in determining the cor-rective action necessary to resolve the problem. Such corrective action could be in the form of a design modification, (such as the addition of prelube and prewarm subsystem), change in operating procedures, requiringadditional per-l sonnel training, or storage of additional spara parts. As was stated before

=

! . OETEMINE CORRECTIVE ACTION i, INFORMATION TO ASSIST CORRECTIVE DECISION PROCESS FOR DETEflMINING ACTION DETERMINATION CORRECTIVE ACTION 4

INFORMATION FROM INFORMATION FROM INFORMATION FROM DEFINE OPTIONS RAMC OPTIONS CHOSE CORRECTIVE PROBLEM ANALYSIS / FAILW E/ ROOT CAUSE INDUSTRY WIDE FOR CORRECTING ACCORDING TO ACTION FOR DETECTION ANALYSIS SOW CES PROBLEMS OBJECTIVES INPLEMENTATION FIGLHE 3.2.6 WORK ACTIVITIES OF THE CETERMINE CORRECTIVE ACTION ELEENT 4

e

. lube and prewarm subsystem), a change in operating procedures, requiring ad-dicional personnel training, or storage of additional spare parts. As was stated before, improvement in reliability and performance is dependent on how

~

the basic problem and cause of' the problem are identified, and which correc-tive action is chosen. An additional function of the corrective action ele-ment is to provide data to support industry wide data bases. --

3.2.7 Task III: Problea Closeout Element 7. Implement / Verify Corrective Action This element will implement the corrective action previously determined, and verify the effectiveness of that action in resolving the reliability pro-blem(s). Figure 3.2.7 shows the work activities by which this element is com-plated. This reliability element must show that indeed the corrective action chosen was implemented and its effectiveness is verified. It may be necessary that the action be monitored for a period of time. By the end of the trail time the problem is either closed out, or there may be a need for reinitiating the previous elements to determine a new and more effective corrective action.

When the problem is eliminated, then this task can provide the documentation i

necessary for problem closeout.

3.3 EDG Reliability Program Process Figure 3.3 illustrates the logical flow of the overall reliability pro-cess. As long as reliability of the EDG is maintained above or equal to the '

reliability target (95% as specified by the Draf t Regulatory Guide Station Blackout)8 , the plant would maintain DG reliability by continuing pre-estab-lished operational activities and monitoring DG performance and comparing the 4

measured reliability with, the target value. This process is described in Figure 3.3 as the " maintain reliability" loop. If measured reliability is

IWLEMENT/ VERIFY CORRECTIVE ACTION 501EDULE CORRECTIVE IMPLEMENT CORRECTIVE IF NOT EFFECTIVE If NOT E7FECTIVE ACTION ACTION CLOSEOUT RECYCLE 1 . ,

MONITOR CORRECTIVE CHEOC PERFORMANCE ACTION AGAINST OBJECTIVE FICtHE 3.2.7 WORK ACTIVITIES OF THE IWLEMENT/ VERIFY CORRECTIVE ACTION ELEENT 4

e

(

n _ _ - - . _ _

lower than the target value, or is exhibiting a downward trend compared to the reliability value measured previously, then the full reliability process as illustrated in Figure 3.3, would coes into play. However, even if overall EDd reliability remains above 'the target value of 95% ro'ot cause analysis should be performed for any failure.

O e

\

O WHITOR EDG PERFORMANCE RELIABILITY EVALIMTE RELIABILITY IN COWARE RELIABILITY (RELIABILITY & CONDITION 5 DESIGN & OPERATION r TO TARGET MONITORING) DATA .

s s + 3 8 I NO PROBLEM CORRECTIVE I  !

ACTION IS ' I IMPLEMENTED  ! MAINTAIN RELIABILITY

& VEHIFIED k PROBLEM

- - - - ~ ~ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ~ ~ - -

1

(

IMPLEMENT AND VERIFY PROBLEM CORRECTIVE ACTION IDENTIFICATION

' l IWROVE RELIABILITY CORRECTIVE ACTION FOUND If DETERMlNE CORRECTIVE PROBLEM CAUSE IDENTIFIED FAILlRE AND ROOT CAUSE ACTION 4 ANALYSIS 4

FIGURE 3.3 EDG RELIABILITY PROCESS

, , 4 RELIABILITY TECHNIQUES AND ACTIVITIES FOR IMPLEMENTING RP ELEMENTS This section provides discussion of reliability techniques which are used to perform reliability activities which, collectively, achieve the completion of

• previously identified elemen'es and tasks of the reliability program for the EDG system. In the full application of the EDG reliability program process, the adequacy of all activities is analyzed. Table 4.1 summarizes reliability activities and techniques that can be utilized to accomplish the reliability -

tasks and elements identified in Section 3.

4.1 Reliability Techniques Reliability techniques are basic reliability tools that are used to per-form reliability activities. Reliability techniques are used for the imple-mentation of the reliability tasks and elements, and can help in determining whether or not all the important failure modes of a component are addressed and can be detected during operational activities such as surveillance testing and maintenance. .

4.1.1 Failure Mode and Effects Analysis / Reliability Block Diagrams Failure mode and effects analysis (FMEA) is a reliability technique which can assist in evaluating system /equipmentdesign from a reliability aspect, as well as determining the effect of all f ailure modes on system operation.

The main steps to perform an FMEA are:

1. Modeling the system (block diagram, fault tree, event tree).

2. Listing all failure modes of the component. j

3. Assigning failure rates to each component failure mode.

l I

4. Listing each component failure mode effect or effects on the system oper-ation.

~

l \

l l

l 1

RELIABILITY PERFORMANCE EVALUATE RELIABILITY COFARE RELIABILITY PROBLEM FAILtRE/ ROOT CORRECTIVE ACTION CORRECTIVE ACTION PROGRAN ELEMENTS MONITORING IN DESIGN & OPERATION TO TARGET IDENTIFICATION CAUSE ANALYSIS DETERMINATION IW LEMENTATION &

ANALYSIS VERIFICATION RELIABILITY TECHNIQUES PRA Based Ibdeling a e a e e FMECA e a e e e e CUSUM Tracking e a e e Data Analysis e a e e e e a Conditional >bnitoring e a e e a e Rallability Ibnitoring a e a e CC/CM Ana ly sis

• e a RELIABILITY ACTIVITIES Data Base Development e e a e e e e Optrational Activities e e e e a e a Previntive Maintenance a e a e a e e l

Corrective Maintenance a e e a e a e Personnel Training a e a e Quality Assurance e a e ,

Design Evaluation a e a e e Table 4.1 Reliability Technique and Activities to Accomplish Figure 3.1 Objective

l l

?, . .

.. FMEAs are usually developed based on the system success block diagram.

Component failures are identified under the assumption that all component failures are statistically independent, component failures are single random failures, and component capabil'ity to function has been deficient. When per- ~

forming a failure modes and effect analysis (FMEA), the following information should be considered for each component failure analysis. -

Failure modes:

Type of failure mode Test interval for identifying failure mode Test mode during operational activities Periodic operational test identification Failure rate (per million-hours of operation)

. Restoration time (in hours)

Component outage time (in hours)

Maintenance activities of the component Description of failures:

Failure effect on the EDG Method of failure detection Allowed outage time Analysis of failure effect on EDG:

+

Loss of one EDG (one train)

Loss of two EDG (both train)

+

Reduction of the capacity of the DG output

+

On-line system maintainability

+

Requires EDG shutdown for maintenance Deferment of repair to refueling outage period

_24 Figure 4.1.1 shows a sample blank FMEA sheet developed specially for an EDG.

The reliability success block diagram is usually constructed based on the sys-tem success logic, with the desired result, of how power is generated due to a loss of off-site power, Each sutr system of the EDG can be 'modeled separately and then the combination of' the subsystem models would show the EDG overall block diagram for both modes of success, start and run. Developing a block diagram is essential in the sense that it can identify all of those components necessary for successful operation of the EDG. The block diagram can simplify the development of fault tree models and FNEA preparation. Fur-ther details on how to perform FMEA can be found in Ref. 5.

4.1.2 PRA Based Modeling (Fault Trees)

Fault tree models are used to develop PRA models for the EDG system. The top event of the fault tree should be defined as the minimum emergency AC power required to maintain the NPP in a safe condition.

Separate emergency diesel generator fault trees should be developed for failure to start and failure to run conditions. Presently EDG fault trees are best analyzed if a computer code (such as SETS) is used. The quantified cut-sets (or failure modes), obtained from the output of this code can be used to .

1 identify the causes for unavailability at the EDG functional level as well as train level. These quantified cutsets can facilitate the selection of the 3 critical component lists which identify the significant contributors to EDG l 5

unavailability . The development of the fault tree model is based both on the reliability block diagram and insights gained from an FMEA performed pre-viously. Subsystem fault trees should be developed on the basis of the de-fined subsystem boundaries and their interactions with other subsystems, specially when modeling EDG control circuits.

Figura 4.1.1 FAILURE MODES & EFFECTS ANALYSl3 SHEET BY: DATE: CHECKED: DATE:

COWONENT:

SYSTEM / SUBSYSTEM:

REF. CW. & OATE OF ISSUE:

, I RE,OUIREO COMONENT FUNCTION: . .

. -1

/ __

FAILURES MAINTE- , COW FAILURE M00E PER P.O.T. TEST NANCE OUTAGE MODE 18 HR$ REF NO.1 INTER ACTIVITN HRS / YRS TESTED 2.

3.

4 DESCRIPTION OF: a) Failure Ef fects, b) Method of Detection, c) Allowed Outage Time

1. 2. 3. 4 STEP FM ACCESS PREPARATION REPAIR INSPECT TOTAL FAILURE EFFECT (UNIT): Identify Failure Mode Loss of Loss of Reduced EDG (one EDG (both Output On-Line Deferrable Train Trains) Capacity Maintenance Maintenance NOTES:

1 P.O.T.: Periodic Operating Test Number 2 Maintenance activity Indicates the time Intervals for preventive maintenance.

G av.ww m ,

o ,' .

. PRA models can be used to identify problem areas in design, operation,.

test adequacy, system interaction, and common cause or common mode prevention.

They also can be used in identifying reliability targets, assessing monitoring and test requirements, and identifying the equipment importances and critical

~

component lists.

~ _

4.T.f P'e' formance r Indicators (CUSUM Technique")

As part of the Operational Safety Reliability Research (OSRR)". programs, Brookhaven National Laboratory (BNL) has explored how risk-based performance indicators and alert levels (CUSUM technique) might be used as part of an integrated reliability program to help maintain an acceptable level of reactor safety through a plant's operational lifetime. BNL considered several approaches to risk-based performance indicators and explored in NUREG/CR-46184 an approach that uses reliability-risk technology to evaluate the risk relationship of performance indicators and related alert levels. This work currently is ongoing at BNL. This technique is a promising one, for monitoring the operation of EDG to flag problems before a reliability target is violated. CUSUM techniques are further described in detail in Appendix A i

of this report.

4.1.4 Data Analysis Data analysis is used for assessing; test adequacy, monitoring require-ments, monitoring equipment, establishing reliability targets, specifying per-formance indicators, operating and tracking alert system, cause analysis, and common cause analysis. Sometime, specifically in the case of EDG, it may be necessary that industry wide data and single plant data be augmented to per-form the activities listed above. Data concerning operating time, for an EDG at a plant are not always available. Other than component failures, another j factor important in the EDG reliability program is the percentage of time that t

' 'o ,

a diesel generator is out of service due to the maintenance or surveillance activities.

Another objective of the data analysis is to estimate reliability para-met'ers needed to quantify the fault tree events, and to learn which are the

~ '

important factors that affect the reliability of the DG. Plant specific data needed for analysis can be tabulated from maintenance work requests. Failure data can be ranked according to its severity into three different catagories; catastrophic, degraded, and 3 incipient failures. Failure can also be classified by the stress cause into demand, standby, and design stress.

4.1.5 Condition Monitoring (Trending)

The functional capability of an EDG is demonstrated by the demand tests required by the Plant's Technical Specifications. The frequency of these tests are dependent on EDG reliability and other regulatory requirements that are imposed on the EDG. During the performance of surveillance testing, sets of operating parameters are monitored. These parameters can indicate the condition or state in which EDG subsystems or component are operating. Such parameters when investigated and compared to the parameters recorded previously, should flag the occurrence of an abnormality or provide evidence of the cause of this abnormality. This reliability technique is currently in use by a number of plants under the name of operational trendir g, and is supported by DG vendors and various industry groups, such as ASME and INPO.

Some guidance on trending of EDG parameters is provided in the ASME Draft Standard OM-16 13 and in INPO SOER 83-1 14 .

There are a few other condition monitoring schemes that may be used to l analyze parameters recorded. One promising method is the multivariate statis-tical technique described in Section B.2.2 of NUREG/CR-4618".

o ,'. .

Condition monitoring is a reliability technique that provides a means to identify component degradation and to help in determining failure mechanisms that have caused component deterioration. Condition monitoring techniques can be'used in triggering preventive maintenance actions to prevent further degra-dacion of components. Monitored parameters can also be useful for root cause analysis.

4.1.6 Reliability Monitoring Reliability monitoring is a reliability technique that is performed to determine the condition of EDG reliability in reference to its reliability target. Therefore, the reliability target determination is a necessary acti-vity for this subtask. Testing can be accomplished to meet specific reliabil-icy targets and by the same token testing can be used to determine if the reliability monitored is within such targets.

Supporting data base development activity is necessary to perform the performance monitoring subtasks described previously. Techniques such as CUSUM tracking measures, condition monitoring, or any other reliability non-itoring technique along with some data analysis can be utilized to track the performance of the EDG.

4.1.7 Common Cause/ Common Mode Analysis Common cause failures are real and they exhibit a perplexing range of characterist-ics. As a result, these events can have significant and far reaching implications on safe operation of the EDG systems. Emergency diesel generator failures due to common cause could be classified into five main groups:

1. Common mode failures due to engineering deficiencies (design and construction).

l',

2. Common mode failures due to the environmental effect (external as well as internal).

3. Com=on mode failures due to ' the procedural deficiencies or per-sonal errors (operation, maintenance, training, ... etc.).

4. Common mode failures due to aging.

5. Common mode failures due to support system failures.

The information provided through a data base development activity, fault tree analysis, and FMEA techniques can be utilized to analyze for potential common cause failures.

In general, EDG auxiliary subsystems such as; service water system (for EDG cooling system), DC power supply (for EDG startup and control), air start system, DG air intake supply, EDG governor system, fuel storage and handling are susceptible to the common cause failure. These auxiliary systems have

~

been found to have caused about 75% of EDG failures. Section 9.6 of NUREG/CR-

, -29 89 16 lists the events that caused or had potential to cause a CCF. In Section 5 of the same report, 32 hardware related common cause events and 88 human error-related events are identified that cause (or had the potential to cause) the simultaneous unavailability of two or more EDG's.

4.2 Reliability Activities

) Reliability activities are those activities that make use of reliability j techniques. Table 4.1 has summarized such activities for an EDG reliability i

program process.

4.2.1 Data Base Development Activity Data base development is an essential activity for implementing a scheme for collecting and monitoring industry-wide information available for EDG.

Information stored by the data base development activity may be used through data analysis to determine potential for common cause failures. Depending on l

-n-a technique used to perform monitoring equipment subcasks, a supportive data -

base development activity is - necessary. Data collected by the data base development could be used for quantification of the fault trees and

~

prioritization of the equipment'important for operation of the EDG.

4.2.2 Operational Activities The basic requirement of a reliability program is to maintain or improve equipment reliability. Such programs should provide a means for detecting un-satisfactory equipment performance and forecasting potential reliability re-laced problems. If a problem associated with equipment, performance is detect-ed, then the program should provide methods and techniques for correcting the problems and observing that the corrective action has been implemented so that the re-occurrence of the problem may be prevented. Previous sections of this report described techniques which can be utilized for monitoring, maintaining or improving the performance of EDG. In this section the important opera-tional activities normally used by a plant and associated with monitoring and maintaining the reliability of the emergency diesel generator systems are de-2 fined and described. The major activities associated with monitoring and maintaining the reliability of the EDG are:

Surveillance testing (monthly, annual, etc.)

+

Walkthrough inspection (performed every shif t, daily or weekly by

means of visual inspection).

Preventive maintenance.

Corrective maintenance.

Personnel training.

i

• Spare parts.

Operations

_ _ . . _ _ _ _ _ _ . _ _ _ _ _ _ _ _ - - ---y-, ,,r -- - _ . - - - _ _ _ _. - ,y _ - - , - . -,- - ..,,.r % _, - . - . . . - . _ , - . - - - - , ~ . .,

.g .

4.2.2.1 Surveillance Testing Surveillance test activities, as described in Section 2 of this report, are an important part of the technical specification requirement which may vary from one plant to another'. Present activities of su~rveillance testing are described in Section 2. In this section surveillance activities are analyzed to determine:

1. Fay surveillance activity is needed.

2. How often the EDG should be tested to maintain the EDG reliability and availability.

3. How the EDG can be tested, monitored, and the results analyzed.

Surveillance testing is one of the major activities associated with main-taining the EDG reliability. A systematic way of possibly improving this activity is by using a prioritized list of critical components, identified through FMEA, fault tree analysis, and data analysis. These allow one to focus attention on potential problem areas when performing surveillance test-ing. The effectiveness and practicality of a reliability program is assessed by evaluating the adequacy of the various activities that could minimize the unavailability at the component level. This is done by the prompt detection of component failure and degradation, so that proper preventive or corrective maintenance can be implemented. A Markovian approach for evaluating the re-duction in unavailability for a standby component, when repair is initiated at the degradation stage rather than after a catastrophic failure, has been developed and described in Section F of Reference 4 and reprinted in Appendix B. The above three areas are addressed as follows:

1. Why surveillance activity is needed?

i l

To detect component failure or degradation prior to a catastrophic l

failure.

i

To monitor EDG reliability / availability and to assure operability and capability of the EDG to start, and load to its maximum design requirement.

2.*

' ~

How of ten the EDG should it be tested to maintain reliability and avail-ability?

The scheduling and frequency of surveillance testing and loading which currently varies from one plant to another is specified by the NRC through technical specification requirements. During normal plant surveillance test-ing, each DG is tested according to its technical specification requirements.

These test requirements increase at some plants when failures occur. This is designed to require the utility to make corrections to improve performance and to obtain more reliability data at an accelerated rate. However, such an in-crease in test frequency may also increase wearout and stress demand related failures of the diesel generators.

Generic Letter 84-152has proposed changes in the test frequency, mode of testing, and the method of calculating reliability based on the successful sta'rt and continued run of the EDG. Figure 4.2.2.1 describes a new process logic diagram for surveillance testing.that has been developed with consider-ation given to the reliability program, the Generic Letter 84-15, and utility comments. This process includes the following features:

t a) Normal testing is maintained once per month and there is no need to test the DG every week if two or more different failures are encountered, provided that through a reliability process the cause of the problem has been determined and, corrective actions have been identified, imple-mented, and verified. Same component f aiaure should not be encountered with the next 20 valid demands (box 1 through 9).

1

16 * ~

A RELIABILITY TARGET (0.95) l o

'r PLANT NORMAL SURVEILLANCE 10 '

TESTING (ONCE/ MONTH) 1-TESTING EVERY 7 OAYS FOR 20 =--

VALIO STARTS WITH FAILURE 11 .

2 g

MS 11- ',

N EDG FAILtRE OCQJRRENCE h 2-ORIGINAL PROBLEM, AND PROBLEM CAUSE HAS BEEN 10ENTIFIED 3 '

r 6 NO 12 4 PROBLEM IDENTIFIED --=

--e FAILURE INCREASED TO > 2 IN YES LAST 20 VALID DEMA'RDS

'4

NO YES ROOT CAUSE OF FAILURE YES NO HAS BEEN DETERMINED? --*

YES GO Tar A I

N0' CORRECTIVE ACTION HAS FAlltRE INCREASED BEEN DETERMINED? TO > 3 IN LAST 20 NO ANd~> 6 IN LAST -

10d7AL10 TEST 6 YES IMPLEMENT & YERIFY I4 CORRECTIVE ACTION -

.- lNFORM NRC 0F YOLR CORRECTIVE 7 ACTION NO CORRECTlvE ACTION WAS SUFFICIENT 7 YES U 8

'I FAILURE INCREASED TO

> 5 IN LAST 20 AND >ll MONITOR CORRECTED ~ IN LAST 100 VAll 7 COMPCNENT OPERATION TEST 1EZ 9 17 17 I

ECG DECLARED NO ' PRCELEM WITH THE SAME I NOPERAEL E. R EQUAL IFY.

COMPONENT IN LAST 20 DEMANDS g YES Figure 4.2.2.1 Process to Determine Changes in Surveillance Testing Requirement Due to the increase in the EDG Failures.

I

'e

, b) If EDG experiences only one failure in the last 20 valid demands, the problem must be -identified and cause of the problem determined,

/

corrective action implemented and verified. Whe'n such process is completed, the plant normal surveillance testing program (once per month) will be. 'followed. If the cause of the probles could not be determined and within the next 20 valid demands another failure has occurred, 'then the proposed requirements of the Generic Letter 84-15 (accelerated testing every 7 days for 20 valid tests to achieve reliability of > 0.95) would be instigated. (Box 1 through 3, 10 through 12).

c) If the EDG experiences two or more failures, and problem causes could not be identified immediately. Sometime in the future, the cause may be identified through data analysis, vendor communications, LERs, or any other type of activities. Then through reliability process described previously, the corrective action can be implemented. In such a case, the frequency of the surveillance testing would be changed from once per week back to the plant normal surveillance testing schedule (box 11 and 3-9).

d) If the EDG has experienced 3 or more failures in the last 20, or 6 in the last 100 valid demands (box 13-15). NRC should be informed of the cor-rective actions planned to improve the EDG reliability and to achieve at least 0.95 reliability values.

In this process the effectiveness and practicality of an implemented reliabil-ity program is assessed by evaluating the adequacy of the surveillance testing that could mininize unnecessary testing of the EDGs. Engineering experiences have indicated that excessive number of surveillance testing, long periods of no load and light load operation, could increase degradation of certain com-Ponents and reduce the operating life of most mechanical equipment of EDG.

., *e *

)

3. How the EDG can be tested, monitored, and results analyzed.

An EDG surveillance test is performed so that its capability to function

~

durinj an actual demand is proven. This basic purpose mist be kept in mind when devising the surveillance test (e.g., test prerequisites should not be devisedsuch.$hattheyartificiallyimproveEDGreliability). Since probabil-ity of wearout due to the test is greater than probability of wearouts due to the actual demand (total hours of operation due to the test requirements is greater than total hours of operation due to the actual demands) a proper balance must be maintained between the concerns for full testing and wearout due to the testing. .

EDG monthly tests are typically manually started and loaded within speci-fled requirements described in plant technical specification. Monthly EDG surveillance tests should test to maximum design basis accident load require-

, ments (LOCA and LOOP) and during refueling cycle tests, loss of offsite power and LOCA should be simulated. To reduce cold start stress related problems, the system should incorporate a design feature which should include a prelu-brication and prewarming system. During any surveillance testing all the measurable parameters should be recorded to allow through trending of the DG performance. Any abnormal values or deviations from previous test should be analyzed and the cause of abnormality should be determined. All the recorded parameters should be evaluated by a system engineer or personnel familiar with design, operation, and maintenance of the EDG; and a preventive maintenance should be scheduled to investigate those components or subsystems that are not functioning per . heir design requirements.

Based on the foregoing, this surveillance test program will reduce the number of fast starts of EDG at many plants. The program is generally struc-tured so that failures do not automatically require more testing, and is not

designed just to obtain more statistics. A reliability program should demonstrate operability, obtain real reliability values, and identify thpending DG problems.

~

'N

/

4.2.2.2 Walkthrough Inspection The adequacy of various problem detection activities should be reviewed to determine the effectivene'ss of that activity in identifying the problem areas. Walk around (walkthrough) inspection which is visual inspection of the EDG, is conducted daily (in some plants on every shift). It is also conducted af ter every maintenance or surveillance test activity to verify proper system lineup. This type of activity can improve the overall system reliability /

availability if performed in a systematic way and based on a procedure that has taken into account the components importance (from component prioritiza-tion task), and the possibility of detecting any mode of failure for that com-ponent during visual inspection. During visual inspection the following com-ponents and failure modes could be inspected and corrected:

a) All resetting of protection relays (such as lockout relays).

b) Leaks within air starting systems.

c) Low pressure leaks within lube oil, jacket water, and fuel system.

d) Proper positioning of the breakers and switches.

e) Leaks in the governor lube oil system.

f) Valve positions (given that all the positions are color marked and listed in the da11.y ? beet and instruction guidelines of the inspection).

g) Proper fuel level in day tanks and proper level indicator functioning.

h) Proper DG room temperature.

If daily umpections include the above items, then the unavailability of cer-tain components is reduced since the time interval between two inspection periods is shorter than the monthly test interval for certain modes of the component failure.

"=.. e When the emergency diesel generator is in the standby mode, some component failures ,(such as air compressor and air compressor tank, pressure, e'tc) and certain off-normal conditions (such as voltage' regulator selector ~

switch, EDG start /run control selector switch, etc.) are usually annunciated in the control room or on local panels.

4.2.2.3 Preventive / Corrective Maintenance Maintenance and inspection of the EDG components vary considerably from plant to plant as described in Section 2.2. Corrective maintenance involves repair of components af ter their failure. Preventive maintenance (PM) in-volves repairs prior to catastrophic f ailure. PM may be either prescriptive (scheduled) or indicative (based on monitoring) maintenance. Maintenance and surveillance activities are considered to be effective if the component un-availability is maintained within an acceptable level. This is accomplished by ensuring component degradation is detected and repaired prior to becoming a catastrophic failure. An analytic approach (Markovian approach) that can evaluate the reduction in unavailability of a standby component when repairs are initiated at a degraded stage (compared to catastrophic failure oc-currences) is described in Reference 4 and reprinted in Appendix B of this report. Preventive maintenance is generally performed on a periodic basis to assure the operability and extend the life of the EDG.

With the prescriptive or scheduled maintenance a pre-defined list of com-ponents is inspected. Indicative maintenance uses the results of visual in-i

spections and periodic operational testing (surveillance testing) to determine ,

conditions that indicate a component may be d.egraded or not functioning pro-perly. -

- - - . ,---y- --w - - - - - - - - -e-, - - , - , - * - +-----n- --n-w

. *e

. I Plant maintenance procedures should consider the examination of the last five data sheets of the surveillance testing activity to identify . those

^

parameters

/

that may be approaching or have exceeded an upper or lower limit of

$ts operation. Typical NPP maintenance procedures usually include most of the l

mechanical components of the system but do not include electrical components that are critical to reliable operation of the EDG. This is important since generic data analysis has shown that 50% of the electrical component failures within EDG control circuits, generator excitation, voltage regulator, field flashing circuits, and protection circuits are due to the switch and relays failure. 50% of these f ailures are attributed to sticky relays, dirty and overheated contacts. Therefore, close inspection of these piece part components .for conditions such as dirty or overheated relays and contacts, corrosion or pitting, and sticking relays should provide early indication of the componeat degradation. Overall DG room airborne dirt is a major concern which may affect electrical circuits. Cabinets should be properly sealed to prevent dirt intrusion.

Preventive maintenance should be principally based on actual manufac-turers recommendation and can be either scheduled, indicative or some combina-tion of these methods. Manufacturer's of important subcomponents (such as governor systems) should be consulted also in established preventive mainte-nance programs. Additionally, ASME Standard OM-16 13 and INPO SOER 83-1 1

" pro-vide possible guidance for establishing a DG preventive maintenance program.

These two documents could be combined with manuf acturer's recommendations in establishing plant specific maintenance programs. Indicative maintenance is commonly practiced in many different industries.

__.,,,,__,-_w--.-- _ . , - , _ _ _ , - - ,- , - ,_

, 's

_37_

Observation and measurements of key engine signs and indications, by their trend or by sudden deviation from their expected normal, can clearly fadicate

/

an immediate problem or that specific preventive Saintenance is need '

ed to address a developing problem. One of the difficulties with this type of l

maintenance. Activity is that it requires personnel fully familiar with the machine and capable of performing effective condition monitoring or trend analysis. If such maintenance is combined where practical (e.g., corrective and preventive maintenance together) it can reduce the EDG downtime during the plant normal operation, and also reduce the cost of unnecessary maintenance and part replacements. )

This also reduces the chances for maintenance related human error, which increases with the increase in the maintenance performed.

9 Data analysis (from LER ) has shown that over 507. of maintenance related pro-blems are due to the human errors caused by personnel not sufficiently fami-liar with the system. Another problem associated with DG activities, involves ,

scheduling the maintenance actions. Diesel generator unavailability due to maintenance has, to some degree, contributed to onsite power system unavail-ability. Therefore, scheduling preventive maintenance actions during reactor operatica should be avoided. Scheduled preventive maintenance, in which sec-tions or entire engines are torn down and inspected during refueling over-hauls, should be scheduled based on manufacturer's recommendation. Since the EDGs actual hours of operation at NPPs are limited, this type of preventive maintenance should follow a manufacturer's schedule which is specially devel-oped for nuclear standby diesel service. In addition to proper and thorough preventive maintenance, a pinnt must ensure that corrective maintenance, when needed, is properly performed. This would include adequate planning, spare part availability, maintenance personnel training, adequate supervision and

QA, use of expert consultants where necessary,. complete post-maintenance testing, and adequate documentation. Many problems with NPP DGs are dompounded through inadequate corrective maintenance. * '

/

Overall maintenance programs can be effective when they are developed in a systematic. order and address the following items:

Maintenance organization and administration.

Maintenance procedures and documentation.

Maintenance facilities and proper equipment.

Maintenance personnel, training, and personnel awareness of the system importance.

Spare part availability.

Maintenance history and records.

Figure 4.2.2.3 (Maintenance Logic Diagram) shows the arrangement of these cri-teria.

Fuel oil quality and related maintenance activities can affect overall EDG performance and is addressed in detail in this paragraph.

Storage of fuel is important because of the possible degradation in long time storage or improper handling of the EDG fuel oil. Proper DG operation and its output power is dependent on the quality of the fuel that is used.

Most engine fuel failures are due to fuel line problems or fuel injector clogging because of sludge, particulate matter, or water in the fuel system.

A good distillate fuel should be selected, transported, and controlled and fuel mixing should be avoided. Site fuel system should include fuel treatment to diesel and fuel degradation control system specially if the long term fuel  !

storage tanks are large enough to store fuel for long periods of time. When 4

' ~

MATERIAL, MANPOWER EDG PLANT

/ ANO RELI ABI LITY SAFETY REQUIREMENTS

/

MANAGEMENT PROGRAMS. RE(X)RDS AND REPORTS

• PLANNED MAINTENANCE

• PERFORMANCE

• EMERGENCY MAINTENANCE

• PRODUCT QUALITY

• RELIABILITY IMPROVEMENT

• COMPONENT FAILURE

• COST REDUCTION

• COMPONENT NISTORY

• TRAINING & SKILLS

• COSTS DEVELOPMENT ANALYS1S AND EVALUAT10N

• REGULATORY REQUIREMENTS

• ECONCMlC Figure 4.2.2.3 Melntenance Logic Diagem i

O

.. F 39-q performing monthly operational surveillance testing the fuel filter pressure t should be monitored and recorded. With the constant power output, pressure

~

dtop/on the filter should not }tary. Any pressure drop change could indicate 7 problems and the beginning pf. fuel degradation.

Regular'.f el inspectionLand system maintenance shm id include: , _

Periodic monitoring of fuel filter pressure drop.

Periodic draining water from tank / filter sumps.

Periodic fuel sampling for quality and degradation.

Periodic replacement of all fuel filter elements based on manu-facturer recommendation, as well as plant conditions.

Through fuel degradatioy control, the storage degradation such as; water, par-ticulate, oxidation, sludge, and microbial can be controlled and minimized.

4.2.2.4 Personnel Training Each nuclear power plant should include in their training programs for non-licensed personnel, training for those persons who operate or manage the EDG. The Training Program objective should be directed toward maintaining at least a level of technical knowledge which is needed to operate and maintain DG and understard problems that may occur during the life of the equipment.

Such trafning can vary from on-the-job training to special training in a school conducted by the utility itself or by the DG and subcomponent manu-facturers. Personnel training and skill improvement should not be limited to the operators and maintenance perconnel only. System engineers, managers, and

?. those personnel who usually perform root cause and f ailure analysis, or ap-prove design changes and system modifications, should also be trained.0ften it is very difficult to develop and maintain such skills when the diesel genera-tors operate only a few hours per month. It is very important for shift or shop supervisors to receive regular training. This would serve to develop

1 I* i f

more knowledge and skills among all the personnel involved in operation and j maintenance, as well as problem analysis. Only in this way can it be expected that malfunctions and identification of root cause of the problems will .be determined and corrected in a' proper manner. ~

Based on the Institute of Nuclear Power Operations (INPO)M, auxiliary operator Class B is the position responsible for " start, stop, and loading of EDG from local panels as needed i

for maintenance or other surveillance tests and alignment for standby mode of emergency operation". This position requires in-depth training in specific systems, procedures, and components t.-ith which the candidate can develop a working knowledge and acquire those skills necessary for EDG operation.

The DG operators should have a good understanding of EDG fundamentals so that he can identify any abnormal behavior before it could affect 5DG opera-tion. Maintenance personnel require more detailed and complete training then operational personnel.

Preventive maintenance on EDG is typically performed by two groups of maintenance personnel: electricians, who deal with the generator, associated components, and engine control and instrumentation; and mechanics who deal with the engine and its associated equipment. Although the maintenance proce-dures and manuf acturer's manuals do explain the maintenance requirements, as well as their importance for riu De operation, it is still very important for personnel to receive hands qr tm .Mng regularly. It is worthy to state that, the importance of each component should be known to the person performing maintenance (corrective as well as preventive). While training personnel in maintenance and adjustment of certain parts of the DG, the importance of that '

part for DG proper operation should be stated and personnel should be aware of its failure impact on DG operation. Since Q/A personnel do not have skill and understanding of the diesel generator in general, then it is more practical

e

, k. ,

, .i

, . . 4

_41 a ,

that the basic record system for operation, as well as corrective or preven-tive maintenance, be in the hand'of a system engineer as well as operators and maintenanc.e personnel. Most diesel engine manufacturers maintain schools 1

~

N .which do provide good advice a'nd training. Such classes can be tailored to suit the needs of operation of EDG for NPP, and can be given at the NPP or the vendors facility. One of the most difficult problems which operational personnel usually encounter during the fast start and fast loading of the EDG is the fast' loading and load rejection problem which. most of the time is

. s caused by governor and excitation circuits (especially field flashing).

Operation personnel should be fully aware of the problem and be trained to cope successfully, with them. INP0 SOER 83-1 W also has several good

, recommendations in the training area.

4.2.2.5 Operations The personnel in theLNPP Operations Department are typically the ones who actually run or operate the EDGs, both for test purposes and for real emergencies. As discussed in the previous section these personnel must have thorough training to ensure reliable DG operation. Further, the operating methods and procedures must be carefully structured to ensure the best possible mode of operation, For example, plants should ensure the prelubrication and prewarm system are in operation before all DG starts to

minimize component wear. Further DG operating parameters should be recorded sad trended for all periods of DG operation. ASME Standard OM-16 13 provides I

some details on this activity.

e

= ,y-, - - -- - - - . , . - - - - - , - - . - , - . - -

1e, '.

o - I

5. INDUSTRY CONCERNS 5.1 Discussion of Concerns This chapter discusses current industry concerns about EDG operation and These concerns do not necessarily

~

reliability. represenc the totality of items contributing to DG unreliability nor is it completely certain that each of the items is in fact detrimental to DG operation. Nonetheless, the summary of industry concerns does provide a good overview of areas needing attention to improve DG reliability. These concerns were obtained from previous work 3

(NUREG/CR-4557 ), and from participation in an industry wide workshop con-ducted by Pacific Northwest Laboratory on May 28 and 29, 1986 in Seattle, Washington 11 The workshop was attended by representatives of U.S. utilities, DG manufacturers, subcomponent vendors, national laboratories, and DG expert

consultants.

5.2 Use of Reliability Program for Responding to Industries Concerns Within industry as well as NRC there have been concerns about the reli-ability of EDG's used in nuclear power plant operations. Analysis performed previously has indicated that the reduction of the EDG reliability signifi-cantly increases the probability of the station blackout. Due to such con-2 cerns the Generic Letter 84-15 , NUREG/CR 4557 ,3 and some other EDG workshops were generated. Through such working groups a list of industries concerns were developed. Utilities are mostly concerned about frequency of testing or unnecessary maintenance performed due to the technical specification require-I ments, while manufacturers of EDGs are concerned with the lack of personnel training in operation and maintenance (MEDG), the lack of proper spare parts, poor fuel management, and not enough running time of the DG's. An engineered

{

t reliability program should be able to address those concerns of utility and

., e, vendors that are based on engineering concerns. In this section of the report the relationship between those reliability elements developed previously and the engineering concerns of industry is investigated. The review of 3

NUREG/CR-4557 and workshop dsveloped by tha nuclear plant aging research program (NPAR)11 has indicated that industry concerns are mainly with:

1. Excessive testing

2. Fast starts and short run time af ter the fast start

3. Manufacturer design problems

4. Components / control system interface problems

5. Spare part availability

6. Inadequate data base, for failure analysis and root cause determination

7. Excessive or inadequate preventive maintenance

8. Inadequate training in DG operation, corrective and preventive mainte-nance.

9. Lack of experience and responsible individual onsite for EDG.

10. Poor instrumentation of engine parameters.

Table 5.2 evaluates these coacerns in reference to the reliability ele-ments identified previously, and shows those elements of reliability programs which can be utilized in resolving industry concerns.

As indicated in Table 5.2, and discussed below, reliability elemec.ts identified for EDG can resolve most, if not all, of the concerns identified within industry in regard to operation of the EDG.  !

1. Industry's concern for excessive testing is addressed and resolved through: '

l l

i l

system design, test adequacy, test requirements,

• ~

comparing reliability, ind aonitoring data base.

2. Concern for fast start and 'short running time can be assessed and evaluated by performing:

conditional monitoring.

test adequacy, and problem identification and verification.

3. The area of manufacturing design problems through:

performance monitoring, evaluation of reliability in design and operation, comparing reliability to target, a

failure and root cause analysis, problem analysis, information from industry, a

problem detection, and checking performance against objective.

4. Components / control system interfacing problems can be addressed by moni-toring data base and equipment performance:

plant information.

reliability evaluation in design and ope:ation, and failure and root cause analysis.

m -- - ,,.y- .,_.-y , , - , . , . , - , . - - - . - - ~ _ , . . - - _ . - _

5. Spara parts availability can be identified through:

reliability evaluation in design and operation,

+

monitoring industry wide data base, and performing FMEA. ~

6. Inadequate data base, the importance of data base are explained in:

+

performance monitoring,

+

evaluation of reliability in design and operation,

+

comparing reliability to target,

+

problem identifications and resolution,

+

failure and root cause analysis,

+

corrective action determination, and

+

implementation and verification of the corrective actions.

7. Excessive or inadequate preventive maintenance are addressed in:

+

operational activities as well as performance monitoring.

8. Lack of personnel training in operation, maintenance, and record keeping.

All reliability elements, identified for the EDG reliability program can be addressed and specify the importance of such actions and its impact on the .

EDG reliability / availability. Table 5 Column 9, shows the impact of the reli-ability program with personnel training.

9. Lack of experienced individual responsible for EDG.

Such individual can minimize the efforts required to perform actions needed to maintain and/or improve the EDG reliability, and can interact with operation, maintenance as well as procedures developing or revising. Reliabil-ity program can be more effective if all efforts are centered within one or-ganization.

i,, .

l

• l

10. Poor instrumentation of engine parameters.

Engine parcseters are needed whenever trend analysis, performance moni-toring and analysis, problem identification, verifying the implementation of the

• corrective actions are performed. Lack of sufficient instrumentation would prevent the analysis of the engine performance.

Reliability Elements Maior Concerns From Industry From Table 3.2 1 2 3 4 5 6 7 8 9 10

1. Performance Monitoring * * * * - * * * * *

2. Reliability Evaluation * * * * * * * * *

• in Design / Operation

3. Compare Reliability to * - * * - * * * * -

Target

4. Problem Identification - * - * - * - *

• and Resolution

5. Failure and Root Cause - - * * - * * * *

• Analysis

6. Determine Corrective - - * * - * * * *

• Action

7. Implement / Verify corrective - - * * - * - * *

• Action Table 5.2 Reliability Elements for EDG vs Major Concerns from Industry 4

• Effective in resolving industry concerns.

7--. . , _ . _ , - - - .

-,-y, - - , . - - . , , , , - , , _ - , _ _ , , _ , _ . - __ ,,

6 '

E., .

l

6. ESTIMATE OF LICENSEE RESOURCES TO IMPLEMENT EDG RELIABILITY PROGRAM i i

In previous sections the effectiveness of the reliability technology to help achieve and maintain an acceptable level of EDG reliability was assessed and evaluated. This section demonstrates that a reliability program, using existing technology and manpower available to most NPP, can be developed and integrated into the current day by day operational activities of the plant to assist personnel in controlling, analyzing, predicting, and determining - cor-rective actions that can impact EDG reliability and operation. Nuclear plants, particularly EDG's have unique operating requirements. Thus, many problems usually require a plant specific evaluation. Therefore, accurately 1

estimating the manpower and resources required to perform and continue an EDG reliability program is difficult to perform on a generic basis. In general, however, the overall cost of EDG reliability program can be divided in two parts; the cost of developing the program, and the cost of implementing and maintaining the program on a continuous basis.

6.1 Resource Requirements for Developing EDG Reliability Program A reliability program can be effective if it can be' integrated and coor-dinated with the existing efforts and activities of the NPP. The activities required by a reliability program, which may not in some instances exist or may not be performed in the plant on a routine basis, are mostly in the areas of design. evaluation, systems analysis, modeling of the system equipment, FMEA, condition and performance monitoring. Therefore, when estimating re-source requirements or in developing an EDG reliability program, some utili-ties may be required to seek help outside of their organization. Actual ex-perience has indicated that developing a full reliability program for an EDG should not require more than ten man months. Such an estimate will certainly

~

vary from plant to plant, and is dependent on EDG design and available- infor-

,mation such as operating history and maintenance records. The overall cost of developing a reliability program can be partitioned into three parts:

1.* Identifying reliability program tasks and elements, n'ecessary for EDG, requiring about 30% of total usapower.

~

2. Performing PRA type system modeling and analysis to identify the critical components of the EDG, requiring about 50% of total manpower.

3. Documentation, requiring about 20% of the total man power.

The resource estimate for developing an EDG reliability program is esti-mated as 10. man months based on a BNL trial application in the OSRR program.

That program required somewhat more time chan is estimated here due a candem EDG design, which necessitated different and more complicated modeling then is usually required. The PRA procedures guide (NUREG/CR-2300)12 estimates a re-quirement of 5 man months per system for modeling and analysis. Considering that system modeling and analysis of EDG does not require event tree develop-ment, then 4 to 5 man months estimated for system modeling and analysis is within the boundary of the overall estimate of 10 man months.

6.2 Resources Required to Implement and Maintain EDG Re. liability Program When development and analysis of the EDG reliability program are com-pleted, then the implementation phase is initiated. The additional resources required to implement and maintain a RP is plant specific and depends again on the resources that may already be available within the plant. To properly implement the program, a system engineer fully familiar with DG design, opera-tion, maintenance, and regulatory requirements is a necessity. Such a systen l

engineer should, monitor the work , performed by operators and maintenance per-sonnel and should maintain complete records of all activities surrounding the

, - - - - . . - - - - , , , , . ~ , ..- --- .. - - - . .

EDG. The system engineer should be responsible for developing, and revising  !

operating or maintenance procedures to incorporate findings from the  ;

reliability program. The system engineer is responsible for developing proper maintenance and operating recor'ds, and maintaining contact with EDG vendors.

The estimate of manpower required for implementing and maintaining RP is dependent on plant arrangement. If a plant has, within its organization, per-sonnel responsible for EDG's (e.g., at the present time some utilities do have system engineers responsible for the EDG) then the additional cost is minimal. If utilities only have reliability engineers and do not have system engineers per se, then the cost of maintaining an RP program could range from 2 to 5 days per month.

g W

+ - . ,<- -_-- - - - - - , ,,,.,-,,c- --r-. --.---,--w- - - - - - - - - - - --, ,,cm...,,_._,y,__,-,-_.__.,- ,- .- - .-, -.-

T

7. REFERENCES -

1. NUREG/CR-3226, " Station Blackout Accident Analysis," (Part of NRC Task Action, Plan A-44) Sandia National Laboratory, May 1983.

2. NRC Generic Letter 84-15, " Proposed Staff Actions to Improve and Maintain

, Diesel Generator Ra11abili,ty," NRC,1984. .

3. NUREG/CR-4557, "A Review of Issues Related to Improving Nuclear Power Plant Diesel Generator Reliability," J. Higgins, C.J. Czajkowski, A.G.

Tingle, Brookhaven National Laboratory, Aril 1986.

4. NUREG/CR-4618 " Evaluation of Reliability Technology Applicable to LWR Operational Safety," A. Azara , E.V. Lofgren, et al, Brookhaven National Laboratory, Draf t Report, May 1986.

5. " Trial Application of Reliability Technology to Emergency Diesel Generator at Trojan Nuclear Power Plant," Report Paper, BNL, April 1986.

6. NUREG/CR-4440, "A Review of Emergency Diesel Generator Performance at Nuclear Power Plant," Higgins, et al., BNL, November 1985.

7. NUREG/CR-1032, " Evaluation of Station Blackout Accidents at Nuclear Power Plants ," P. Baranowsky, NRC, May 19 85.

8. Draft Regulatory Guide for Comment " Station Blackout," NRC, March 1986.

9. Licensing Events Reports, NRC.

10. INPO Document # GPG-04,1981, " Guideline for Qualification Programs."

11. NPPAR, " Nuclear Power Plant Aging Research Program for EDG at Pacific Northwest Laboratory," Draft Paper for EDG, Based on a PNL Industry Workshop, May 1986.

12. NUREG/CR-2300, "PRA Procedures Guide," Draf t Report September 1981.

13. ASME Draf t Standard OM-16 "In Service Testing and Maintenance of DG in Nuclear Power Stations," ASME, November 1985.

14. INPO Significant Operating Event Raport (SOER) on EDG Failures, January 1983.

15. NUREG/CR-0660, " Enhancement of On-Site EDG Reliability," University of s

Dayton, February 1979.

i l 16. NUREG/CR-2989, " Reliability of Emergency AC Power Systems at NPPO, " R.

E. Battle, ORNL.

t e

.i A-1 APPENDLT A CUSUM Technique The mathematical and statistical basis of the CUSUM technique (indica-tors *) are discussed in the li'terature. The present discu'ssion concentrates -

on the application of CUSUM techniques to monitor synthesizad indicators.

However, the CUSUM indicato'es or techniques can be applied at any level. It also can be applied at the level of basic indicators to obtain direct statis-tical measures of the frequency of initiating events or component failures.

The CUSUM, by its statistical basis, is the optimal technique for a given type of data. Here, the optimal technique means the statistical technique that for a given type of data gives the greatest power in detecting anomalies and trends while controlling the false alarm rate. The remainder of this section describes how risk perspectives can be used to determine the CUSUM parameters for evaluation of synthesized indicators.

• CUSUM techniques are usually call.ed CUSUM indicators and the words techniques and indicators can be used interchangeably.

- _ . - - . _ _ . , -__ -_---. y,. __ -

.. a.

A-2 There are two general types of CUSUM indicators, the counted data CUSUM and the time-between-events CUSUM. The counted data CUSUM is useful when it is convenient to record the number of counts in a given sampling interval, while the time-between-events CUSUM is useful if it is convenient to update the CUSUM with each new count and it is possible to record the time since the last count.

The measures of performance for synthesized indicators are their calcu-lated unavailabilities or expected down times within a given period of time.

These unavailabilities or down times can be looked upon as equivalent fre-quency measures with certain probability distribution functions. The counted CUSUM technique can be applied to the synthesized indicators. However, the time-between-events CUSUM technique can be applied only if we are interested in alerting on the basis of the time interval between two consecutive times that the performance of a synthesized indicator exceeds a specified level (i.e., the specified level may be considered to be the average performance predicted by a PRA). Here we will be concentrating on the single-sided counted CUSUM due to simplicity of application. The double-sided CUSUM which accounts for both improvement and degradation of performance and the time-between-events CUSUM will be investigated in the future.

- , , - - . . ,,-g , . - . _ - . - - -

A-3 Determination of CUSUM Parameters There are four independent parameters that need to be determined for 'a counted Poisson CUSUM process. These are:

L = the sampling time int'erval for updating synthesized indicators, fa = the acceptable rate of performance measures for synthesized indi-cators, fd = the ' alert level rate of performance measures for synthesized indi-cators, SA = the decision parameter for alert violation.

In addition to these four independent parameters , . there are four dependent parameters, namely:

a, = f,

• L = acceptable number of counts in a sampling time ad"fd

• L = number of counts to be detected in a sampling time m = (ad ~ "a)/(In(a }d ~ 1"("a)) = calibrated number of counts S+ = S A

/2 = fast initial responses parameter (FIR)

,m-, _ , , - . . . , , ~ . . - _ , _ _ . . . . _ , . . . - -_ _ - , . . . - , . . - . -,-,_s_,-- . _ , _ - _ . , , _ . . _ . - . _ , _ + 3 - _ - _ _ _ . _ - _ _ _ - _ . . , _ _ -

A-4 where:

"in" denotes the natural logarithm.

One possible method of selecting these independent variables, which has been investigated as a part of this OSRR program, is discussed here. A step by step procedure is provided in the following.

1. fa can be initially selected on the basis of the predicted perfor-mance of a synthesized indicator as predicted by PRA. As more exper-ience data is obtained, reference to this value can be made through methods such as V-mask.

2. fd is selected on the basis of the alert level determined for syn-thesized indicators.

3. The values for L and SA can be derived from risk perspectives. Two potential criteria are needed, namely:

a. If the performance of an indicator is always acceptable (the pro-cess is running with the count rate equal to fa), the CUSUM should not trigger during the life of the plant (i.e., the Average Run Length [ARL] of the CUSUM should be in excess of plant life or 40 years).

• . 4 A-5

b. If the performance of an indicator is equal to its alert level, the risk induced should be controlled at 10% of the safety goal (i.e., the ARL of the CUSUM in this case should be a maximum of ~

four years since the safety goal is allowed for 40 years of plant ife).

For each value of a and L, the AHLs for acceptable and detectable perfor-mance for no FIR. feature (S+ = S = 0) can be obtained from Table A.I.

The smallesc value of SA for a given value of a that satisfies the above criteri'a (namely a ratio of acceptable to detectable ARLs of 10, resulting from division of 40 years by 4 years) can be selected.

Table A.2 illustrates an example of the process for a case in which f d a is 3.5, where f, and f d are assumed to be obtained from steps 1 and 2 above.

The following equations were used for this example.

m = Lf, (f d !f a ~ l)/(1" f d a m,/m = L f,/m = Ind(f !'a)! (Id !'a) - = 0.5 4

"d! " " b Id !" " (Id!Ia )/(l"(f a/Id )I " I" I

1

A-6 These values of ma /m and ab/m are used in Table A.1 in the following manner. For each set of values of m, the two columns for ma/m = 0.5 and dd/mp= 1.7 are scanned until 'the first time the corresponding ARL values are in the ratio of 10:1 or greater. The resulting values of a and SA corres-ponding to these rows are shown in the first two columns of Table A.2. Then knowing that the ARL for detection of the alert level, in the column ad/a =

1.7 corresponding to the selected rows, is a multiple of L, and using the criteria that the ARL for detection is four years, the value for the sampling time interval, L, can be calculated from:

L = 4 x 12/ARL(f d (in a n s) i If there were no additional constraints, all the pairs of a and S A de-I termined in this manner and consequently their associated sampling interval L  ;

would have been acceptable. To reduce the statistical variations on observed or calculated performance measures, one would prefer to select the values of a and SA that give the largest value of L or smallest value of ARL(fd ).

l However, an additional constraint derives from the risk implication of the actual performance measure, which may exceed the alert level. In cases when the actual performance of an indicator is significantly in excess of the 2

alert level, one should assure that the induced risk (f ARL) is controlled in the same manner as when the actual performance measure is close to the alert level, namely, we would like to haves h*ARL(f)=f d* (I d)

A-7

, Table A.1 Poisson CUSUM Average Run Lengths (Increasi.q Rate Case, -

j No FIR Feature) me. .=mee..e.

=

• S a *a : .s 2 .2 .4 .s

. .s .s t.o 1.2 t.4 s.1 2.0 2.s 2.0 s.o 1.0 .25 .0 $t 8. 14s. 71.5 44.3 31.0 23.4 15.4 11.3 4 94 7.24 5.88 4.91 za9 3.28 2.11 2.0 .25 .0 1tL800. 1.440. 440. 21 2. 120. 78.7 40.0 25.4 18.1 13.3 10.3 8.14 8.00 4 S2 2.35 10 25 .0 . 17.300. 3.390. 1.070. 464 234. s1.0 48.5 31.1 22.4 t $.8 11.3 8.si 6.75 184 5.0 .25 .0 . . . 27.800. 5.Sec. 1.790. 332. 120. 63.5 41.5 28.3 19.s 18.4 102 5.78 7.0 .25 .0 . . . . 74.00s. 12.20s. Sea. 223. SS.s 61.3 38.4 27.3 22.4 14.4 7.s0 10.0 .25 .0 . . . . . . 1950. 438. 158. 31.2 55.5 39.3 32.4 20.4 10.4 1.0 .50 0 297. 82.1 40.1 24.7 17.2 12.3 8.48 8.22 4.91 407 127 2.75 2.45 1JO 1J4 2.0 .50 .0 8.900. 888. 276. 124 68.4 414 22.2 119 9.84 7.54 5.57 4.43 3.35 2.73 1.72 3.0 50 0 e 11.600. 2.110. 641, 284 131 49.8 26.1 18.6 11.9 813 8.36 4.63 164 116 50 50 .0 . . . 18.705. 1440. 1.000. 178. 62.7 310 21.5 14.0 10.4 7.23 188 3,14 70 .50 .0 . . . . 42.808. 8.840. 509. 115. 51.2 31.4 IS.7 14.4 S.96 7.68 4.14 10 0 .50 .0 . . . '. . . 2.080. 122. 80.2 44.4 28.3 20.4 14.0 10.7 164 15 0 .50 .0 . . . . . . 18.305. 21 4. 130. 71.4 42.5 20.4 *0.6 15.7 8.14 10.0 50 .0 . . . . . . .18205. 350. 180. 36.4 58.2 40.4 , 27.3 20.7 10.6 2.0 10 .0 5.650. 684. 201. 86.3 45.8 27.3 116 3.0 to .0 S.180. 1.560.

8.21 5.70 4.32 3.17 2.52 1.33 1.60 1.13

. 446. 174 83.8 29.5 14 8 124 6.56 4.54 3.45 2.56 2.06 las 5.0 1.0 .0 . 96.800. 11.500. 2.250. 620.

70 f.0 .0

. 102. 34.2 17.8 11.4 7.33 5.49 3.89 30s 1J6

. . . . 27.300. 4.220. 286. 61.5 28.8 16.4 10.2 7.49 5.23 4.05 2.23 10.0 to .0 . . . . . 72.700. 1.160. 117. 41.3 23.8 14.5 10.5 7.23 5.54 103 15 0 to .0 . . . '. . . 10.500. 251. 66'.2 36.3 21.7 15.5 10.6 8.08 4.28 20.0 1.0 .0 . . . . . . 30.900. 434. S t.1 48.8 28.8 10.5 13.s 10.6 5 53 2.0 2.0 .0 17 00. 1J00. 264. 33.3 410 218 138 5.60 174 2.79 2.05 1.68 1.33 1.17 1.01 10 2.0 .0 . 13.700. 1.830. 444 113 5.0 2.0 .0 .

152. 45.0 S.27 5.57 190 2.70 2.10 1 59 1.33 1.03 70 2.0 .0

. . 11.100. 1.900. 488. 64.3 19.8 S.87 6.34 4.12 3.09 213 f.79 1.13

. . . . 21000. 3.160. 176. 34.3 14.5 8.82 55.5 4.09 2.31 2.30 1.34 10.0 2.0 .0 . . . . . 54.400. 705. 63.6 21.8 12.6 7.69 5.59 3.91 1 05 1.76 15 0 2.0 .0 . . . . . . 8.320. 132. 34.3 18.8 11.3 8.09 5.57 4.30 2.33 20.0 2.0 .0 . . . . . * . 54.400. 228. 44.7 25.1 14.4 10.8 .714 5.55 3.02 2.0 3.0 .0 83.100. 2.480. 404. 118. 47.7 23.7 8.85 4.66 102 2.23 1.65 1.37 1.15 1.00 1.00 10 30 .0 .- . 24.300. 2.520. 520. 158. 82.3 18.7 7.28 4.25 2.36 2.06 1.62 1.22 1.12 1.00 10 30 0 . . . 12.200. 1.310. 430. 51.5 14.7 7.16 4.57 2.SS 2.28 1.66 1.35 1.02 7.0 30 0 . . . . 21700. 2.300. 139. 24 9 10.3 8.23 3.34 2.33 2.11 1.69 1.07 10 0 10 .0 . . 49.800. 548.

15.0 37 .0 . .

. . . 45.1 15.2 8.73 5.37 3.93 2.73 2.21 1.27

. . . . 4.890. 92.2 23.5 12.3 7.75 160 3.89 1 04 1.78 20.0 10 0 . . . . . 42.400. 156. 31.8 17.1 10.1 7.26 101 3.47 2.18 2.0 50 .0 . 12.000. 1.070. 214 88.7 4

t 3.0 50 0 27.6 8.13 3.85 2.38 1.75 1.32 1.14 1.04 1.01 1.00

. 96.500. 5.690. 836. 201. 67 0 14.3 5.57 3.12 2.15 1.53 115 1.07 1.02 1.00 5.0 50 0 . . . 18.300. 2.180. 428. 41.1 10 4 4 87 3.09 2.04 1.57 1.21 1.07 1.00 70 5.0 .0 . . . . 28.500. 2.870, 108. 17.0 8.78 4 09 2.61 10.0 50 0 .

1.97 1.44 1.19 1.00

. . . . 49.400, 422. 29 8 S.73 5 59 3.48 2.58 1.87 1.49 1.01 15.0 5.0 .0 . . . . . 3.740. 512 14.7 20 0 10 .0 8.09 4 SC 3.58 2.54 2.04 1.13

. . . . . . 32.300. 98.5 19.7 10.4 6.33 4 58 3.21 2.52 1.47 2.0 7.0 0 . 61.300. 2.350. 406. SS.4 34 2 8.16 3.47 2.07 1.52 1.15 1.07 1.01 1.00 t 60 3.0 70 0 . . 14.300. 1.460. 278. 78.4 116 4.79 2.53 1.78 1.30 1.12 1.02 1.00 1.00 5.0 7.0 0 . . . 24.000. 2.880. 465. 37.0 8.47 183 2.43 1.63 1.23 1.07 1.01 1.00 70 to 0 . . . . 30.000. 1040. 95.5 13.4 5.20 3.14 2.02 1.54 1.17 1.04 1.00 10 0 70 .0 . . . . . 52.600. 369. 23 0 7.32 4 21 2.65 I SS 1.44 1.16 1.00 15 0 70 0 . . . .' . . 3.250. 44 6 10.3 6.00 3.67 2.71 1.97 1 57 t ot 20 0 70 0 . . . . . .

• 28.100. 73.4 14 4 7.79 a ss 3.42 2.44 1.25 1.07 i 20100 0 . . 14 000. 1.090. 181 48 5 8 53 3.16 1 81 1 34 1.09 1.02 1.00 1.00 1.00 30 100 0 . . 61.600 3.610. 479 105. 13 5 4 16 2.16 1 50 1.15 1 04 1 00 1 00 1 c0 50100 0 . . . 48.500. 3.180. 566. 34 5 6 30 3 01 1 91 1.32 1.11 1.01 1 00 1.co 70 100 .0 . . . 733.000. 40.*00. 1470. 86 8 10.6 3.37 2.40 1 57 1 23 1.04 1 00 1 00 100 100 0 . . . . . 59.400. 332. 17 6 5 45 3.15 2.01 1 58 1.13 1 02 1 00

, 150 100 .0 . .- 2.320.

20.0 100 0

.- . . . 33.4 7.35 4 41 2.73 2 06 1 46 1.18 1 00

. . . . . . 25.200. 54 1 10.4 565 3.44 2.55 1.88 1.48 1 00

. . vow. ..,o, 20

(

... .- - _ . _ . - - . , . _ , - _n,m. . , , _ _ _ , . , . , , , , . , , . _ . , _ , _ _ _ _ _ _ , . , _ _ , , - , _ , _ . _ , . _ _ , _ _ , , , _ , . _ _ _ _ _ . , , , , , _ , .

A-8 Table A.2 Example for Selection of CUSUM Parameters whend f /fa = 3.5 L , max a S (in M nths) (f/f d A. ~

/

0.25 2 4.8 3.8 0.50 2 8.6 3.0

~

1.00 2' 15.0 2.0

_ . . . _ __._ _. 2.00 2 24.0 -

-1. 5 -

The near optimum value for f/fd < 3.0 corres-ponds to a = 0.5, = 2 and L = 9 months.

SA where:

~

f and fd = the actual and the alert value of the performance measure, respectively, and ARL (f) and ARL (fd ) = the average running lengths for f and fd ,

respectively.'

1 i

O b

<--- , , , - - - - -..,--_,,--.---,n -

p. _ , - - . - - - - - - - - - , - - . , - ,,m-,_

, o A-9 In general, there are no values of SA , a and L that satisfy the above equation for all possible values of 5. However, if f is limited to a multiple of tyo or three times f . dOptimum values for S , Aa and L can be determined (Note, in the ave =ple in Table A ., 2 the optimum values of SA , a and L are 2.0, 0.5 and 9 months, respectively, when f is limited to three times fd+)

It should be noted that the process discussed thus far assumes that the performance measures are samples from a poisson distribution. The extension of the CUSUM technique for other distributions will be investigated in the future.

A.l.5 Application Two applications of the CUSUM technique were carried out. In one appli-cation the objective was to investigate if the event of June 9, 1985 at the

, Davis-Besse nuclear power plant could have been prevented by means of detec-tion of substandard performance at some earlier time. In the second applica-tion, the CUSUM technique was applied to diesel generator test results for three selected plants. The purpose here was to see if the CUSUM would have triggered prior to completion of 100 tests. The next two sections of the appendix discuss these two applications in detail.

m-- w - , , - - - - , , - , - - - ,--,,y--,_.,--, ,-,,nw._mm,,wnm_,,,-,v-.--,,,,----.---ma,.y--,-_ _e,wn- - - - - . -n -

. .. l A-10 l In this section, the CUSUM technique was applied to the results of Generic Letter 84-15 in regard to diesel generator performance in nuclear pdwer plants. "he CUSUM technique is applied at the level'of a basic indica-

/

tor, namely the performance of diesel generator unit. The baseline value for diesel generator reliability f is assumed to be 0.025 (per demand) and the alert value fd is considered to be 0.05, cons * : tent with Generic Letter 84-

15. The response time for detection of substandard performance was considered to be 100 test demands in accordance with Generic Letter 84-15 requirement 1

(i.e., ARL (md) < 100 demands).

Taking the above considerations into account, the CUSUM parameters can be determined. Table A.3 presents the behavior of the CUSUM in terms of its response time for various sets of parameters. There are two sets of parameters which satisfy all the requirements of the Generic Letter and they are speci-fied in Table A.3. Out of these two sets of parameters, the one which has the longest false alarm response time has been selected. These parameters are:

L = 15 demands SA=2 m = 0.5 The results of the CUSUM application to Generic Letter data for three plants are provided in Tables A.4 through A.6. From these results, it can be concluded that the CUSUM technique is a powerful statistical tool that not only enhances rapid detection of substandard performance but also minimizes the possibility of false alarms. '

O

- - - - --- - - - - - - , , - , , . - - - . , - - ,.,,-,.r- ,-,,,---.-.----..-------w_ .

w ,y ,-, ,-,,,_.,.,,-----,-_,n, , - - - - - - - - - . , , - - _ . - -

A-11 Table A.3 Potential CUSUM Parameters for Generic Letter 84-15

• a L S ARL (fd )

g ARL (f,) f /f

• d

,.(Demands) (Demands) (Demands) 0.25 7 1 52.0 126 -

0.25 7 2 97.0 406** 4 0.5 14 1 28.5 140 -

0.5 14 2 105.0 462** 3 1.0 28 2 120.0 588 -

Note that:

fa = 2.5 (-2) -

, fd = 5 (-2) , a = 3.6 (-1)

• L a,/.xd = 0.7 a d*" *

• fmax/fd relates to the ratio between the poorest performance and the alert level which can be detected fast enough so that the resulting induced risk is less than the risk induced by the alert level. Therefore, faax is the largest value of f which satisfies the inequality.

r f

• ARL (f) i fd ARL(fd }

• • These are the two acceptable sets of CUSUM parameters that meet Generic Letter 84-15 requirements.

---,-,---,.wa ,,..,---..,-,y,,_,eww,.,w,

i, *.

A-12 Table A.4 Application of CU, SUM to Beaver Valley Generic, Letter Response Initial Final Demand D.G. Unit No. of Value of Value of Alarm No. I.D. Failure S g S 4 Signal

~

10& 85 1 7 0 6.5 1 85-70 1 3 0 3.5 1 70-55 1 2 0 1.5 0 55-40 1 1 1.5 2.0 1 40-25 1 4 0 3.5 1 25-10 1 0 0 0 0 10-0* 1 1 0 0.5 0 100-85 2 6 0 5.5 1 85-70 2 1 0 0.5 0 70-55 2 3 0.5 3 1 55-40 2 0 0 0 0 40-25 2 1 0 0.5 0 25-10 2 0 0.5 0 0 10-0* 2 -

0 0 0 0

• Demand numbers 10-0 which constitute 10 demands were assumed to be 15 demands.

4 M, O me l

1

. - - - - - ....--,-.-.-.._--------m -

A-13 Table A.5 Application of CUSUM to Trojan Generic Letter Response Initial Final' Demand D.G. Unit No. of Value of Value of Alara

, No . I.D. Failure S ~~~

Sg A 'SiE**1' ~ ' ' ' ~ ~ ~ ' - ~ ~

100-85 K106A 0 0 0 0 85-70 K106A 1 0 0.5 0 70-55 K106A 0 0.5 0 0

, 55-40 K106A 0 0 0 0 40-25 K106A 3 0 2.5 1 25-10 K106A 1 0 0.5 0 10-0* K106A 3 0.5 3 1 100-85 K106B 2 0 1.5 0 85-70 K1068 0 1.5 1.0 0 70-55 K106B 2 1.0 2.5 1 55-40 K106B 1 0 0.5 0 40-25 K106B 1 0.5 1.5 0 25-10 K106B 4 1.5 5.0 1 10-0* K106B 0 0 0 0

• Demand numbers 10-0 which constitute 10 demands were assumed to be 15 demands.

.g.

A-14 Table A.6 Application of CUSUM to TMI-2 Units DF-X-1 and DF-X-1B Generic Letter Response Initial Final Demand D.G. Unit No. of Value of Value of Alarn.

No. I.D. Fai, lure S A

S A - SiE"*1 100-85 DF-X-1 1 0 0.5 0 85-70 DF-X-1 0 0.5 0 0 70-55 DF-X-1 2 0 1.5 - -0~-- " -

55-40 DF-X-1 0 1.5 1.0 0 40-25 DF-X-1 1 1.0 1.5 0 25-10 DF-X-1 2 1.5 3.0 1 10-0* DF-X-1 0 0 0 0 100-85 DF-X-1** 0 0 0 0 85-70 DF-X-1 0 0 0 0 70-55 DF-X-1 2 0 1.5 0 55-40 DF-X-1 0 1.5 1.0 0 40-25 DF-X-1 0 1.0 0.5 0 25-10 DF-X-1 1 0.5 1.5 0 10-0* DF-X-1 0 1.5 1.0 0 100-85 DF-X-1B 0 0 0 0 85-70 DF-X-1B 2 0 1.5 0 70-55 DF-X-1B 1 1.5 2.0 1 55-40 DF-X-1B 3 0 2.5 1 40-25 DF-X-1B 0 0 0 0 25-10 DF-X~13 0 0 0 0 10-0* DF-X-1B - - - -

100-85 DF-X-1B** O O O O 85-70 DF-X-1B 2 0 1.5 0 70-55 DF-X-1B 0 1.5 1.0 0 55-40 DF-X-1B 0 1.0 0.5 0 40-25 DF-X-1B 0 0.5 0 0 25-10 DF-X-1B 0 0 0 0 i

10-0* DF-X-1B - - - -

l

• Demand numbers 10-0 which constitute 10 demands were assumed to i be 15 demands.

• • Human error during testing is removed.

_ - _ - . _ - . . . - - - . . _ . _ - - . - . _ - . . - . . . . - - . . .--._..,,n__ - -. . - - , _ _ _ _ . .

B-1 APPENDIX B B.1 Effectiveness Measures for Maintenance Activities

• The evaluation of the effectiveness of surveillance and maintenance pro-grams in nuclear power plants on the basis of component repair history are discussed here. The surveillance and maintenance program is considered to be i

' effective if the component unavailability is maintained within an acceptable level by assuming that component degradation is detected and repaired prior to i

t bec6 ming a catastrophic failure.' -

The reduction in an unavailability of a standby component when the repair initiated at the degradation stage compares to catastrophic failure occurrence i can be evaluated through the Markov approach. Certain types of. components ex-perience failures which are a result of a transition from a degraded state to 4

a catastrophic state. For these component types, the impact of an. effective maintenance program, in terms of reduction in component unavailability, can be as high as a factor of 30 as shown by the analysis. In addition as a result of this analysis, a proper measure is identified for evaluating the effective-ness of maintenance policies. This measure, namely, the ratio of repair counts of degraded failures over the total number of repairs is directly re-lated to the ability of detecting a failure at a degraded stage, therefore in-dicating an effective preventive maintenance program. The results also show that the expected number of repairs (regardless of catastrophic or degraded) within a fixed period of time (several times the maan time to degradation) is constant unless the built-in design reliability of the component has been de-graded either due to aging / wear out or inefficient corrective maintenance (band aid fixes). A CUSUM type of technique can be applied to monitor these measures in order to determine any improvement or reduction in the effective-ness of the maintenance program. This type of approach can also identify both the existence of band aid fixes and initiation of component wear out. Further investigation in this area and application to actual plant experience data is needed.

B.2 Methodology to Evaluate Maintenance Effectiveness The approach taken for evaluating the effectiveness of the maintenance program is based on the discontinuous Markov process. The discontinuity re-suits from the periodic inspection / surveillance and it is represented by a periodic Dirac delta function. The state diagram for the Markov process is given in Figure B.1 and the associated description of the states and the transition parameters are presented in Table B.1.

Markov Equations Due to the discontinuous nature of the Markov process, we need to define nT = n T-6 nT+ = n T+6 i

j l

,, c.

B-2 where:

5 = the smallest possible number to differentiate nT+ from nT-

, T = the surveillance interval. '

Assumption: Repair is assumed to be completed before surveillance can take place, i.e.,

P3 (nT-) = Pg (nI-) = 0 Since repair always initiates right after surveillance (t=nT), actual and running time (actual and repair state residence time) would be the same and repair distribution can be generalized (time dependent hazard function for repair).

Initial conditions Po (a f) = Po (nT-)

P1 (n d) = Pt (nT-) * (1-c)

P2 (nT+) = 0 P3 (nT+) = P3 (nT-) + Pt (nT-)

• c Pg (nT*) = Pg (nT-) + P2 (nT-)

Equations of State Dynamic for nT+ < c < (n+1)T-1 P(t)=(1-Fi(t-nT))P1 (nT-) c 3 ,

Pg(t)=(1-F2(t-nT))P2 (nT-)

where Note Ft and F2 are cumulative repair distribution associated with pt and 42' that based on the assumption that repair is completed before surveillance, F1 (T) = F2 (T) =.1. Therefore, the remaining state equations are:

P0 (C) " -(Ad+AC1) P 0 (t) + Pt (nT-) *caf1 (t-nT)+P 2 (nT-)f2 (t-nT)

Pt (t) = Ad Po(t) - Ac2 Pt (t)

P2 (t) = Ac2 P1 (c) + Act Po(t)

.-------wy -, v----- -%~f--.-- ----,_e---, r-%, . -,----c------

,;;' i t

, .. a sg) .. c

, s*

\

s X

-  ;\ )

3'-

s

, i it f t l [; L'

( ., t'  :

i y x ..

(

JV . U V' '

.MARKOV STATE DIAGRAM

. J -

. ( Po )

t , s , A4 -- .

\

\I ,,

t Aci 001 .

.' (Pt ) #1 c6(t-T AC:

0 11 i(P2 ) 101 6(t-T) .

h 111 Figure B . I. The state diagram for the Markov model.

4 4 S

,- s a

. 5-4 Table 3.1 Discrete Markov Modeling to Analyze the Impact of

, Survei,llance and Maintenance ,

• Markov State:

000: operable state 001: degraded state but not detected 101: degraded state which is detected 011: catastrophic state (failure) but not detected 111: catastrophic state (failure) which is detected Test interval is T and test duration is neglected Transition Parameters:

s Ad  : mean time of occurrence for degradation c  : test efficiency in detecting degradation AC2  : mean time of occurrence for a catastrophic failure as a result of an existing degradation Act  : mean time of occurrence for intermittent catastrophic failure 6(t-T) : Delta function ut  : mean repair time for degradation p  : mean repair time for catastrophic failures 9

6. < '. .

B-5 Analytical Soluticas Analytical solutions when repair time is negligible compared to surveillance interval (f1 and f2 are approximated by delta functions) are:

P0 (E} " 1(" )* + 2(" ~I + O(nT-))exp[-(Ad+Ac)(t-nT)]

y Pg (t) = Pg(nT-)(1-c) exp (Ic2 (t-nT)] +

[P(nT-)*c+P(nT-)+P("

g 2 0 )

Ad+Ag - {eXP(-A(t-nT)]-exp(-(Ad+Ac)(t-nT)]}

2 y P(t)=[P(nT-)c+P(nT-)+P("T)]

2 t 2 0 g+

Ac

[Pg (nT-)c + P2 ("T-)] (1'**PI~(Ad+A"l)(E~"T)I}+

1 Pg (nT-)(1-c) (1-exp{-Ac2(E""T)~ (Pg (nT~) c + P2 (nT-) + P0 (" )I Ad+Ac1 -Ac eXP (-Ac3 (tmT)] - 2 Ae. Ad+Ac exP [-( % g)(t-nT)]} l State Probabilities at Test To find the probabilities of various states at test is equivalent to in-serting t=(n+1)T into the analytical equations. The resulting equations are in the form of Pgg = A P n where P and P contains the state probabilities at testnumbern,namelydu 'd* 2n i v e

B-6 The Transition Matrix A is defined al cat _ at A= c4 (1-c)a2+Cc4 c4 c2+c6-c5 Ec2+ec6+ c2+c6-c5 (1-C)(1-a2)"Ec5 -' where: at = exp (-(Ad+Act)T] .. . a2

• 8xp (-Ac2T)

'                      et = Ad/(Ad+Act-Ac2) c2 = Ad/(Ad+Act) c3 = Act/Ad+Act                                                                   ,

c4 = ct(a2-at) c5 = ct(a2-(Ac2/Ad+Act)*at] c6 = c3 (1-st) , Numerical Results Definitions, Degradation dominant failure path: those components for which the dom-inant f ailure mechanism is due to degradation rather than intermittent fail-i ures. That means: No. of catastrophic failures resulted from degradation is much larger than number of intermittent catastrophic failures, or mathematically, j 1/Ad + 1/Ac2 < 1/Act Some of these components are bearings, seals, packings, breaker latches, etc. Semi-degradation dominant failure path: those. components for which the failures result from both degradation and intermittent causes. That means: No. of catastrophic failures resulted from degradation is comparable to the number of intermittent catastrophic failures, or mathematically, 1/Ad + 1/Ac2 = 1/Act Some of these components are motor windings, governors, high powered SCRs, etc. l i l' l 1

                    , -- ,        , , , , -           . , , - - . - - - - - - -     - , , . - . - -            ..-,--------e         ,.-

3_7 Intermittent dominant failure path: those components for which the dominant failure mechanism is due to intermittent failures rather than degradation. As such are those components with large expected service life, small electronic devices, shafts, pipings, relays, etc. Similarly, No. of catastrophic failures resulted from degradation is much smaller than number of intermittent catastrophic failures, or mathematically,

                        ,           1/Ad + 1/Ac2 > 1/Act                           ,

Knowing these definitions, various sensitivity runs were performed using the mathematical models discussed previously. These are given in Tables B.2 through B.4 for three categories, namely: degradation, semi-degradation and intermittent dominant failure paths, respectively. Some of the conclusions were highlighted which may help the determination of indicators for effective-ness measures. B.3 Conclusion The total number of repairs (regardless of catastrophic or degraded) in a fixed period of time tends to be insensitive to test strategies and inter-vals. It is only a function of component reliability performance parameters. This indicator is appropriate for identification of components that are trend-ing towards substandard performance. It should be noted that the total number of repairs provides more abundant data compared to catastrophic failure counts. However, due to the significant effort involved in data collection, this indicator is to be used for highly risk-significant components. Also, it should be noted that normal variation of repair counts in a fixed period fol-lows a binomial distribution. (Parameters are number of tests within the period, number of repairs, and probability of repair.) The ratio of repair counts of degraded failures over the total number of repairs is quite sensitive to surveillance and maintenance policies. The change of this ratio given that the total number of repairs is within the ac-captable bounds will provide a proper measure of effectiveness for evaluating , maintenance and surveillance policies. i I e y- , , - - ,,- - . . _ _ - , - - - -

38 Table B.2 Results of Degradation Dominant Failure Path Ad = 1.E-5 Act = 1.E-6 Ac 2 = 1.E-3 E=1 E = 0.5 Test ' Int:erval NOR/10 yrs RONC/R ~ RAT NOR/10 yrs RONC/R RAT 2000 0.95 39% 0.98676 0.949 21% 0.98289 1000 0.95 57% 0.99534 0.955 35% 0.992936 720 0.95 64.7% 0.99722 0.956 42.8% 0.99504 500 0.95 11.5% 0.99844 0.958 51.33% 0.997339 200 0.95 82.4% 0.99961 0.959 69.7% 0.999338 NOR/10 yrs : Expected total number of repairs in 10 years RONC/R  : Ratio of non-catastrophic repairs over total number of repairs RAT  : Reliability at test 9 L 6 e

3-9 Table B.3 Results of Ceni-degradation Dominant Failure Path Ad = 5.E-6 Act = 5.E-6 Ac2 = 1.E-3 E=1' E '= 0.5 Tesd Interval NOR/10 yrs RONC/R RAT NOR/10 yrs RONC/R RAT

                                        /

2000 0.867 21.5% 0.9845 0.865 11.5% 0.982 1000 0.8715 31.5% 0.9932 0.870 19.3% 0.992 720 0.8727 35.6% 0.9954 0.871 23.5% 0.994 500 8.737 39.3% 0.9969 0.872 28.2% 0.9964 200 8.746 45.3% 0.9989 0.873 38.3% 0.9988 NOR/10 yrs : Expected total number of repairs in 10 years RONC/R  : Ratio of non-catastrophic repairs over total number of repairs RAT  : Reliability at test e e e

            ~

B-10 Table B.4 Results of Intermittent Dominant Failure Path Ad = 1.E-6 Ac t = 1.E-5 Ac2 = 1.E-3 E=l' E '= 0.5 Tesd Interval NOR/10 yrs RONC/R RAT NOR/10 yrs RONC/R RAT

                            ~

2000 0.953 4% 0.979 1000 0.958 5.7% 0.9897 NOT 720 0.959 6.4% 0.9926 CALCULATED 500 0.96 7.15% 0.995 200 0.96 8.2% C.998 NOR/10 yrs : Expected total number of repairs in 10 years RONC/R  : Ratio of non-catastrophic repairs over total number of repairs RAT  : Reliability at test t e l l l

I*

• C-1 APPENDIX C

SUMMARY

DOCUMENT FOR TRIAL APPLICATION C.1 Objective / Scope The purpose of this appendix is to detail progress on a trial application , of the reliability technology d'escribed in Sections 3 and 4'of this report to an actual EDG. In order to gain experience in interfacing with a utility this ap-plication was conducted with cooperation of the Portland General Electric (PGE) Company. The particular system presently under investigation is the emergency diesel generator system at PGE's Trojan plant. The concept underlying the trial application integrates reliability technology into routine operational activi-ties to help in:

1. controlling and monitoring performance against set goals;

2. recognizing deviations from these goals, prioritizing important devia-tions and identifying their root cause; and

3. taking corrective actions and tracking the effectiveness of the ac-tions taken.

The primary focus is therefore on the operational activities presently conducted at the Trojan plant for maintaining the reliability of this particular system and how insights in enhancing these operational activities can be gleaned from reliability technology. Section C.2 briefly describes the basic approach employed for investigating the reliability of the emergency diesel generator system at the Trojan nuclear power plant. Eight basic general tasks are identified for potentially detecting, correcting, and resolving reliability-related problems. A reactive as well as predictive reliability program process is indicated for respectively correcting problems when they arise and for forecasting when reliability-related problems may occur. For this particular application, a four stage analysis process is described for identifying critical components (i.e., components having various failure ' modes that can contribute to system unavailability) and operational activitier that can be used to improve component reliability. A detailed presentation of the crial application process is contained in Reference 5. A description of the diesel generator system and its subsystems can be found there along with sources of failure data, both generic and plant specific and other information needs required to implement the specific reli-ability techniques listed in Section C.2 for identifying critical components. Section C.3 lists the critical components that have been determined through a synthesis of analyses that includes: fault tree quantification, reliability block diagrams, failure modes and effects analysis (FMEA) and plant-specific l data analysis obtained from PGE-supplied maintenance records. l l l

C-2 Section C.4 highlights present operational activities conducted at the Trojan site and ascertains the extent to which the failure modes associated with these critical components are identified by the current operational activities. Those that are not identified are noted. Insights and conclusions gained from this study are described in Section C.S. C.2 / Basic Approach This section provides an overview of the methodology employed for investi-gating the reliability of the emergency diesel generator system at the Trojan nuclear power plant. The goal of this study is to develop an apprasch for form-ulating a reliability program plan and eventually, with the cooperation of PGE, implement the plan at its Trojan operating unit and subsequently track its ef-factiveness 'in improving or maintaining the reliability of the diesel generator system. The methodology used is largely based on the reliability techniques, tasks, and activities outlined in the OSRR project plan. This methodology is further elaborated in Section 2 of this report which describes the tasks and structureof a reliability program. This reliability program structure is implemented in this application through a four-stage process to seek approaches and develop strategies for improving or maintaining the system reliability. Within this process, the effectiveness of current diesel generator tests were specifically examined to identify problems (or potential problems) and determine their causes. As will be further described, the surveillance tests were determined not to be able to detect all failure modes. The analysis also shows that some of these failure modes were not crucial to the operability of the diesel generators. Accordingly, the process described and utilized addresses the three basic, top-level tasks that a reliability program must undertake to accomplish its basic 'obj ectives for reducing the frequency of transients, for controlling faults that challenge safety systems, and for providing assurances that a safety system functions properly when called upon to mitigate abnormal occurrences. l

C-3 , l I Reliability program casks, activities, and techniques that have been en- ' ployed in this study are depicted in Figure C.1, which also portrays the process  ; employed to analyze, identify, prioritize, and resolve either recurring problems I or problems that can occur. Figure C.2 details further the problem identifica- I tion, problem resolution, and corrective action implementation tasks.  ! Risk- and' reliability-based techniques, along with engineering analysis, dere employed to ferret out credible potential reliability' problems and to sug- i gest / alternatives to the plant's maintenance, surveillance, and operational activities which can conceivably eliminate these potential problems.

                                               /

Reliability block diagrams, failure modes and effects analysis and fault trees were developed to identify what faults may exist to prevent emergency ac power supply'in the event that off-site power was lost for a given period of time. For a loss of offsite power (LOSP) initiating event, the system boundary not only included the support systems required for performing the desired system function, but also portions of logic systems designed to assess the functional performance of the diesel generator system. Reliability block diagrams of each diesel generator subsystem were con-structed to identify the system success paths in system operation for further analysis. The logical simulation of functional failures at the system and train level was provided by constructing fault trees at the subsystem level and ob-taining cutsets at the train / system level. In addition to constructing and analyzing the fault trees / block diagrams, failure modes and effects analysis was employed to verify this " top /down" approach as well as identify failure effect and assess the adequacy of tests to discover component failures. Using existing data (generic or plant specific), major components of the system were priori-tized to indicate the potential consequence of an undetected failure. With this information, a four ' stage approach can be undertaken in order to eventually develop a reliability program plan for the emergency ac power system. The steps that are considered necessary in developing a system-specific relia-bility program plan include the following: System Analysis - identify critical components from system / train level fault trees. l For each dominant cutset, the possibility of recovery actions was investi-gated by review of the FMEA sheets and the plant's Off-Normal Instruction (ONI) procedures. The possibility of human-induced common mode failures during test and maintenance were also examined. This process was repeated at the diesel

                                                                           +

g w

                 , g

C-4 e

                     /
                                                                              .g                     .
                            /,,                                                                                    g
                                                                               !      Pfent Ooerotional            j j           Actmties               j

:. nosenisity improvement

: Program

                                          ,                                    I
                                                                                ' Walk-thru inss,ection       '

t Wask thru inspecten

                           ,                                ,            ,a      Periodic operational testing     .,    procedures frevennve maintenance          ,      2. Periodic operanonal Train level                                                                             test procedures Faut Tree Cutsets
                                *Fedure to start'

3. Preventne mamtenance

                                                                               ' Data tracking                          and optimum senedule
                          , Failure to run'                 ,

Priontized , Items Ust < **

4. Condition monitoring

                                                                               , Multivariate analysis       .

j Prioritization j j Dend analysis j j of system j j of component j { j components j j performance j Figure C.1 Reliability techniques used to enhance plant operational activities. O

\ FLOWCHART OF PROBLEM ANALYSIS TASK

                                                                          ~

INDUSTRY

WIDE /

GENERIC DATA , ! DEFINE SYS1tM 8OUNDARY CONSTRU G FAULT TREE j DESIGN INFO. P & ID > ' " ' " ^ "^ CONSTRUCT GUANTIFICATIONS l O S > > FUNCTIONAL  ; & IDENTIFICATION O IDENTIFY SUPPORT DIAGRAM

                                                                                 ^                                  ^

SYSTEMS J, PLANT DEVELOPMENT IDENTIFICATION OPERATIONAL u OF FMEA OF CRITICAL ACTIVITIES '

    & PROCEDURES

• COMPONENT UST i

I PLANT IDENTIFICATION COMPARISON EXPERIENCE > AVAIMBILW u AND INTEGRATION

                                                >    OF MAJOR               .                       .

ANALYSIS DATA CONTal8UTORS OF CRITICAL 7A

,                                                                               COMPONENT UST l

Figure C.2 Methods Used in Prioritizing System Components for Reliability Improvement g 9 __ _

FLOWCHART FOR IDENTIFICATION OF POTENTIAL RELIABILITY ACTIVITIES - TO MAINTAIN AND IMPROVE RELIABILITY . IDENTIFY THE CURRENT

-> PR ACTICES FOR I

DETECTION OF DEGRADATION IDENTIFY THE CURRENT

                  ->               MEANS FOR EAF.'LY DETECTION OF FAILURE
  /^\                                                                                '

RECOMMENDATIONS - FOR OPERATIONAL EVALUATION OF EXPECTED ACTIVITIES *

• EFFECTIVENESS TO MAINTAIN AN P OF IMPLEMENTING

                                                                ,E   BU                 THE RECOMMENDATIONS t

IDENTIFY OTHER ASPECTS

                 ~           + CURRENT OPERATIONAL LIMITS
                             + MAINTENANCE PROGRAM
           ,

• ADEOUACY OF EXISTING DESIGN CURRENT MEANS

[

  /3\

FOR DEFENSE AGAINST MULTIPLE i COMPONENT OUTAGES Figure C.2 Continued  !

            ,                                                                 C-6 generator train level in order to identify. additional critical components. On the basis of insights gleaned from the system / train fault trees, the critical component list was developed.                       Analysis of plant experience data was also used to supplement this list.

Operational Activity Analysis - identify operational activities that can be used to improve component reliability through (1) rapid detection of failures, (2) timely and proper corrective maintenance, and (3) effective preventive maintenance and condition monitoring schemes. _ For possibly detecting failures rapidly, the types of component failures that could be detectable during plant " walk-throughs," by existing system-speci-i fic alarms and instrumentation, and through the adequacy and efficacy of period- - - ic testing were investigated. Investigations into timely and proper corrective maintenance activities focused on those controlling measures that could assure proper maintenance is performed. Issues such as type testing after failure should be addressed. Detecting degradation of critical components on the basis of parameters measured during periodic operational testing and triggering when preventive maintenance should be performed is the purpose of preventive maintenance and condition monitoring. Developing recommendations for these operational activi-ties requires that (1) for each critical component a set of measurable para-meters is defined for developing a condition monitoring scheme and (2) a refueling outage program and schedule be developed to help in detecting degraded components. Reliability Improvement Analysis - the effectiveness of the recommendations made in the previous two stages that are geared towards component level relia-bility enhancement were quantitatively analyzed and the improvements in relia-bility that accrue for a diesel generator set and for the entire emergency power i- system are investigated. Reliability Performance Analysis - The analysis conducted thus far concen-trated on those operational activities which, when implemented, could improve system reliability. This stage, which has not been undertaken but is considered

important in the development of an overall reliability program, should concen-l trate on how to effectively monitor risk and reliability performance. This stage of the overall process is to identify two types of indicators

perfor-mance indicators and effectiveness measures. Performance indicators determine the reliability of the emergency ac power system; so-called effectiveness mea-sures de.termine the outcome of the operational activities in achieving reliabil-ity performance.

C.3 Identification of critical Components In this section, components that contribute to unavailability of the emer-gency ac power system are listed. They have been identified on the bases of the i plant-specific data analysis and fault tree analysis.

u l% C-7 Plant-specific data was analyzed from available maintenance records while the analysis of the system fault tree model utilized a generic data base for quantification. The diesel generator experience data for the Trojan nuclear plant were ob-tained by analyzing maintenance work requests from 1983,- 1984 and the first 5 months of 1985. A total of 91 maintenance records were examined and each diesel generator failure was categorized by severity, engine condition at the time of fault detection (standby or running), stress cause, repair category, and effect upon the system (immediate or 'long term). The severity of a diesel generator failure was ranked according to three degrees: (1) catastrophic, (2) degraded, and (3) incipient. The results indicate that catastrophic failures are largely caused by failures in electrical components. Degraded and incipient failures are dominated by faults in mechanical components. The fault. tree model of the Trojan emergency ac power system (EPS) provides the basis for analyzing EPS failures at various levels of system unavailabil-icy. The multi-level analysis was used to identify the dominant component failures at four levels: (1) the unavailability of the emergency ac power sys-tem, (2) the unavailability of one train of emergency ac power, (3) the failure of one tandem unit to start, and (4) the failure of one tandem unit to run (or operate) af ter a successful start. The methodology and truncation values used to obtain " critical components" lists at each of these levels of analysis is l discussed in Reference 1. An integrated list of critical components based on fault tree analysis is also given in this reference. The list of critical components that contribute to system unavailability on the basis of fault tree analysis is intenated with the list of components that cause catastrophic and degraded failures on the basis of plant-specific data analysis. Table C.1 shows all the critical components on the integrated list. The identification of critical components that cause system unavailability on the basis of integrating fault tree evaluation with plant-specific data analysis shows a more complete mix of " active" and " passive" components that may fail the system. About 30% of these components are electrical while the rest are mainly mechanical component's. C.4 Reliability Techniques to Enhance Performance In this section, the integrated critical components list is used to systematically analyze the adequacy of current operational reliability activi-ties at Trojan and to identify areas where component reliability can potentially be improved. Strategies for reliability enhancement are presented for implemen-tation considerations. L___ __ __ _ _ - - _ _ _ _____-~- _._ _ _ . - - - - - - - _ - - - - - - - -- -- - - - - - - - - - - - - - - - - -

          .*>                                                              C-8
  , . .- ~

Table C.1 Integrated Lists of Critical Components

1. Field flashing circuit

2. Generator excitation circuit

3. Voltage regulator (automatic / manual)

4. Diesel generator " Start /Run" control circuit

5. Circuit breaker 152-108 closing coil

6. Generator lockout relay (186-1D1, 186-1D2)

7. Generator stator winding

8. Service water / jacket water heater exchanger

9. Service water motor-operated valve

10. Main lube oil pump strainer

11. Lube oil scavenging pump strainer

12. Air compressor unloader

13. Jacket water thermostatic control valve

14. Engine main bearings

15. Camshaft / timing gear

16. Generator bearing / coupling

17. Generator slip-rings and brushes

18. Crankshaft-to piston connecting rod i

19. Lube oil scavenging pump

20. Main lube oil pump

21. Engine jacket water pump l 22. Crankshaft i 23. Fuel oil day tank outlet valve l

24. Lube oil cooler

25. Turbocharger aftercooler l 26. Engine crankcase pressure instrument 1 27. Expansion tank' i 28. Annunciator

29. Engine speed control switch

30. Fuel oil transfer pump breaker

31. Voltage regulator selector switch

l ** C-9 C.4.1 Approach to Improve Operational Activities The basic requirement of a reliability program is to maintain or improve equipment reliability. Such a program should provide means for detecting un-satisfactory equipment performance and for forecasting potential reliability-related problems. When problems associated with equipment reliability are detected, the program should provide methods and techniques for correcting the problems and observing that corrective actions have been implemented. After application of corrective actions, problem closure is assured by verification of problem correction. Therefore, the essential tasks of an operational reliabil-icy program are: (1) problem detection, (2) problem prioritization, (3) problem cause analysis, (4) determination of corrective actions, (5) implementation of corrective actions and (6) verification of problem correction. These tasks are elements in a closed-loop process for integrating reliability-based problem analysis .and corrective actions into day-to-day plant operations. This con-tinuous process is depicted in Figure 2.3 of Section 2 which portrays how a reliability program operates. The methods and techniques for performing each individual task in this reliability program structure are shown in Tables 2.1 and 2.2 of Section 2. In day-to-day plant operations, equipment failure or degradation is , detected by mechanisms such as walk-through inspections, existing alarms and I annunciated instrumentation, periodic operational testing and condition monitoring schemes. When problem areas are identified, prioritization is important to focus attention on system components that have impact upon plant safety. Prioritization of system components can be performed using reliability techniques of: PRA, failure modes and effects analysis (FMEA), fault tree analysis (FTA) and analysis of plant experience data. The application of these techniques provide an information base for analyzing problem cause. The root cause of failures, common cause failures and the adequacy and efficiency of l testing can be identified. When the cause of problems is identified, corrective l actions can be determined. Corrective action strategies are design changes, l procedural changes, preventive or corrective maintenance and improved training. I When a proper corrective action is selected, the implementation is decided by schedul'ed priorities. Problem closure is assured by verification of problem correction through documentation. If corrective actions are not successful in maintaining equipment reliability, additional methods such as condition monitor-ing schemes may be implemented to flag-out component degradation before de-terioration to complete failure. The major activities associated with maintaining the reliability of an im-portant safety system in a nuclear power plant are: (1) surveillance testing / visual inspection, (2) preventive maintenance and (3) corrective maintenance. A systematic way to possibly improve these operational activities is by using a prioritized list of critical components as presented in Section C.3. These components were identified by using risk- and reliability-based insights, in order to focus attention on potential problem areas. If the critical components that are identified are not considered in present operational reliability activities, then they would be candidates for improved surveillance including

          - . - - - , - -        ---m-e,   - _ .  ,,--_,w-    ,,_,,,,m,,_ , . _ _ , _                ,_, ,,, ,_,, .__

_,,_,,_,,_m.,_y , , , . . . _ _ , _ _ _ _ , , _ , .

C-10 visual inspection and condition monitoring. This list of critical components also indicates where preventive maintenance needs are more useful. The effectiveness and practicality of a reliability program is assessed by evaluating the adequacy of the various activities that could minimize the un-availability at the component level. The activities that reduce unavailability are prompt detection of component failure and degradation, timely and proper corrective maintenance and effective preventive maintenance and condition monitoring schemes. The systematic approach for assessing ,the effectiveness of a reliability program therefore involves a review of current activities that are performed on a specific plant system. Potential problem areas that were identified on the basis of ' a prioritized list of critical components become candidates for reliability improvement. An analysis of the strategies for reliability improvement of system components can then provide a measure of the effectiveness of the reliability program. C.4.2 Strategies for Reliability Improvement Although quantitative analysis has shown that the system reliability of the Trojan emergency diesel generators is high (96.7%), the above analysis of current operational activities to maintain this reliability indicates that .there are areas for improvement at the component level. The specific areas are in walk-through inspection, periodic operational testing and preventive maintenance for critical components that were prioritized from plant experience data and fault tree analysis in addition to FMEA. Table C.2 shows the prioritized components that are monitored by the various plant operational activities and Table C.3 summarizes the appropriate strategies for addressing the problem areas in the operational activities for enhancing the reliability of critical components. C.5 Insights Gained from the Application of Reliability Techniques This section summarizes the insights gained on the application of reliabil-icy techniques for investigating the adequacy of operational activities for improving reliability of the Trojan emergency diesel generator system. C.5.1 Fault Tree Analysis The fault tree analysis of the emergency diesel generator system was per-formed at the function, system, and subsystem levels to provide an information base of generated cutsets for prioritizing critical components in the system. Obviously the generation of cutsets at a lower level of analysis is less burden-some, but the incentive for analysis at a higher level is that system interac-tions and interdependencies are more easily identifiable. e _ _ , , , , _ , . ,- - . - , , _ _ . _ , . , _ _ ,, .._ _ ,, ,, ~, -

 -.~           * ""                                                                            C-11 Table C.2 Prioritized Components Monitored by Plant Operational Activities Operational Activities -
                                                                                                                                                                                      ~

Components Walk- Petiodic Semi-Annual

                                    /                                                                               Through         Operational and Annual Inspection                Testing    Maintenance Field flashing circuit                                                                                        X Generator excitation circuit                                                                                  X Voltage regulator (automatic / manual)                                                                        X Diesel generator " Start /Run" control cire.                                                                  I Circuit breaker 152-108                                                                                       I Generator lockout relay (186-lDI, 186-lD2)                                                   X                X Generator stator winding                                                                                     I                              I Service water / jacket water heat exchanger                                                  X               X                              X Service water motor-operated valve                                                                            X                             X Main lube oil pump strainer                                                                                   I                             X Lube oil scavenging pump strainer                                                                             X                             X Air compressor unloader                                                                      X                X                             X Jacket water thermostatic control valve                                                                      X                              X Engine main bearings                                                                                         X                              X Camshaft / timing gear                                                                                       X                              X Generator bearing / coupling                                                                                 I                              X Generator slip-rings and brushes                                                                             I                              X 4

Crankshaft-to piston connecting rod X I Lube oil scavenging pump I X X Main lube oil pump X X X Engine jacket water pump X X X Crankshaft I X Fuel oil day tank outlet valve X X Lube oil cooler I X X Turbocharger aftercooler X X Engine crankcase pressure instrument I X Expansion tank X X X Annunciator X 4 Engine speed control switch X Fuel oil transfer pump breaker X Voltage regulator selector switch X

                                              , - , -   ,-,---y---- -m- - - - - -- - - - - -   # w- -r-- --- - ~ - - - - - '             ' ~ ~ ~  - - - - - - - - - - ~ ~ ~   ~ ~ ' -

  ~~                       k C-12 Table C.3 Reliability Strategies to Enhance Daily Walk-Through Inspection, Periodic Operational testing and Preventive Maintenance Operational Activity /                                                  Strategy,for Reliability Component                                                 Pro *cedure Affected
                            /                                                                                                                                            Improvement                                                                             ,

Govaraor & associated Shift walk-through hydraulic lines Check for signs of oil' leakage (POT 24-1)

                                            ~

Fu:1 lines Shift walk-through Inspection for signs of rupture or leakage Fual oil day tank Shift walk-through Verify consistency of fuel tank level indication (visual vs instrument reading) tSnevice water motor Shift wall-through Verify proper indication of cparated valve MOV with respect to remote indication in control room

• Engine lockout relays Shift walk-through Verify that relays are reset (186-lD2, 186-2D2) t* Maintenance / Auto Shift walk-through Verify that selector switch is S21setor switch in the auto position (if no (krylock switch) maintenance is being performed) l Air receiver discharge Monthly test (POT 12-1) Provide a means to verify that velvas 18-month test (POT 12-2) no single receiver discharge valve is clogged Jcekst water / lube oil Monthly test (POT 12-1) When feasible, perform chemi-systins 18-month test (POT 12-2) cal analysis of lube oil, jacket water for indications of metal wear, corrosion, or contamination across systems Field flashing circuit, Preventive maintenance Perform inspection of relays gansrator excitation procedure MP-7 l circuit, diesel control for signs of sticking, contact l circuit degradation, or foreign matter l

     *Alcra provided for off normal conditions (local and remote).

tIndication provided in control room. v.-y, - - - - - - -s - _ - - - . - _ _ . _ - - . - . _ - . , . , , , . , . , . - , . _ - - . - . __mm,__.._%- - . _ _ , _ - , - - - . _ _ _. --_- _., .

m '" . C-13 The generation of fault tree cutsets is limited by truncation to keep the large number of cutsets manageable. For. example, analysis at the system level with a truncation value of 10 g generated about 10,000 cutsets whose majority are of high rank. However, analysis at a truncation value of 10

• generated about 500 cutsecs that are mostly double and triple cutsets. Therefore, the identification of critical components on the basis of dominant cutsecs is limited by truncation effects.- The choice of an appropriate. truncation value 13 essential so that the infohnation base of cutsets does not exclude high consefuence, low frequency failures.

C.5.2 Generic,and Plant Specific Failure Data The application of generic data for the quantification of the system fault tree requires the selection of a conservative failure estimate to fit the specific failure mode. However, in some instances, the failure data are not available to model fault events defined at a more " microscopic" level, i.e., at the piecepart level. Due to the nature of this problem, it was nec-essary to model the failure at a higher level. For example, a motor failure has 3 failure modes under an "0R" gate, namely short circuit, overload, or a broken shaft. The generic data base provides only data for the "all modes" failure. This necessitates removing the individual failure modes from the fault model with the result that assessment is at the higher level. This limitation requires th'e fault tree to be " pruned" and some cutsets are elim-inated in the process. Although there have been few catastrophic failures of the diesel genera-tor system or its pieceparts at the Trojan nuclear pisnt, plant-specific data for dominant failures were applied in the quantification process to provide more realistic results in terms of the information base of generated cutsets. The incorporation of plant-specific data in fault tree quantification results in the identification of a realistic mix of " active" and " passive" components that constitute the prioritized list of critical components for focusing upon problem areas. C.S.3 Failure Modes and Effects Analysis When failure modes and effects analysis (FMEA) is performed in addition to fault tree analysis to identify possible failures, it extends the informa-tion base to include incipient failures. However, the FMEA does not indicate how serious a potential incipient failure is, i.e., how many additional fail-ures are required for diesel generator failure. This limits the ability to prioritize components for reliability improvement. l M

                                                                                                                                                                                                                                  , , , , . _.-.,,.--_--,,y}}