ML20236C893

Transcript of 890301 Briefing on Status of Performance Indicator Development in Rockville,MD.Pp.1-60.Viewgraphs Encl

ML20236C893
Person / Time
Issue date:	03/01/1989
From:	NRC COMMISSION (OCM)
To:
References
REF-10CFR9.7 NUDOCS 8903220368
Download: ML20236C893 (89)

Text

{{#Wiki_filter:. s w z i + c 'h .i . UNITED STATES OF AMERICA 4 NUCLEARTREGULATOR-Y COMMIS SION i j $k6 BRIEFING ON STATUS OF PERFORMANCE INDICATOR DEVELOPMENT i i Location: ROCKVILLE, MARYLAND OO MARCH 1, 1989 ) PagGS 60 PAGES 1 NEALR.GROSSANDC0.,INC. COURT REPORTERS AND TRANSCRIBERS 1323 Rhode Island Avenue, Northwest Washington, D.C. 20005 l (202) 234-4433 i s,.' + 8903220368 890301 ph,7 PDC w_-_

n i b DISCLAIMER This is an unofficial transcript of a meeting of the United States Nuclear Regulatory Commission held on March 1, 1989 in the Commission's office at One White Flint North, Rockville, Maryland. The meeting was open to public attendance and observation. This transcript has not been reviewed, corrected or edited, and it may contain inaccuracies. l The transcript is intended solely for general g. informational purposes. As provided by 10 CFR 9.103, it is not part of the formal or informal record of decision of the matters discussed. Expressions of opinion in this transcript do not necessarily reflect final determination ] i or beliefs. No pleading or other paper may be filed with j the Commission in any proceeding as the result of, or 1 addressed to, any statement or argument contained herein, I except as the Commission may authorize. f NEAL R. GROSS court REpoaTER$ AND TRANSCRIBERS 1323 rho 0E ISLAND AVENUE, N.W. (202) 234-4433 WASHINGTON, D.C. 20005 (202) 232-6600

f{( 'I ~. 5 na p x. l' iy:: l._c 3 1 1 hf.e

1 UlIITED-STATES OF AMERICA L

'2, . NUCLEAR REGULATORY COMMISSIO!I 1. ..t p L 3-4 (BRIEFI!!G ON STATUS-OF PERFORMANCE-5 IliDIC ATOR LDEVELOPMEliT 6 7 . P U B L I C : M E E T I N'G t' 3 '9' liuclear-Regulatory Commission . 10 One' White Flint IIor th 11 Rockville, Maryland. 12 13 Uednesday, MarchL1. 1939 14. 15: The Conmission met i'n open session.-pursuant ~ to 16, notice, at 9:30 a. m.,- the Honorable LANDO U.

ZECH, JR.,

17' Chairman of the-Commission, presiding.

j 18 i ' 19 COMMISSIONERS PRESENT: 20 LAfiDO U.

ZECH, JR.. Chairman of the Commission 1

- i 21 THOMAS M. ROBERTS, Member of the' Commission q 22 KElitiETH M. CARR, Member of the Commission 23 KElitTETH C. ROGERS, Member of the Commission I 24 JAMES R. CURTISS, Member of the. Commission - 25 (202)234-4433 MEAL R. GROSS & COMPANY, IliC. (202)232-660 1323 RHODE ISLAl!D AVENUE, ti U., WASHINGTON, D.C. 20005 j 1_ _ _ _. _ :__________._______ _ ___________________________.___ _ ________________________________________ _ _ _ _ _ _ _. _ __ _ i _ __ _ _ J

c .fl

• r.

. 9; c x + 1. 6' .t. ESTAFFfAllD PRESEliTERSl SEATED AT!THE COMMISSIOti TABLE: kI- -2 ' SAMUEL J'. CHILK, Secretary-l- l \\' ~ i ~3 JOSEPH SCIIITO, Assistanti General Counsel l l -4 JAMEScTAYLOR,:DeputyEExecutive Director, Operations 5 'ED JORDAff, Director, AEOD-t 6' H A R K ITI L L I A M.S,.AEOD 7 TOM ?!OVAK, AEOD 8 _ C ARL JOHMSO!!, RES 9' ~ 10 '11 e 12 ~~ 13-14 15. .16 17 18 } -19 20 21 a 22 23 ) I l 24 4 i J s-1 25 I l (202)234-4433 tiEAL R. GROSS & COMPAliY, Ilic. (202)232-660 j 1 1323 RHODE ISLAl!D AVE!IUE, !!.W., WASHIIIGTOli, D.C. 20005

+. y 31 -.3 l' .P_ R q C_.E_ E_ D Lll G S~

.v.

2 ( 9 i 3 5 a'. m.' ) 3 CHAIRMAli ZECH:- . Good. morning,- ladies and L '4( gentlemen. 5~ Today's meeting is ;for. the IIRC' staf f to brief-n C 6 the' Commission concerning the status of development'of the m

7 new. performance indicators.

'8 ' The ' Commission 's Perf ormance-Indicator Program y 9L is: an important component-of IIRC's overall' ' capabill.ty. to 10 monitor-licensee safety perf ormance. at nuclear; power - plants. The program, using seven: approved-indi c a t c r s,-- 12 provides an additional + view of. operational.performan.ce'and ( 13 enhances our: ability to recognize areas. of pcor or ~ 9 14 ' declining safety performance.

15 This program'provides~the 11RC'a tool to be used i

16 in conjunction ' with ' other tools,, such as the' results of: 17 inspections. and our systematic assess: tent -of licensee 18 performance program,. for providing an 'inpurl i to 11RC 19 management and will help us make decisions-regarding the 20 need to adjust plant-specific regulatory programs. 21 Last year, the Commission approved a six-month 22 trial period for development of cause codes and corrective ] s 23 acti ons from licensee event reports, as potential j j 24 performance indicators. i i %./_. 25 The Commission also approved proceeding with the (202)234-4433 IIEAL R. GROSS & COMPAliY, I!!C. (202)232-660 1323 -RHODE ISLA!!D AVEt!UE, M.W', WASHIliGToli, D.C. 20005 w

s '*I 4 e l 1 validation of safety system function trend indicators and I h-- 2 further developmental efforts focusing-on indicators of L l .3 . maintenance perf ormance, g 4 Today's briefing will up-date the Commission 5 concerning the status of the staff's actions in each of l 6 these significant areas related to the development of the 7 new performance indicator program. 8 I understand that copies of the slides are l-9 available at the encrance to the room, and I should point 10 'out, -too, there is no scheduled vote today, on this 11 subject. j 12 Do any of my fellow Commissioners have any 13 comments to make before we begin? 14 (IIo response.) I' 15 If not, Mr. Taylor, you may proceed. 16 MR. TAYLOR: Thank you, Mr. Chairman. As you 17 indicated, the staff has been using performance indicators l 18 as an important tool in its safety eversight 1 19 responsibilities, and today the staff will be talking, as 20 you indicated, about some of the efforts to improve those 21 that we currently have, by adding potentially improvements 22 or a new indicator particularly in regard to maintenance. 23 AEOD is the lead office on this, and I have at l l l 24-the ta' ale with me, starting from the left, my left, Mark S~.- 25 Milliams, Mr. Ed Jordan, Tom tiovak, and Carl Johnson from (202)234-4433 ITEAL R. GROSS & COMPANY, IITC. (202)232-660 1323 RHODE ISLAliD AVEITUE, fi. W., WASHIliGTOli, D.C. 20005

erWFT79psyt v - - 0 3 :. -q ;; -. f L*- a 5' il 9 i% g- .1: .the.O'tficelcf.Resea'rch,: the*three previous-' gentlemen being-pt M 2 .;fromEAEOD.- w 3 Ed will takeithe' lead'and start the briefing, 74: CHAIRMAN ZECH: Okay. i m q Thank'you very=much. Mr.. Chairman,. -5 MPc-JORDAN: c 6-wel' are ' pleased to: be ; here - todayr'and bring you what : we 7[ belie've i's ~ g o o d news regarding the development' of 8 performance indicators. I-9 We.have been; I think, rather successful.in the'

j i

10 past four' months. We1have made some break-throughs that.. l 11 we would like to share with.you. 12 We' previously briefed the Commission in April,._ I 13 Lon mai'ntenance indicators, safety system availability and. 1 4 b --14 cause codes ' and, as you indicated, we - proposed a. t r i a l-l-

1 15' program that we have been conducting since thathtire.

l l'6 ' The Commission ~ accelerated the : development cf 17 .maintenan'ce ' indicators,. and you requested further 18l validation of the cause codes through your SRM in June'of-19: 1988. 20 We met with the Commission in October, on the ~21 maintenance rulemaking, in conjunction with the of fice of

v 22 Research and the Office of Reactor Regulation, and a part 23

. o 'f that discussion was preliminary results o f. the -24 maintenance performance indicator development. q s.' > 25 At that time, we had done site visits at 23 (202)234-4433 NEAL-R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND-AVENUE, M.W., WASHINGTON, D.C. 20005

i, 6 8 K 1 l 1- ' units, ~~and $we ' came ' to :the Comm.ission with = the. view.. tiha t [. '2-the NPRDS" component failure rate data showed the greatest-p l-l 3 potential, out'Lof.the inf ormation ' we had, for maintenance-I 4 effectiveness measures, and we recommended some revisions' H ii 5 to.the draft rule at that time, to add.that as a feature. ) l '6 We f ound - also, a t ' t h a t'

time, that-process 7

indicators, such as the estio of preven t!ive ' main tenance-8 to-total corrective.activ cacklog, although it..was usefu1~ .9-for utility purposes, didn't really correlate well with-L ~10' cur other data and was not useful for NRC purposes at that - 11. time. L. ' 12 Since October, the effort has continued very 13 ' consistent, I believe, with the Commission directive to-14' further develop and validate the maintenance, cause code 15; .and safety system availability. 16 The participants -in this program have included-17 'the Office of Research,. assistance from liRR and from the .I 18L regional offices-and we 've had very strong centractor 1 1 19 support - f rom Brookhaven, Pacific liorthwest, Oak Ridge, l 4 20: Science Application International, and the AEOD staff 4 21 itself. .I 22 Two of the areas, the maintenance and cause 23

codes, have interacted.

We've found data that was-24-beneficial mutually between those two areas, through the L 25-close coordination of those activities. (202)334-4433 tiEAL R. GROSS & COMPAtIY, IliC. (202)232-660 1323 RHODE ISLAND AVEliUE, U.W., WASHINGTON, D.C. 20005

t o e e 1-

• Te' vel. looked at a great deal of data during this 1

2 time frane, and it's sunrarized in the three reports that i 3 have been submitted to you in the last several weeks. 4 Ue've also continued discussions with IIIPO 5 during' this time frame, and - we ' ve maintained a. join t - i 6 swareness of activities with ItiP O. We've maintained j 7 coordination with the foreign performance indicator 3 development through the IAEA. I, personally, participated 9 in a workshop in September of this past year, in Vienna, 10 and there is another workshop, a follow-on, scheduled for l 11 September of this year.

And, so, the international

) 12 community is aware of our actions,.and we're quite aware i 13 of theirs. 14 As.I indicated in opening, I believe that we're 15 further along today than I e::pe c te d four months agc. I 16 think my only personal concern is that we adequately 1 1" represent the trenendous amount of work that has been done j l 18 by the staff and the contractors, in this discussion. 19 Mark and Carl are going to be making the 20 presentation and, if we have detailed questions, we have ] 21 additional staff that will be able to get down to the very 22 lowest level of detail, 23 Ue also ha've representatives here from Oak i 24 Ridge, who were instrumental in developing the cause code u-25 database. And, with that, Mark, I'd like to turn it'over (202)234-4433 IIE AL R. GROSS & COMI-AITY, I1!C. (202)232-660 1323 RHODE ISLA!!D AVE!!UE, fi. U., UASHIIIGTOti, D.C. 20005

7 1 to you. g l H- .2 MR. WILLIAMS: Thank you, Ed. 3 CHAIRMAti ZECH: Thank you very much. iou may 4 proceed. 5 MR. WILLIAMS: Thank you. L 6 I'd like to cover the areas in the order of' 1 l '7 maintenance and then cause codes, and then the candidate 8 indicator of safety system function. trends. s 9 As Ed explained, there are two reports that have 10 been submitted to you, one February 6th and the 6ther

i 11 February 7th, AEOD S804B, and then a report on the crial r

i 12 program on cause codes. And, in addition, we sent down a y 13 paper recently on safety system function trends. r i f 14 (Slide) An overview of our accomplishments in 1 15 the area of maintenance is, first of

all, we have

.j 1 4 j 16 developed what we believe is a reasonable indicator, a 17 useful indicator, of maintenance effectiveness 18 performance, for use by the industry and, hopefully, by f 19 the staff in the future. This is based on tiPRDS component j i 20 failure information. 21 Our future work will be to develop simplified 1 I 22 methods to acquire this indicator. Right now, it's fairly 23 labor-intensive to obtain the indicator from the !!PRDS { 24 system. 3.- 25 On the next slide -- (slide) -- in the area of l 1 (202)234-4433 NEAL R. GROSS & COMPAliY, IIIC. (202)232-660 4 f 1323 RHODE ISLAllD AVENUE, ii.W., WASHINGToti, D.C. 20005 1

.. "  ;,x '^ . c a- -9

v..

.l-- _c a us e L e ode st, ;;right now' we have; : developed- ;the' LER L2-inf ormation: forJeause codes ' to be' a.perf ormance indicator. 4 .t'.' '3 ~ W e,'Iv e, conducted a: ~ rather ' thorough : analysis,. w h i c h " i's H -1 ~4 documented in..the; Oak Ridge. report,.and.we've: resolved the

h ;

5: questions, tofourLsatisfaction, that were; initially raised > d 6 .on cause co es. 7-W e' also 'f ound ' tha t-the cause? " codes revealed 1 'S problems 'in -' th'e. areas of maintenance that' correlated"well" j ..q 9 with our'propos'ed maintenance. indicator. For future work, .] 4 s .10. we' hope to. develop a performance indicator display methoa R. f 11: for :cause -codes, and - we will present various options' in-12 May, at the senior manager meeting, and then we'll hope to (n' 13-have that indicator, with Commission approval, implemented 1

j lb into the program, i

l 11 5 We.will continue our trends and. patterns-LL 16 analysis in. the area o f. cause' codes -and corrective u M 17' actions. We found some areas'where, for certain sets of' ] 3 ,a 18 cause codes, a knowledge of the corrective actions, n, y i 19 provided additional interesting information that we'd'like i 1 1 20 'to pursue. 21 (Slide) In the area of safety system function 7( -22

trends, we've completed plant visits and the model 23 development with actual plant data.

We were encouraged ~2 4 that we could see differences between the plants in the si 25 magnitude and the variation of this indicator, and Carl -(202)234-4433 NEAL R. GROSS &, COMP AliY, IIIC. (202)232-660 n 1323 RHODE ISLAUD AVENUE, N.W., WASHINGTON, D.C. 20005

y 7---- ---- .F i .10 _ -;4. ' p y,.. y w-s . g, \\ p_ -l Jr -1

Johnson,.-from Research,Jwill cover this11atier, r.

~ We ' re' also. recommending-that1 we ' continue: that ~ s

2 -

b 1 program with :more J data,. bef ore we can 'make.a -decision onL 1 L .:3 ~ i (, (: L4- .the. indicator.* f. l -5 -(Slide) F i r s t. - o'f a'l l', on 'm a i n t e n.a n c e a 6 indicators,. we thought it woul.d : be. beneficial to briefly. l l,

7 --

-covercsome of the? earlier work from-the October commissi'on L-that.we issued; and. this: >J L 8 briefing.and the-December report !q 9 was AEOD:.S804A,.where Ed mentioned -we made-- the ' plant ,l a 10 ' visits. l 11-In that work, we found that there: were really ] 12' two k'inds-o f.' indicators that we wanted to describe,L i 13 process ' indica tors and e f f ectiveness. indicators. Thef 1 t '.1'4 . process indicators were indicators such as corrective 1 15 maintenance-b a c k l'o g, or.the ratio-o f-preventive l ~ 16' maintenance to total' maintenance'at the plant. -l 17 We found the. use of ' these indicators enhanced d 3 18. the management control of. the" maintenance process. Ua 19 -found some plants that made real gains.in the use'of these .20 indic a t.o r s. - q 21 We found that plant-specific flexibility in the i l 22-implementation of the detailed definitions of these 23 indicators was very important to the success of the t l 24 program. We also found that during our statistical k,a 25 correlations, we were unable to correlate these process (202)234-4433 NEAL R. GROSS & COMPAtiY, ItJC. (202)232-660 1323 RHODE ISLATID. AVENUE, N.W., WASHIliGTON, D.C. 20005 j = -

m, s ..r. g-- m - i l:lIh- ,.N f %{'h i A i,1 :'c

7

,( [ g'g '1, m, N t b fi-vindicators,to output!orLresults indicators efi maintenance.: l ~ .2' The maintenance < ef f ectiveness -indicators = were-1 i -1 3 -items such Las f ailure t rate, mean time return to service,- ] I t '4 ' or= the number and" dura tion' -'of equipment out-of-servicenin;- i 5 the plant. These-were - based. on - actual compo'nent failure 6- 'd'a t a, and we found these to be. the most promising. ~7 indicators to pursue. '8-The itPRDS had the best-data,-tha best structure 1 -9 in the de finition. and the consistency of the. data, te '10 proceed.to develop _ indicators. So, as'a result, that was 11 our li n e' ' o f ' pursuit .and, consistent 'with: o u r-- I 12' recommendations, the statement of consideration of a, 13 proposed rule e mpha si::ed and ' encouraged the use of MPRDS. .j 14 for~ maintenance-indicator monitoring. d .15 Our second. report-was a follow-on of the first' j q 16 report. 1 i I l 17 CHAIRMAtt ZECH: Before you get into that, on 13 ~your review of !!PRDS data, you were obviously inpressed i 19 with the capability it has to support an effectiveness j 20 indicator. How could you correlate the data to, and did 21 you try to correlate the data to actual effectiveness 22 performance at the plants? ~23 MR. WILLIAMS: Uhat we did was, we extracted 24 data from the liPRDS on the mean time to return certain cq) ' 25-equipment to service and the equipment out-of-service (202)234-4433 tiEAL R. GROSS & COMPATIY, Ilic. (202)232-660 t 1323 RHODE ISLAtID AVEMUE, M.W., WASHINGTO!!, D.C. 20005

,; _ s. s >e r D. 1 indicators, potential indicators, and then we correlated } W l L._ 2 those - with the equipment forced outage rates and ' the l-l 3 forced outage rate overall of the plant, plus.the other 4 indicators, and we conducted a program where we looked for I' 5 some leading behavior of - these indicators, quarter-by-1 6-quarter, as to whether-they led the forced outage rate. j 7 What we found was that all the analysis was 8 mixed. There weren't any clear-cut findings, but the'best 9 consistency, the alignment of the data, was between the 10 actual component failure

data, the effectiveness 11.

indicators, and the forced outage rates, equipment forced H 12 outage rates, and the like. P 13 CHAIRMAi! ZECH: There was a correlation there? l 14 MR. WILLIAMS: In some cases, there was a 15 correlation, and it was consistent across a few plants, ' we weren't 16 for some indicators. We weren't overly 17 convinced, based on the strength of the statistics, that 18 there was a correlation there, but it was promising, and I ( 19 think that was our result. We said it was promising. 20 CHAIRMA!! ZECH: Promising, and that's where we 21 stand. I'm sorry -- that was the 22 MR. JORDAN: tio 23 December report. We have further work to report on today ) 24 I l 25 CHAIRMAli ZECH: Fine. I (202)234-4433 tiEAL R. GROSS & COMPAliY, INC.-(202)232-660 1323 RHODE ISLAUD AVEMUE, N.W., WASHINGTON, D.C. 20005 l

s,n.

.c -~ x ._,3 - k. !a,. ._u m, >..4 s W. m (g g 33 ., ~ " l' ~MR.; JORDAN: -~ so, things are batter today.

l p.,

2 MR-. WILLIAMS: Yes. 1 '3' MR. JORDAU: You're talking-about the iDecember i "4 , report? 'S .MR. WILLIAMS: Yes. ~ i 6 CHAIRMAli ZECH: Okay. Go ahead, then. 'Tell u's 7 about what happened today here, when you can'get to-it. 8 MR. WILLIAMS: The new report' tha t - -we ' ve 9 submitted has worked. since the. December-repor t, AEOD ~10 'S804B'. .I t provides. inf ormation regarding -some of the 1 4 l'1 inspections conducted by the staf f and-the :need fot i 12 component trending. It also documents our development-and .13 . validation of a maintenance indicator -- and this is the { 14. --new work, and we'll get into this in great detail. 15 LI t provides a. methodology-to extract the -i '16 indicator from the 11PRDS 'and display the' -indica tor such I 17-that it can be used by licensees today. It ~ shows the 1 1 18. benefits in monitoring,.and it also continues the j 19 development in our program. i 20 (Slide) The next slide gets right into the work I ia 21. we've done since the December report, and this is the 22 concept of the indicator that we've developed. l 23 There's quite a bit on here. First, on the X-i 24

axis, what we have is the failures extracted for one 25

system, tiow this slide, the line on this slide, is really I

(202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 13 23 RHODE ISLA!!D' AVENUE, N.W., WASHINGTON, D.C. 20005 a

..j 14 for l 1 the cumulative number of failures in one system i that are L_ 2

example, the main steam system at a plant 3

extracted from the NPRDS. The ifPRDS has three different l l 4 kinds of failures -- catastrophic or immediate failures. 5 and degraded failures, and incipients. He only use the 6 first two clear-cut failures to map this failure history l l 7 within this system. The Y-axis, as I said, is the l l 3 cum.ulative number of failures. 9 The indicator really scans this data and locks 10 for an increase in the failure rate. The arrow on the 11 curve indicates an up-turn in the failure rate, and our 12 indicator is a mathematical model that checks for this 13 increase in the failure rate and, as you see, conceptually i and this is real plant data on an 14 on this picture 15 actual system -- when that fsilure rate turns up, w e-16 obtained an indicating mark. 1~ COMMISSIONER CARR: That's a pretty -- there's 18 another up-turn there that didn't indicate anything. You 19 know, that's pretty -- it's not a sharp drop in the curve. 20 MR. JORDAtir It's not sharp. What was done was, 21 an empirical determination was made from looking at lots 22 of plant data, to see clustering of failures.

And, so, 23 the way I look at it is that we developed a methodology 24 that is clearly empirical in nature, but identifies

'~ 25 clustering as then a count, so when there are multiple (202)234-4433 IIEAL R. GROSS & COMPAITY, INC. (202)232-660 1323 RHGDE ISLAND AVETIUE, N.W., WASHINGTON, D.C. 20005 I

.s 15 1 failures that occur in'a short period.of time, you get a 2 count that then is registered in this system. 3 COMMISSIO!!ER CARR: So, it's a monthly look l 4 then. 5 MR. JORDAti: That's correct. 6 MR. WILLIAMS: That's' correct. 7 MR. JORD A!! : Accumulated on a monthly basis, and 8 we looked at considering just the slope

itself, and 9

decided that it was less confusing, in fact, to pick out 10 the rate of change of slope, or a bump in the curve, and 11 we have some interesting correlations. Why don't you 12 proceed. 13 COMMISSIOtiER ROGER'S: Before you move on, what 14 is the solid curve? That's a fitted curve to those data? 15 MR. WILLIAMS: Yes. 16 COMMISSIOITER ROGERS: And how did you fit that? l 17 MR. WILLIAMS: The mathematical model that l 18 generates the ' indicator is actually based on taking the l 19 average of the latest three months, the count of the 20' failures for the latest three months, and getting the 21 average of that -- or the latest two months -- and then 22 subtracting the average of the prior three months. 23 So, the indicator itself is based on the 24 arithmetic of subtracting the points. 25 MR. JORDAli: The question was how do you fit (202)234-4433 liEAL R. GROSS & COMPAtiY, INC. (202)232-660 1323 RHODE ISLATID AVEtiUE, M.W., WASHIt!GTON, D.C. 20005

N w A. ]

1. X p

• ^

e. ,I ,\\\\. E1' that= curve?: = '2 MR.. WILLIAMS: The curve was just a smooth-fit 3' curve, with'a standard. personal. computer-based package. '4 ' COMMISSIONER RO'GERS: What kind of a function 5 ..do e s : i t fit to it, to those points? i 6' MR. JORDAN: You want' Bob to answer that?- ,l 7 MR. WILLIAMS: Well, maybe so. l 8' MR.' JORDAN: Bob Dennig. .l 4 '9 CHAIRMA!! ' ZECH : Step to the microphone, pleasu, 1 i 1 10' and identify yourself for.the Reporter. l i 11 'MR. Delit!IG : My name is Bob Dennig, AEOD. J J 1.2 That curve is.-- actually, it's on there to eid !j j 13 the eye ' in following the points'. The kind of functice 'I j 14' that's fit to draw tha t ' curve is not inherent in tb-- 1 15' machinery for generating the indicatcr. l 16 .As Mark indicated, it's based on a concept very 17 similar to looking for trends in existing performance 18 indicators, looking at the average of the two' latest 19-quarters, if you will, versus the three previous quarters. 20 There's some kind of -- there's smoothing that's done in a 1 i 21 very local way. That particular curve, for purposes of j 22 our discussion, is actually visual information or excess 23 information. The curve, itself, there doesn't play a role 24' in the indicator, itself. ~ 25 COMMISSIONER ROGERS: Okay. (202)234-4433 t!EAL R. GROSS & COMPAtiY, Ilic. (202)232-660 1323.RHODE ISLAND AVENUE, II. W., WASHINGTON, D.C. 20005 y

yy _.- y ,e 2 i i J:)y. J, ,j i -i, e;

-t,_

6 :- F,, Ni. ,J [, _~! g 1

r, l;

1 iCHAIRMAli LZECH:

- All. righ t. -

..Le t 's proceed. Go'- a L.. L i j hi[' '2 ahead',DCommis'sion'er.. [- ' 3. COMMISSIO!!ER "C ARR: But'. if ?:I understand this ln 4' right, if the' guy's consistently bad,'it. won't? trigger: an' 15 indicator. I-6' 'MR WILLIAMS: .A. consistently high' failure.ra,te. 7 -for this system, that is absolutely' correct,- it will ' not-- 8 trigger.that indicator. 4 MR..JORDAli: But I would hasten to add.that'from I l s 10 the data _we looked-at, which was-substantial, consistently: J j i 11 bad -that. is,.a rather' steep slope - .has, frequently,- U 1 ~ L 12 greater numbers of clusterings than consis tent 1y' good. J 13 does, Land so -- L. 2 14 COMMISSIO!!ER CARR: It might trigger something 15 else, but -- ' 16 MR. JORDAN: Right. l l i 17 COMMISSIOffER. CARR: -- ' this system wouldn ' t be. ] 18 triggered unless the guy'had a sudden' jump. 1 e_ 19-MR. WILLIAMS: Well., this was designed not to be ] ~ l 20 triggered on a constantly ' high failure rate because-of 21 reporting variations across the plants, and when we -- tha 22 way we got into this indicator, in fact, is by laying out, 23 for all the boiling water reactors, this history for five l 24 systems, for 23 or 28 plants, and that's in Appendix A of-l 25 the report. (202)234-4433 fiEAL R. GROSS & COMPATIY, Itic. (202)232-660 1323 RHODE ISLA1TD' AVENUE, II. W., WASHINGTOli, D.C. 20005 o-I

5 5

g 1

CHAIRMAi! ZECH: All right. Let's proceed. 2 MR. WILLIAMS: Given this indicator concept for 3 one system, we applied it to five systems. We applied it j 4 to five systems, and the major components in the five 1 5 systems that we called " outage dominating equipment". The 6 equipment that we selected, was selected in order to 7 obtain consistency across the plants, for reporting and 8 off-set known reporting variations. (slide) -- has the indicator 9 The next slide l 10 for each of the systems -- the reactor recirculation, main l l 11

steam, feedwater, neutron monitoring and control rod 12 drive.

This is a plot of the indicating marks that were .13 shown on the prior page. 14 Basically, one could add up the indications at l l 15 the bottom of the page for each month, for each quarter, 16 and look at the overall performance of this particular 17

plant, for this indicator of maintenance effectiveness.

18 That is revealed on the next slide -- (slide)-- 19 and what we see is that the trend for this particular 20 plant that we have looked at is overall improving.

Again, l

21 this is just a best-fit line. Maybe it should turn up at 22 the end, but it was just a resquare's best-fit line on the 23 indications that we saw on the prior page. So, in this 24 manner, one can obtain a trend from this indicator. l l"~~ 25 On the next slide -- (slide) -- we discuss some (202)234-4433 NEAL R. GROSS & COMPAt!Y, Ilic. (202)232-660 1323 RHODE ISLAIID AVENUE, N.W., W A SHIIIGTOff, D.C. 20005

m .~ .s. 7-o, ;. .s l:- r. . some i of; the important-aspectsiof '1 l of 1the; key attributes, 2-this. indicator. The first,- as Commissioner Carr '3 mentioned, is tlia'tL the indicator 'is really normalized to '4. 'the plant,. itself,- to the plant 's ; reporting.: practices. m 15' It's'not sensitive lto its total'. magnitude. ~ 26 .An indicator 1could be' consistently high;but,~.if-. 7 it' didn t have an increase,. 'it ' would L not. trigger' --' ' a 8 plant could.be. consistently.high, _but it wouldnlt trigger 1 9-the indicator if it didn't have an' increase.in the failure 10-rate within~a system. 11= This was done to o f f-s et - known ' reporting 12. variations. In fact, we have ' some - systems tha t had - J '13 consistently high f ailure rates, and tha t' 3 plan t has L 14 improving. trends in this indicator. 15' The reporting variations are'due to thingsLlike '16 resource allocation, conservatism in the - de fini'tions - 'i 17-

applied to failures which = varies slightly. across- : the' l

-j 18 -- . plants, i 19 Some other key aspects of this indicator is that. 20 it's generated on a system basis. There's a 1~arge l- . 21 variation in the number of components within a system. We 22 must remain within a system and plot-it on a monthly l .o

23 basis, to obtain an accurate indicator.

u 24 The. systems and components selected for this t \\ l' i F" ' 25 particular indicator, were selected for consistency across l 1, l. ,(202)234 4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 l ~ 1323 RHODE ISLAND : AVENUE, N.W., WASHINGTON, D.C. 20005 i

r------------- e' 20 1 the plants, 'so that we could hope for some success.in a 2 validation program for this indicator. So, as a result, 8 3 we limited our selection of systems and components, to 4 ensure that. 5 A plant using an indicator like this would not 6 be limited to this select set of systems and components. 7 They could use it much more widely,'and it certainly can 8 be used on safety systems and other systems. Again,. l we 9 limited ourselves primarily for the validation. (slide) -- we discuss the methods i 10 On slide 11 11 that we used to validate this particular indicator, and 12 there are three facets to it. The first thing that we did 13 is, we looked at the root cause associated with the 1 14 indicator. 15 We took about 500 failure records and about 40 16 indications, and we looked at the root cause as described 17 in the written narrative statement of the failure by the 18 NPRDS coordinator at the plant, and we analyzed that and 19 binned them according to cause code definitions. 20 The next thing that we did was, we correlated it 21 with other data, data that we obtained from LERs on 22 maintenance problems. We also looked back at our a 23 technical studies on major components that we studied that 24 are the same components that are constituting this i. 25 indicator. And, in addition to that, for all the boilers, '~~ (202)234-4433 tiEAL R. GROSS & COMPANY, It!C. (202)232-660 1323 RHODE ISLAND AVEtIUE, 17. W., WASHIIIGTOff, D.C. 20005

f ci; J - -' q.s } [ f,l. ;,1 ' L'" t 1 2 b-V -.y J analysis,. a' retrospective look' at i

11 we-did a retrospective y +

.l C ~ 2 their1 histories for a few years, for all these'. systems,-at ) [ 0 3-all' theiBURs. I 1 4 lOn : the ' root' cause area, on the next. slide-- 5' (slide)' -- whati we found is'that tue dominating cause for j 6 the indicator,, the increase in the. failure rate for the i

)

tr that. constituted that indicator, was' m'aintenance. l 7 fallures ~ -8 associated-The definition 'of " maintenance" here is a 9' .f airly Ls tandard one. It includes -up-ke ep, repair,:- 10 surveillance. testing, and errors of commission and-j

11-'

. omission. .i 12 .The causes, again, were obtained f rom - reading' J /;, 13 the' narratives. The causes were not 100 percent 1 N, -14 . maintenance, and it's a good point to make, that these L 15 Lfailures are due to a number of reasons. Some of them are 16 "due to design, some' of them due to random equipment 17 -failure but, overwhelmingly, we found that the main cause 18 was maintenatec errors. 19 For the correlation with other data, on slide 13 20 -- (slide) -- AEOD had performed studies in the past. 21~ - couple of years, on main steam isolation valves, feedwater 22 reg valves and their bypass valves, and main feed pumps, 23 curbine and motor driven, and the operators for the valves 24' I mentioned. 'Y. 25-These studies included plant visits, they '(202)234-4433 tiEAL R. GROSS & COMPAliY, IITC. (202)232-660 1323 RHODE ISLA!!D AVENUE, N.W., WASHINGTO!!, D.C. 20005 .= =

?) f,,f N jy w wa .o li i n c l u'd e'd ' a ' d e t a il'ed : ' an alys i s. o f' the ' reasons f or-the 7 dz component.. failures, and we ' looked at' the-'manuf acturer of-a_ J3, tfhe valve,..the model,- the size,; the environment-- . ; y 'f I 4L internal and ' external -- and :what we found from these' . that the primary reason'. f or-5

studies, overwhelmingly, was 6

differences in -- f ailure rates across. the' plants, vas the 7' ' differences in maintenance practices. ~across the plants; y 8 and those results are documented in-the studies'. '9 CHAIRMAN ZECH: They'are'what, again? q 10' .MR. WILLIAMS: Those resul'ts are documented in .j

l i

11' our studies. 12 CHAIRMAN'ZECH: Thank you. [13 MR. WILLIAMS: On the' LER' correlation,, u' sing what we did is, we used the-trial program-14. cause-codes, 15 cause codes and extracted the maintenance problems from 16 those LERs', and'then correlated.that with the maintenance. 17 indicator-that we extracted from a totally different 18 database, the Nuclear. Plant Reliability Data System. The '19-plants correlated were the same ' plan ts, for the same 20 . period of time. 21 The next slide -- (slide) -- 22 COMMISSIONER CARR: Let me go back to the 23 previous slide for just a minute. When you say the root 24 cause analysis, that's 77 percent of the failures reported t 25' to the NPRDS, right? (202)234-4433 NEAL R. GROSS & CONPANY, INC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005 l } t

. l' % ' l .D' 11 ~ - .MR. WILLIAMS: Of 6 the.- 500 that we analyzed.for 29 3 i c2- - thi s l s e t,.f y e's, :' s i r. , J. ~ '. COMMISSIONER C ARR: Are attributable to (some 3 J4

mai'ntenance-relatdd' subject.

5 MR. WILLIAMS: 'Yes, sir. 3 6' COMMISSIO!!ER CARR: Okay., '7-COMMISSIONER ROBERTS: Well,.on that same' slide. 8' .you said the ' definition of " maintenance" you used is the - h-standard o n e ', and I ' unders tand ' " des ign",. " random" and. J 10 '" unknown". . Why isn't the "wearout" subsumed under-j i 11" " maintenance",? '12 MR'. WILLIAMS: Well', it's'in our construction of 13 , the fdefinit ions. I In general, we'didn't consider wearout, t.- 14 in this'one, as maintenance. l u '15 MR. JORDAN: 'I-think I.would agree with you that 16 .,j 1.7 COMMISSIONER ROBERTS: The purpose of 18 maintenance-failure is from wearing'out? 19: HR.. JORDAN: That's correct. 20 COMMISSIONER CARR: Failure to replace. l l 21 MR. JORDAti: And I think -- you make an argument 22 that by seeing that most of the failures, the root cause 23 was traceable to maintenance, then we were able to make .)

24'

.the-jump and say, well, we're going to assume that all of 25 then are due to maintenance. And, so, the wearout is then l b_ (202)234-4433 NEAL R. GROSS & COMPANY, I!!C. (202)232-660 ) l[ - 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON., D.C. 20005 j D C _ __ o u

.c g 1

,, -, -.a..

.t 24-m 1 j - li in; t h'e ' 's e t,- as weJ use them. So, we were trying' to K - ~ '~ 2 demonstrate.that'this particular indicator of failures of 4. .3 these isysten s! and. components were principally.due' to 4 maintenance. 5 CHAIRMAli. ZECH: 'S), you.' re saying wearout here '6 ' -was due to maintenance?- Fourteen percent of :the 7 'maintenan'ce problems resulted in. wearout,.is that what i 8 you're saying? q 9. MR. - JORDAti: In binning the root cause,- the-10-staff binned wearout as a separate item but, in our use of-11 this' data,:we're saying that since the majority of these '12-failuresLare due to maintenance, then there was'npt.a'need 13 for:usito bin, for the purposes of analysis, the failures. -14 Ue-simply. assumed -- 15 COMMI S SICITER CARR: You didn't separate chose i 16 out. 17 MR. JORDAli: That's correct. We simply assured i 18 then, based on this distribution, that the failures were 19 due to maintenance. 20 COMMISSIONER CARR: So, you were really 91 1 21 percent or so correct. i 1 22 .MR. JORDA!!: I think so. '23-CotiMIS SIOllER CARR: But I am trying to figure 24 out -- everything in - the !!PRDS database, you're saying, 91 1 h a,,- 25 percent of that stuff is due to lousy maintenance, and i 7 n (202)234-4433 ttEAL R. GROSS & COMPAt!Y, ItJC. (202)232-660 1323 RHODE ISLA11D AVE 11UE, M.U., UASHINGTO!!, D.C. 20005 = _ _ _ _ _ _ - _ _ J

y. 7,_ 's; l =.e.. 25-1 1 .1 1-that'-- you know,'I have a hard time with.that. 1 ~ V I' 2 14R. JORDAll: Of-the sample that we used'-- 3 MR. WILLIAMS:

Well, this is the indicator.

L i I4 This ~is.only the-failures that made up those increases in l 5 the failure rate,'40 of them -- 6-Col 4MISSIONER CARR: Oh, okay. across those boilers. So, 7 MR. WILLIAMS: 8 this is the constitution of'the indicator. 9- 'COMMISSIOliER CARR: This is the trigger. 10 MR. WILLIAMS: This.is'the. trigger. 11 COM14ISSIONER CARR: Ninety-one percent of those 12 triggering items were -- 13 MR. WILLIAMS: Yes, sir. 14 COMMISSIONER CARR: Okay. 15' MR. WILLIAMS: We did a root cause analysis only 16 of'the -- '17 COMMISSIONER CARR: I was having trouble there 18 seeing that database. .19' MR. WILLIAliS: -- only of the indicator. Right. 20 COMMISSIONER CARR: Okay. I'm all right. 21 CHAIRMAN ZECH: All right. Let's continue. 22 MR. WILLIAMS: The LER correlation with cause s 23 codes is on slide 14.. (Slide) We portrayed it like this we can relay or describe the 24 so we can see that the 4 V 25 .effect that it had on us. (202) 234-4433 !!EAL R. GROSS & COMPANY, INC. (202)232-6o0 r 1323 RHODE ISLA!!D AVENUE, N.W., WASHINGTON, D.C. 20005 _ _ =

l 4 4 26 I l I 1 Basically, what we found was the maintenance gave us the I 2 problems that we found from LERs, moved l 3 same picture as that we got from the liPRDS. tiow, let me 4 explain the graph. 5 On the abscissa of the graph is the plant 'name. I 6 The boilers -- this is actually ordered in increasing 7 number of the candidate maintenance effectiveness 8 indicator hits, so the plants at the bottom end at the 9 left side would.have the lowest number, and the plants at 10 the top end would have the highest number. 11 The maintenance indicator is the crosses,- and 12 that is, again, a best. fit line through those. For those 13 same plants -- in other words, if we draw a vertical line L the aste:.isk 14 for the plant directly above the cross 15 directly above the cross is the same plant, for the same 16 period of time. 17 We extracted from the LER databasa, the 18 maintenance-related problems that we got from LERs. And 19 we extracted that, plotted uhat on the right side, the 20 IIPRDS indicator on the left side, and got a correlation. 21 Actually, we cross-correlated them, and the correlation l 22 was statistically significant, and this graph shows you, 23 essentially, how they move in the same direction and by l 24 plant, and we felt this was very reinforcing to both the 25 maintenance effectiveness indicator and to the LER-based (202)234-4433 IIEAL R. GROSS & COMPAliY, IITC. (202)232-660 1323 RHODE ISLATID AVENUE, fi.W., UA SHItiGTOli, D.C. 20005

T g 1 1 information that we extracted from the NPRDS. i 2 On the next slide (slide) 3 MR. JORDAti: It might be appropriate to say that 4 when we first looked at this data, we did jump up and down 5 because it was nice to see these two databases that are / 6 quite independent of one another, giving such good i 7 correlation, so we are restraining ourselves today, a bit. .{ I 8 CHAIRMAN ZECH: All right. Let's continue. 9 COMMISSIONER ROGERS: Don't be too restrained, 10 we may miss the point, l i 11 CHAIRMAti ZECH: I understand your point. Let's l 1 1 12 continue. (slide) -- in the 13 MR. WILLIAMS: On slide 15 i 14 plant analysis, what we did is a retrospective look at the l 15 plant hi. stories. This is, again, in Appendices A and B of l 16 the report. 1 17 We did this because there was a logical 18 relationship between the outage dominating equipment that l 19 we had selacted and the forced outage equipment. We did i 20 not expect to find a large correlation because of the 21 redundancy within these systems, and also the magnitude of 22 the failure is not catastrophic, so it would not 23 necessarily take the system down; but we did look through 24 all the boiling water reactors. We looked at about 3,000 l 25 failures and about 200 forced outages. l l (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)23d-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

3 23 l' Basica11yr the results of this work were that in 2 10'of 'the 28 plants,-at least once, we-found the indicator t 3 . occur in advance of'a forced outage. tio w, the time period ^ 4 in advance of a forced outage varied, and this is all i 5 described in the report'. 6 During the analysis, the engineers found some 7 very, very interesting things. They found very logical l S relationships within a plant. They could see an indicator 9 occur once within a plant, twice within a plant. They 10 would predict it would occur a third time, and it would j i 11 occur. They saw some consistency within plant data that 12.- was very, very, very encouraging. 13 The results of this we don't believe are 14 statistically s ign:. f ic a n t, but they are what we e::pe c ':e d 15-to see, and the data analysis makes sense to us and it has 16 convinced us that the indicator is a practical and useful 17 one. 18 COMMISSIONER CARR: And that lead time, as I i 19 renember, was from zero to si>: months. 20 MR. WILLIAMS: It varied quite a bit, sir. 21 One of the things we then did is, we said, then 22 we should see what this indicator tells us about the 23. boiling water reactor industry, and that's on the ne::t 24

slide, i

( 25 (Slide) We looked at the indicator overtime for (202)234-4433 IIEAL R. GROSS & CO!1PA!!Y, I!!C. (202)232-660 9999 esaae vetAtta Avatme. M u. uramvwavan a_e_ 2cann

e a i I 29 l' all the plants. We had talked about this indicator before g

L 2

we- ' plotted it out. We expected that it. would improve l 3 becabse, in general, we believe the industry is improving, 4 and all of our indicators show general improvement. 5 This is a newly developed indicator and, lo and. 6 behold, it shows improvement in the industry over thes'e 7 years. 8 Within this trend, although the overall industry 9 is improving, I think it is important to note that not all l 10 plants' are improving. The majority of the plants were i 11 improving. There were some plants that had a trend upward 12 but, overall, on the average, ~the overall industry trend j 1 13 is improving. 14 COMMISSIOtIER ROGERS: Excuse me -- what group of 15 reactors was this? Was this all BWRs, or just those -- 16 MR. WILLIAMS: BWRs in commercial operation 1 *7 before 1985. 13 COMMISS10 TIER ROGERS: How many were there in 19 that -- was that 23? 20 MR. WILLIAMS: I want to say 23, but -- 21 MR. ' JORD All: I think that's right, 23. 22 MR. WILLIAMS: And the reason, Commissioner { 23 Rogers, we selected those is because the reactors don't 4 24 put information into the Nuclear Plant Reliability Data i 25 System until they are commercial, and we wanted to have a (202)234-4433 NEAL R. GROSS & COMPAtIY, IIIC. (012)232-660 1323 RHODE ISLATID AVEIJUE, 11. W., WASHIt!GTOli, D.C. 20005

m. -,-

.....'{., . c _., -. q L'.,4 t ,e. a 30- -: p : 1 consistent ~ set. So, we.took:all those commercial at.the g -2~

beginning, Sand.used:their' data.

i 3'

COMMISSIOliER. ROGERS: 'All right. ] i \\ ~4 CHAIRMA!! ZECH: 'All right. . Fine. Let's-l 5 continue. 6. MR. JORDAti: I believe that the conservative 7 trend,- or ' the improving. trend, is conservative 'since. 8 utilities have done a better' job of reporting to the tiFRDS' l 9 since 1985, that that level of reporting has improved and, J10 s' o, i-f one were assuming that the greater -number of 11 reports would result in increased number of hits, then.i t 12 would make this. line look even more conservative in 13 showing i.trpr1ovement. 14 CHAIRMAN ZECH: That'.s a very good. point. All 15 right. Let's' continue, m [ 16 MR. WILLIAMS: Slide 17 is the conclusions. -17 regarding the maintenance effectiveness considerations. .1 18 won't touch on all of them. We do believe,.on item number 19 2 here, that the method-is suitable for use in the 20 regulatory guide by licensees. We think it's a key to 21 improving self-assessment in the area of maintenance, that 22' things like this be used, the results of maintenance be 'I

123, monitored.

We haven't found it being done very widely, i 24 'and we would really encourage it, and we think that this j H-25 'is a reasonable example of the kind of thing that people L -(202)234-4433 NEAL R. GROSS & COMPA!!Y, INC. (202)232-660 1323 RHODE ISLAND AVENUE, 11. W., WASHIliGTOli, D.C. 20005

q. v i;< .y 31 1 1 1 could.do. l .4 1 2: The method.is suitable f'o r E c t h e r designs. We i found that we could minimize the effect of industry-wide 4 Variations in'tiPRDS reporting, by use of selected data set-ji 5-within the !!PRDS. i . 6 One'.of the other major. points-that will:1.ead-us- -7 into the next' slide is that the correlation ~'with LER-data j i 8 was very. encouraging, and we believe that we can also, j l 9' ex trac t. 'some maintenance-rela ted inf ormation from the LER 'f 10-database. 11-MR., - JORDAti: You.did skip over.one item that I 12 think' is important, and that is that.this. was labor : q ( 13: intensive. It was not easy. to get: the data out of the j 14 11PRDS ' system, and so to apply it on a wider'use, we will 15 have to come up with more automated ways of getting the o 16 data out.'and presenting it. 17 COMMISSIONER CARR: What. was the resource i l 18 impact? l 19 MR. JORDati: Sir? 26 COMMISSIO!!ER CARR: What was the resource impact-21 on doing this particular part of a job? NR. WILLIAMS: I think this particular report, 22 4 23 including the analysis and the data, was roughly 1.5 to.2 r, 24 FTE over a four-month period. d 25 .COMMISSIO!!ER CARR: So, we could put a couple of l (202)234-4433'!!EAL R. GROSS & COMPA!IY, IIIC. (202)232-660 19 9 9 smaae T ettua Avetme. w_m_ mA as TwaPats. rd _ c _ 9600%

a. '4 d ,j j. i o l: / 1' guy s : p'ermanen tly ' on 'i t,. they could 7 hack 'it, for the. p i' F '2 lindustry? l .) -3 MR.-UILLIAMS: Yes. 4' MR'. JORDAN: On'a utility basis, it's something - i 5 .they should be doing anyway, 'but for us to extract it and' ') l- .i 6 .try to put it on a n'ational basis was difficult. l L

. 7 COMMISSIONER CARR:

I predict if'we start..using l ~ 8 it,.they'll start-using it. \\: 19 MR. JORDAN: That may well be. .f '10 CHAIRMAN' ZECH: All right. Let's continue, 'l '11 please. l 12 MR. MIL'LIAMS: This led us into the discussiqn this is slide 18. (Slide) .This is the 13, of cause codes 14 .second area we wanted to cover. 'First of all, the trial 15 program was. completed, again, in response to the SRl! from 16-June 24th. The staff initiated a' trial program to assass 17 the. benefits in the cost of cause codes. He also exanined 1 P, other issues relating'to the accuracy of the information 19. in the LERs and to the subjectivity of the coding of the ~ 20 inf ormation :into our databases. 21 In this program, Oak Ridge !!a tional Laboratory 22 ' supplied the major effort and major support. One of our 23' AEOD people was in it full-time working with them. They 24 used engineering judgment in the analysis of the LER data. y L.: the codes that 25 They'didn't just accept the cause codes (202)234-4433 tiEAL R. GROSS & COMPAt!Y, INC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

________y .;: >y 33' a > 1 'the licensee used or theEcause codes on.the: frent of the p.e(( ' 2: LER. ThAy used. independent-judgment, and read the entire

. 3

. event' description,'and'then assigned multiple cause codes ll-4 to e'a'ch'of these' events. 5 A major.f eature of. this. program is that the 6- . actual new' coding-was restricted to six months, but.by use 7 of:different' search strategies, with the. sequence encoding -8: search system' database, which is' our primary LER database, m 9 we were.. able to map the existing'LER data for about two 10 and - a' half ' years before that, into the new.cause code 11 areas,.with.a degree of confidence that. allowed'us to use ~ 12-that data-for.this cause code analysis. i 13 So, rather than having only six months of cause -14 code data, we have a few years of cause code. d a t a ', - and. 15 that was used for the trend analysis in the report that i 16: you have. 17 The corrective action data was only six months 18 of new coding effort because we had not previously coded 19 . corrective. actions into the sequence encoding search 10 system. 21 On the next page is the cause codes.-- and this 22 is just a background from which we can continue to discuss 23-it.- This has the causal areas that we used. Let me say 24 at the outset that the cause codes here are different than i '/= 25 the cause codes we've seen before in maintenance. The ~ (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

y, . - = - - e 3 o ( 3 4., ^ ,.. c : l 12 ' primary. reason is l -tha t :we- ~ s tar ted - this -l program - at, an'- t, 2 earlier time',:- and_ Lthe cause codes were' laid out? in ' : S ECY y> j .3 88-103--from May,of 1988. 'We stuck ~with that st.ructure in ' i 4' developing the,cause code program. .5 The= causal: areas we used'are maintenance,'which-6. we subdivided'into two categories - .I'll discuss that'in" 7. - a - second~ --' adminis tra tive control error, other1 personnel y '8

errors, licensed operator error, and ' random equipment, l

9 design / fabrication and construction. l

1 10-

'We subdivided maintenance into two' areas, 11 really, because we had.no other unknown category 11n r this' ] 12

exercise, and -many o'f the

• events that they coded. were

- 13 -clearlyf associated with maintenance, 29 percent. Some-C .have some examples -- were not clearly 14 events. -- and.we maybe an increase in 15 associated with maintenance i.--- 16-downstream temperature of a check valve that caused a 17 penetration to exceed some design value. Although they H 1 1 L 13 couldn't be clearly associated with maintenance, overall, 1 19 .the. judgment of the evaluator was that a good, aggressive L 20 maintenance program could have caught that in a timely 1 21 manner. 22 So, those items are coded in Maintenance-2, l 23 which constituted 11 percent of the total industry average 'l 24 cause code. data. j 1 25 COMMISSIONER CARR: Let me clear that up. When l' (202)234-4433 NEAL R. GROSS & cot 1PAIJY, INC. (202)232-660 1323 RHODE ISLATID AVENUE, N.W., WASHINGTON, D.C, 20005 l ^ 1 1________ -J

5[ 3s fl. I read the definitions in.their study, I couldn'U figur6 k 1 .l 2" out' how'to code somethingLHaintenance-2. It looked to me. i: q .3; like you decidedcHaintenance-1_was okay and the other one .4-was all-other, 5-MR. WILLIAMS: Well, we have some examples that .] 6 .might be the easiest way to:show this'. t i 7 COMMISSIOtiER C ARP. : I' read tho' e. That's what' '] s 8-confused'me~a little' bit. 1. 9 (Laughter.)- '10 MR. WILLIAMS: Ue tried to come up with a'very .I 11 simple explanation of those, Commissioner, and'I.think the 12 -.we had - the same trouble in reading *the definitions in-13 the report. .i 1 L 14 Oak ': Ridge is

here, and they do have isome

'l a 15 examples of what became Maintenance-2. The only way I i 16 could'come to grips with it after reading.several events j l 17 that were categorized as Maintenance-2 tyself, was - that, i 18 overall, one could see a strong maintenance implication in i 19 the - event, that the conditions, the temperatures, or 20 environmental temperatures, or the leaks in the' valves, or 21 whatever it was, were allowed to progress to a point _where 22 something else happened, which may have -a design -23 implication. l 24 On an event tha't I'm thinking of, a downstream c v. there was back-leakage from the 25; temperature of a valve (202)234-4433 !!EAL R. GROSS & COMPAliY, Itic. (202)232-660 F c an. amma macum c omamm. ma weawowsmau a a eaaaa-

7 --o-- -1 ,j

p. y, ;.,

i.

O.'

'g .._t, a. 36. 71x ' steam' generator finL the ' aux..f eed system. The temperatures.- s 2 were-increasing,-bu't the;only reason that.the'back-leakage Lj .i k 3l _ was' being driven' was because of some leaky valves, .1 i '4 Jambodied upon leaks from some' valves. Those were' allowed i 5 to get worse. They put some gum sealant into those, and 1 6-the. leaks continued, and the'back-leakage continued. l] 7 'The fix was to increase the.DP across the check a valve, which was' : really more of a design fix, but the-9. . problem ' had a. very ' strong maintenance tone to it. That 10' kind of. thing got put in' Maintenance-2. It was difficult' -j I '11. 0 1 o '~ 12 COMMISSIOi!ER CARR: Well, that one I would have 1 W 13 thrown into-Maintenance-l', but. the one that was in a~ t. 14 ' Maintenance.2. t h a t-bothered me was the invertor 19. transformer-shorts'to ground for an unknown reason -,you l 16J 'know?' If = a transformer shorts, it's pretty tough to blame 17 it on maintenance, but. I. canL clearly understand the and 'I-just assumed they threw } 18 Maintenance-1 preblem,. 19' everything else in all-other -- j 20 MR. JORDAN: Maintenance-2. as being an other 21- ' bin is not too far off. ~ 22 - COMMISSIGtIER CARR: Yes, okay. 'Well, that's 23 enough on'that. 24 MR. JORD Ati: I wouldn't argue. r~7 U 25 CHAIRMAli ZECH: All right. Let's proceed. l (202)234-4433 NEAL R.-GROSS & COMPAtiY, INC. (202)232-660 1323 RHODE ISLA!!D AVE!IUE, !!. W., WA S HI!!GTON, D.C. 20005 _ _ _ _ - = -

'k. .f. el-b\\ t' 3 i, - 37 1 ,2 y .m V 1' MR*, WILLIAMS: This is Ethe industry 1 average- }lJi 2~ cause-code distribution for the six. months of data, about .o 3- '1230 LERs. 'What we can see is.that roughly-30 percent are i 4 clearly.due to maintenance, with 11 percent in this second j i -5. category that we ve discussed, which -is a f airly. large. .6-portion. '7 - .One item on thi's-slide ~.is that across different .y . { t 8 . designs, the vendor designs, we did not find a substantial . { 4 9 variation in'the causes. We found them fairly consistent-10 'across all the'. designs, j 11 Another point that people would lika to note.is 1 4 9 12

that across the NRC regions, the causes were.found.to be

- f 13-th'e.same, in'judgl.ng the LERs. So -- ( i MR. JORDAN: They were remarkably consistent, 14 .15' cut by regions -- remarkably consistent'. . f 16 MR. WILLIAMS: We.found consistency there. 1 ' l 17 An interesting fact on the next slide is---- la COMMISSIOllER CARR: Well, Admin. Control there, 19 is that procedures?- 20 MR. WILLIAliS: Communications procedures., 21 COMMISSIONER CARR: Okay. (slide)-- 22 MR. WILLIAMS: On the next slide 23 the corrective action, this is the industry average 24' corrective actions. What we can see is, it does help us 1 i M 25 focus our views on corrective actions. Over half are (202) 234-4433 !!EAL R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

q .t:p,, y-y fl y .c 1 - ~~

]

p, yV.., < . 3 8 !- .n t 3 s -1? . procedure changes' in ! training in ' the " industry 4 today; 'and-8,; 0 ,t 3

2'

_ equipment replacement-repairois.'a'little,under 20' percent', L L 3 So, thi'stgives;us.a background.from which"we can- ) '41 ' unders tand!' the distribution-of causes 'and corrective. l 5,

actions, j

(slide) - -:.a l s o, during'. the t'riall 6 - Slide 21 7. program, we looked at some issues associated wi th - the - 8~ accuracy of the information in the LER, whether the truth .3 1 9' was'1 in.. the ' LER, essentially, as'. the event occurred,.and-i l 10 also the: accuracy of~our coding of:the information in the-i 4 i 11 LER. j o ,12 ' On the-first'one, we.' compared LERs to one of-'our j 9-13' best - estimations of the. truth,. which is the indepe6 dent; ] i 14 findings of 'an < augmented. inspec tion team, or an IIT 15 finding. We looked ~a t - about.19 of - these cases, and. : wha t 1A .we found i's in about 80 percent of' the cases, there was ,} 17 .very good'. agreement-between the causes assigned by the IIT-18 and the'causes in the LER, and this gave'.us a sense that '19-the accuracy of the information in the LER was 'f airly q 20 good. 21 We also checked the enforcement history'of about j 1 22 licensees, from about 1985 up to-date, and we found' 1 23 none cited for inaccurate reporting of the causes in LERs. 24 We did find some that were cited for not reporting at all. i W-25 We also looked at SALP assessments. We looked i !(202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005 _ - _ _ _ _= _ _ _ _ - = _ - - _ _ _.. _ _ _ _ _ _ _ _ _ - _ - _ _ _ - _ _ _ _ _ - -

, ;; y V '39

I p

1: Into.SALP's toLdetermine'if people were being mentioned for by 2 (inaccurate-. reporting in -the SALPs, 'and we did not-finda '3 ' major problem there. Overall, in'the SALPs, many programs 4 are'very, very good.~ Just about all of them we. looked'at 5 had the' capability. to 'identif y the root. cause :. and ' put" it 6 ~in thelLERs. 7 AEOD. also conducted LER quality reviews. For. 8 about three

years, we. h a~d -a program that very U

9 systematically looked at' the LERs and evaluated. them-

10 quantitatively gave them ' grades in. each of several

'll areas. One of the areas'was root cause. ~ .} 12' 'We found.an overall improving ' trend. in the. ; LER I 13: ' quality ~ and.in the root causes and, as a result - that-i s 14- 'was a continued improvement trend, so. we stopped that; i 15 program. We found,. generally, all the plants that we were 16 l'ooking at for a second and third time, were. improving in i ~ i. 1 the quality of the LER. We-think the program had a lot to 17: 18.. do with it~because we would mail our evaluation back to '19 .the plant and the people who write.the LERs got it. 20 So, there has 'been an improvement in the LER' ( 18J 21-quality, especially since the June, '84 -- January of 1984 '22 . rule, and'since the implementation at that point in time, 23 the quality of information has gone up. 7 24 We-turned to look at the accuracy of the coding b.. ( 25 of the information in LERs. One of the ways we did this (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232'-660 '1323-RHODE ISLAND AVENUE, N.W., WASHINGTON, .D.C. 20005

t' I 40 r j ' ' 1, i s ', weL looked,at ~.theE codes extracted by. one:.o f, o u r 12: . con trac tors. -- Idaho N.ationalf 1 En.gine ering-Labora tory- ' ~~ 3 3 extracted. essentially what the licensee'said the cause o'f d I 4L 'the" went was. i 5 Oak Ridge used : independent ' judgment during the ,i 6 trial; program. so we compared these' two. We ' f ound. the'- 7.- agreemen't rate ' and it ' was dif ficult because we hAd-8 '- slightly different definitions,.but the' agreement rate was-9 about to-73 percent, or ;in that neighborhood -- it 's in 10 lthe' report, i 11 We also looked at the. inf ormation' in,the' S ALPS,.- ~ 1' 2 to see.whether the regions were coming to the. same 1 13 conclusions that the oak' Ridge. judgmentwas, regarding.an 1 14 overall plant,-and we found agreementthere. I 15 The only problem we found' across-- 16 consistently, across some of the groups, was this 17 personnel error. Sometimes there would be a sense of 18 personnel error and it wasn't listed as a cause, but that 19 wasn't a major problem. 20 The other thing we ' looked at is the overall 21 controls on the program for coding and subjectivity. Oak 22 Ridge has a very rigid quality assurance program for all 23 of our information that we code into the database. It's a i 24 formal program, and they have an error rate that they { 25 report regularly to us. .That program has very good (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)23{660 l

41 1 control of subjectivity, and the accuracy of the coding is e,. 2 excellent. ) 1 3 In addition to that, we had an independent check 4 of the SCSS coding, by one of our contractors when we were 5 evaluating the system just recently last year, and we had 6 people in AEOD and people of the contractor independently 7 code events and compare them to the' coded information and 8 the sequence encoding search system matrix, and we had j 9 very good agreement there. 10 So, we're satisfied that the coding efforts are 11

okay, and that the information in the LER is fairly 12 accurate and is good enough to usel 4

13 tiow, the next slide discusses the benefits. 14 Before we get into the next slide, I wanted to mention 15 that the report does discuss the validation of the cause 16 codes. We did find agreement between the cause code 17 identification of plants and the existing performance 18 indicator identification of plants, and the cause codes ) 19 use a larger body of information than the existing i 20 indicators. The cause codes use all of the LERs. 21 The number of LERs has gone down slightly, but 22 there's still roughly 2500 or so a year because the events 23 have decreased and general improvement, but it's a large, L... 24 a robust database from which we can extract the h l 25 information. (202)234-4433 tiEAL R. GROSS & COMPA11Y, Itic. (202)232-660 1323 RHODE ISLAtiD AVEtIUE, 11. W., WASHIliGTOli, D.C. 20005 . 1

y-u a u < y, l' 'M .42' '1-We also compared the cause codes with,the SALPs,. 4 2= andfwe~got good agreement lthere, as far as validating the 1 1 .] p 3 use of those cause codes. On-slide 22'-- ( slide ).,--' there were two. kinds ' 5 -of benefits we found from the cause codes ~, one was as 6 performance monitor. - We looked ~ at a' benchmark as being 7: 'the senior management ' list, and. it's described in the 8 report from January of '87 to June of '88, and-we found-9' that ' the hit' and the miss rate on the identification of '101 plants by cause codes was :just about as good as ~ the i 11' indicators. i l 12. There'were some plants that were identified by 13 cause codes that were dif f erenb - than those identified by l 14 the indicators, that we: felt. added to the identification i 15 of plants by indicators. i 16 The second value of the cause codes are that 17 they are programmatic in nature, and they allow us.to do.a-18 trending diagnosis so that we can see actual changes in 1 l9-operator error and we can trend these and give the 1 20 information out to people, and it's good supplemental l 21 information to help them understand without too much work, 22 what's changing in the events at the plant, a 23 We did do an outlier' analysis. The results of 24 that are all documented in the report. The next t _. 25 COMMISSIONER CARR: I was certainly encouraged I (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1323-RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005 -{ f:

a 7- ~-~ 'i .p, }. - 'a-43 a y ,j i 1-bie c a u's e i t looks ~ 1' i k e 'lt's really. an ' independent - l 1 2 indicator,. and it turns up something that nothing else 3-turns up -- 4-MR. WILLIAMS: 'Yes. and what you've already l 5 COMMISSIOliER CARR: 6 got a subjective feeling for, you may get.. confirmed by 7 this. '8 'MR. WILLIAMS: That's right. i 1 (slide) ' is :ene of the. 1 9 The next slide 10 interesting ~ f eatures that we haven't fully explored, but: y 11- ,we'd like to get into a little bit more. This shows the t 12 line-up of the corrective actions.and the-cause codes for; 1 13 one' area. The report would show f or.. maintenance or. 14 ' administrative ' controls, how each of the causes has a .15 distribution of corrective actions associated with it. 16 For the licensed operator error here, procedure 17 change is used as the corrective action 27 percent of the 18-time. What we found in the exploratory program, the trial if we look at the corrective-19

program, was that the 20 actions for a given cause for different ~ sets of plants--

21 inliers v e r.s u s outliers, if you will we can see J 22 differences in the corrective actions that are used, and 23. it's just another tool that we can use to analyze event l 24: causes and corrective actions, and what seems to be b 25 effective and'what doesn't, and what the patterns are. (202)234-4433 !!EAL R. GROSS & COMPAtIY, Itic. (202)232-660 1323 RHODE ISLATID AVENUE, 11. W., WASHIl1GTOli, D.C. 20005

,t fg T}f .-~ 7 T 'y' 7' y g4 s. 1-We haven' t gotten into the. work very much yet. I"~

i.. '.

'2 Ue ' intend.to pursue. - tha t-l area. And, with that,' that i i 3 ' brings.us-to our-future plans i l 4' .COMMISSIOtiER CARR: Well, that tended to be, to 1 4 5 me when I' read it, a management indicator, really, more.

)

6 than,anything else. 7 MR. WILLIAMS: There were some features that~ .l 1 i they looked at plants within a-8 they could do that 9 -Otility, and they -- 1 i 10 COMMISSIOliER CARR: And it was kind of' l 11 interesting,. the ' correlation between the goods' and the .12 bads and how they approached their corrective actions. 1 .13 MR. ' JORDAti: Yes. There are patterns 'in. the i L--- 14

corrective. actions for groups of plants that need. a lot 1

15 more work on our part. I think there is something there 1 i 16. that will be beneficial to us. l i 17 CHAIRMAN ZECH: All right. Let's proceed. 18 MR. WILLIAMS: Our future plans are contingent 5 19 on approval'of SECY 89-046, which is the cause code paper Li .1 20 that we submitted. We need to develop a new display 1 l t. 21_ format for the cause codes. We do have a number of l l 22-candidate display methods we're looking at. They will all -2 3 - have different efficiencies and different people, like j 24 different displays. 4 ? ~ 25 What we were hoping to do was to send some (202)234-4433'NEAL R. GROSS & COMPANY, INC. (202)232-660 i 1323 RHODE ISLAND AVE!!UE, N.W., WASHItiGTON, D.C. 20005 i . ~ _

-45 1 1 different displays to the management meeting, next senior )j ) 2 management meeting coming up, and have people come to a 1 3 consensus on what they thought they liked, and then we 4 would'go ahead and pursue that display for the next 5 performance indicator report implementation, contingent on L 6 approval. 7 We would also begin corrective action coding for 8 the 1989 LERs. We think that we'd like to code corrective 9 actions because we feel it's an area we will get some 10 benefit out of. 11 That brings us to the conclusion of the 12 maintenance indicator and the cause codes. Carl Johnson 13 and the Office of Research has provided primary support 14 for the continued development of the safety syster 15 function trends indicator, and Carl is ready to proceed. 16 CHAIRMA!I ZECH: You may proceed. Thank you. j 17 MR. JOHtISOtt: Thank you. 18 The third indicator that we analyzed is safety ) J 19 system function trends. This is an indicator of the i 20 unavailability of selected risk significant systems. ] l ) 21 Unavailability we mean to be the fraction of time that a l 22 system is not operational, or it could be looked at as the I 23 probability that a stand-by safety system will not start 24 up when called upon. (~ 25 This candidate indicator is a potential (202)234-4433 NEAL R. GROSS & COMPAtTY, IIIC. (202)232-660 1323 RHODE ISLAllD AVEtIUE, II.W., WASHINGTO!!, D.C. 20005 o

y.. L., [. d 46' [ -l'- replacement f or - the existing indicator safety system 2 failures. This indicator is calculated from two kinds of 3 data, one is the duration that trains of safety system are 4 taken out-of-service for maintenance and 'the second kind ( 0 5 of data is the number of train. level failures. 6 tiext slide, please. (Slide) In response to the 7 Commission meeting last

April, we performed a

6 8 retrospective analysis. We collected two or three years 9 of data from five units at three sites. Brookhaven 10 tiational. Laboratory and Science Applications, Inc. I 11 analyzed the train level availability data in plant 1 cgs. 12 These logs say when a train of a safety system l 13 was removed from service for maintenance and when it was 14 restored to service -- and this is one of the pieces of and the other is tha ' 15 data we use in the indicator 16 number of failures. The indicator includes an estimate-- 17 when a failure is found, it includes an estimate for how 18 long the failure existed before it was discovered, se the 19 sum of the two times then contribute to the 20 unavailability, and this turned out to be -- did a system 21 fail, or train fail, rather, turned out to be one of the 22 problems in using the historical data. 23 Sometimes the logs did not clarify whether a 24 train was taken out to perform maintenance because the l 1 L_ 25 train had failed already, or whether it was taken out for (202)234-4433 !!EAL R. GROSS & cot 4PAtIY, It!C. (202)232-660 l 1323 RHODE ISLAND AVEtiUE, ii.W., WASHItiGTOli, D.C. 20005 L_-__-______-__-___-__-.

}Q:c/f Q f, q-! 47f W.,, '1L maintenance before it failed. .l o. art 3 7: f

2-

.The results offthis work were promising, but,not 3 conclusive. - They are promising in the sense that ~ the 4 i '4 'indibatorEcan -- we found~that the' indicator can evaluate 'S the magnitude' and trend of this u' availability versus

m

.;~ 6 time, and we found.it is feasible to.; compare the indicator 7-versus a benchmark 'of expected unavailability, to help

8 flag indications of higher than expected unavailability-of

i ) x e 9. a-safety syste.r. 1 10 The results.were not conclusive, in that for two _l } (11~ of the 'five units' that we analyzed,. we did not get. the. 1 !q 121 indications:that wrt. expected to see. They didn'.t turn cut ) i .,) .13 the'way we : thought' they would. One reason may-bs that we l 1 i . '14 may not have a complete data set because of.the historical j 15' nature of the data.

\\

\\ I .' 16 itext slide, please. (Slide) We propose tc 17 continuei this validation with prospective data instead of -l 4 i 18 ' historical data, from a few volunteer units. Where plants i 19 .'have good data on one or two systems, we would like to try 20 out that data, to evaluate this performance indicator. 21 Also, we plan to examine existing data sources, 22 such as IIP R D S, the INPO safety system performance l i 23-indicator, and LERs prior to 1984, which include train l, Q., 6 24 level failures, and we expect that this work will complete M1 u 25 the validation of this indicator and will evaluate the i l [, (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 L 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005 l b li___ __

6 i' 48 1 extent to which existing databases can indicate ] 2 unavailability of_important safety systems. 3 This completes _ our discussion on safety system 1 4 function trends. l 5 CHAIRMAti ZECH: Thank you very much. 6 MR. TAYLOR: That concludes our presentation. 7 CHAIRMAN ZECH: All right. Questions from my 8 fellow Commissioners. Commissioner Roberts? 9 COMMISSIOtiER ROBERTS: I have no questions. I '.m i 10 encouraged to hear that you are so encouraged that this 11 seems to be giving you results you can use. 12 MR. JORDAN: Yes, sir. 13 CHAIRMAN ZECH: Commissioner Carr? 14 COMMISSIONER CARR: I agree. I think it's a 15 good piece of work. I'm a little uncomfortable with the 16 safety system function trends because if the guy's doing 17 preventive maintenance, it's still going to show up in 18 your trend as one of the indicators, but it lends itself 19 to misinterpretation, I'm afraid, so you're going to have 20 to be careful with that one. 21 MR. J O Ht!S O !!: The indicator is dominated by 22 failures. About 75 percent of the unavailability is due 23 to failures in the sample that we -- sample plants we 24 looked at. And by including both halves, it allows us to 25 look at a balance of -- if somebody is perhaps doing (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

---

p 1.,,

-t 5 1_ - excessive n:ain tenance ' ratherJ than letting' things-run. 1 h. -l I ~ cot!M I S S I O!!E R CARR: The. fo t h e r L. t h i n g ~ I _' d'-

{

2 t ] 3.: 1 encourage !you. ;to 1do i s. t o -- I unders tand you ' re i L s 14 considering balance-of plant:inisome of thesa maintenance- { 5 . indicators, and I certain11 en courage.. you. to do. that, _ ({ h ~~ j i (6' continue that consideration. I' don't.know what_your. plans, '7 are inL that-area. q -8 CHAIRMAN'ZECH: Can.you tell,us, Mr. Jordan?- i c MR. JORDAN: The 'IIPRDS. database. doesn ' c extend l l 10 'very.;far into the balance-of-plant -- .11 - CHAIRMAI! ZECH: Does not? 12 MR. JORD Ati: Does not. Industry is considering ( [ 13 extension of 'that data and, so, our. ability would be 14-entirely dependant on whether or not industry does, in 15 fact,- extend that system. So, there is a lack of data 16 that one can use. 17 COMMIS SIO!!ER ' C ARR : .Can the. LERs come i n-on 18 balance-of-plant? ~l i 19 MR. JORDA!!: No, sir, not unless it caus'es a j '20' plant outage, a plant trip. j l 21 CHAIRMAti ZECH: Well, you're obviously focusing i j 22 on ' the safety systems, but you know how: the Commission I d 1 23 feels about balance-of plant, too. We all know that' .24 balance-of-plant does cause problems.and, so, I'm that's, q. O 25 I'm sure, Commissioner Carr's question. I would certainly L(202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1 s o s ' amaae vatina Avrmre n X MAgwTwavaw_ a_e-26068

,'r; s[,, ~ 1.L :. t ;- i '50 '

y p.e

1; s__.-

6: "w 2 COMMISSIONER-CARR: - W e l l,. you've already. 3' janalyzed, I thought,'about 50,p'ercent;of theirldown-time r 1 jm.. '4 . comes,out'of balance-of plant problems -- 1 7 5. MR. JORDAN: It'does.- ~ '6 COMMISSIONER CARR: s o,.. we ' re 0 overlooking' a 7 -big bunch of data here. 8 MR. JORDAN: That's correct. f 9 CH AIRM AN - ZECH :

Well, let's pursue 'that.

10 Commissioner. Rogers?

11..

COMMISSIONER ROGERS: Yes. I.think it 's ' an .12 excellent piece of work, and-I'm really-very.encouragedIby1 K 13 what y.ou've done,- and impressed by the: high quality -of the L__; j t l'4 ' effort so far,.and I really want to congratulate you :on 15' everything you've done. 16 What do you see-as the false-positive aspect of 17' this? What have you seen, in looking at-these indicators, 18 -:that ' would suggest that there really ought to have been a 19 . problem'but there wasn't one? -Have you been able to look d j 20' at-that.at all, that aspect of this? '21 MR. WILLIAMS: We have looked at it. Regarding j i 22 the false-positives, we would consider a false-positive 1 23 something where the indicator hits and it's unrelated to 24 maintenance. l 25 I think there is a section of the causes that ) (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 j U 1323 RHODE ISLAND AVENUE, N.W.,. WASHINGTON, D.C. 20005 l u= i

..u r y i V, s y 51 ? . y + N",r r ,4 . 1c arein'St Ldue E to ma'intenance,. but it 's 'a. sma11R s'ection ;on 61 4 V J2l this indicator.

Thisf particular indicator lands ~itself' si g-

=3~- well:to culling,out'some maintenance error. J 4' So, I think i t 's. 'a screening tool- ' that : we--- 1 5 .again,- we would screen. plants with-it', and screen U 6 perf ormance :with it, ' b u t ' o n c e' we got indications, 'we 9, 1: 17 really_ have to look' underneath and look ' at L the ' actual '8' failures,.what-they ' were, 'what the -corrective actions 9

were, and' do. some straight engineering work.

It's a i

10 screening tool.

' 11. It's very useful to systems' engineers in' plants. 12-It's-useful to'us because we can get it to a point where (> l'3 - it could provide - good screeni'n'g information in.a very-14. timely, efficient manner, if we continue to: work'on it. 15 That's the : primary area of false-positives, I

16 would

think, where those' areas were not-d u e' to 17 maintenance, the causes were.not as socia ted - with-

'18 maintenance. 19 COlitiISSIONER ROGERS : I suppose this question is. 2' O. also related to the extent to which there are conflicting-21 _ messages coming back from the set of indicators that don't 22 seem to be'in agreement with each other. Have you seen .23 instances of this?

24 MR. WILLIAliS:

We have looked at the plants for-Q 25. consistency, with this indicator. We looked at the plants i -(202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1323LRHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

n, c = 1 ll V ~ .52-e ,m -c [' 1 'th'a tf had 'a1 declining: perf brmance, ;in-theimaintienance F, [2 - ' indicator that. we -looked at f or the boilers : at: least, k.i- ~ n '3 because'tha.t's the only.cla'ss of plants'we've done. '4 We saw m i x e d. r e s u l t 's,. but.we - did see some 5 consistency with the mainten'ance indicator,-with.the other-t 6 -indicators. I.think we ; we re ' -- in looking at those 7: results, we were very encouraged. 4 8 We saw plants -- in. the' development of this 9: indicator,.one of the things we did was, if a plant had a-10 high: reporting of f ailures ~ and a.high failure rate, we 1 11 ' f ound _tha t t h a t. w a s - n'o t that revealing, in the IIPRD S. 12 failures, but the: increase was. '13T We f ound' plants that had that characteristic. -l 14 that were ~ improving 'on this indicator, and their other ~ ' 15' 't r end s. we re improving. We found plants that. were 16 . declining.and they.were declining in this area. 1 17 So, this: together with the other indicators as'a (18 -set, does seem.to help the picture. 'It do'es seem to add a .19 . piece of the picture, at least for that set of plants that 20. we looked at. 21 Now, our experience is limited. We'd like to go 22~ ahead, and we are trying to go ahead and do the analysis 23

for the PWRs, with one of our contractors in Idaho, to j

i 24' begin downloading the data and the labor-intensive job of l x.+ 25 laying out the data. So, we'll continue to look at that. l

l..

(202)234-4433 UEAL R. GROSS & COMPANY, IUC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON,.D.C. 20005 l _ = - o

\\ -*- t 53 i 1 MR. JORDAli: I think you're entirely right. If l 2 we expand the numbers of indicators we use, then we're o 1 3 going to get some additional false-positives out of it i l 4 and, so, we have to rely on looking behind it. If we only 5 use it to nominate plants for review and then we look l 6 behind it and see whether there is or is not a problem. 7 COMMISSIOtiER ROGERS: Right. Well, on the other 8 side of that question, were there any plants that had 9 serious problems that, in retrospect, looking back at your 10 indicators, none of the indicators suggested that they l 11 were getting into trouble? 12 MR. JORDAN: I don't think we've come to that 13 view yet, but I think it's highly likely that 14 COMMISSIONER ROGERS: I'd really be interested 15 to know the answer to that one. 16 MR. JORDAti: We've clearly seen a number of 17 anomalies, and the kind of anomalies that bother me are 18 two unit or multi-unit facilities that have common 1 19 management -- 20 COMMISSIOITER ROGERS: Yes. common practices, and where 21 MR. JORDAli: 1 22 they show differences on different indicators, that always i 23 bothers me, and I think it's one of the more sensitive 24 ways of seeing the limitations of the indicators, and that v 25 still occurs. (202)234-4433 NEAL R. GROSS & COMPAtIY, Itic. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

'), ) '? y Ai t, 1 MR. TAYLOR': -I think=if?we_ expanded the. number 2 . of ^ plants, ' that would be clearly a correlation-we 'd be 3 _looking-for -- how does this. stand up with what we believe \\ 4 to.be the rest of the performance? J ,5 MR. JORDAtl: But in 'looking, 'for ins tance, '_at 6 cause-codes with our other present methods-of 7 understanding plants, there's very' strong correlation of- =8 the outliers'of that, so that was the reinforcing side of 9 i t,._ bu t it ; does. bring along a few f alse-positive; plants 10 that show up for cause codes that, otherwise, we don't1 -11 really have a concern about and, in fact, still don't. ~ 12 COMMISSIONER CARR: .It was interesting-the-cause 13 codes-highlighted at one plant, that had an increasing-j s-. 14 ' S ALP. perf ormance, but it'd be interesting'to look at the-15 next SALP'on it. 16 MR. JORD Ali: Yes. It bears watching. 17 COMMIS SIONER ' ROGERS : Uhat is the additional l' 18 . cost to the licensees, to fully implement the kinds of-l putting the kinds of data that we need to really .{ 19-data l 'I .20 make this thing work,' into -- l 7 21 MR. JORDAN: For the maintenance performance 22 indicators, they put the data in already -- 23 COMMISSIONER ROGERS: Full in and out, so 24 there's nothing -- .g. U 25 MR. JORDAN: -- so there's no additional data, .j + l (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND AVENUE, M.W.,! WASHINGTON, D.C. 20005

a. --

',m. u: +

r

? +v, 7 '55 215 uhless'we expanded it to other'than safety-related. W... W 2'

COMMISSIONER CARR:

But we need.them.to'be.more 3 consistent than'they are','probably. m 4 MR. JORDAN: That would be helpful, but there is 5-useful data there now' We're really convinced ~ that 16' there's useful.' data now, so it's a matter of them applying-' -_ 7 it, inEtrying to me'asure their maintenance effectiveness,. 8

and ' the-scheme' that we-came up with' here is'not, by any 9

means, the only way of doing it.

This ' is.one we ' re 10 confident works, but it may be that if you had consistent' 11 reporting within a plant, just looking at the slope, the .f '12 rate of these-failures, would be a very strong signal, but' 1 13' we couldn.' t' use it across a number of plants. -Comparir.g 14-the slopes we.s not'useful to us. 15 MR. WILLIAMS: This particular indicator is very 16 helpful.in balance-of plant. We looked at.the feedwater 17 systems and other things in this particular exercise for 18 the boilers. For the plants to implement it, they could j l 19 use the failure information they send to NPRDS, but thev 1 I 20 have that information available for all of their systems. f a 21 The scope of NPRDS is not yet expanded to l condenser, rather, 22 include the turbine, EHC condensate I;i 23 some systems, but it shouldn't be very costly for them. l 1 24 COMMISSIONER ROGERS: Just one final question. ] ,a l 25 In interpreting any of these, or all of them together, (202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 i 1323 RHODE. ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005 j _J

r. &x 56 m 3,. 1, zthese performance indicators,-in searching for a potential-- F.C-- 2' . problem plant, how'sophisticatedcdoes the observer have to-3: be?' 'Do you think ~ that ' this is something -that requires-4 special training, to be able to spot? 'Obviously,-digging. ~ ~ 5 below. the : perf ormance indicators is another matter, but: '6 -just looking a't the' collection of indicators,. themselves, 7- 'does that require a considerable degree of. sophistication, '8 to judge something from them, or not? 9 MR. JORDAN:

From our method of using existing

~ 10 set is really, to me,-like shuffling'the cards and trying' l -11 -to find-outlier sets. And,. so, it's a-visual look, and 12 then after.you've sorted and found the outliers, then you I. 13 look behind.that data. So, I find it to be - if we have L if,' by 14 the'right presentation,'it should be a visual 15 going. through it, you get a visual norm, and then you 16 throw the outliers into the file. t 17 COMMISSIONER CARR: User friendly,. you'd class ,l i' 18 it, then? 19 MR. JORDAN: I'd like it to be, yes, sir. 20 COMMISSIONER ROGERS: Thank you very much. 21 MR. JORDAN: Thank you. 1 22 CHAIRMAN ZECH: Commissioner Curtiss? 23 COMMISSIONER CURTISS: No questions. 1 24 CHAIRMAN ZECH: Well, just a couple of comments, i. b 25 I noticed on your chart 7 -- we talked about it before-- l (202)234-4433 NEAL R.' GROSS & COMPANY, INC. (202)232-660 '1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

B. 57-- 6 1-butiyour failure-rate change indicator. Commissioner Carr , 5

2

. pointed out, - too,-- and you' acknowledge that if the trends -3 are consistent, you don '_ t ge t any indicator.

Well, we:

4: recognize the. li~mitations that brings. So, that,.of L 5 _ c o u'r s e, would be useful information i t s e l'f, bu t _ it' i 6 r e q u i r e s >. a d d i t i o n t.1 evaluation and analysis, to see 7 whether that steep ~ slope that you referred to, Mr. Jordan,: 1 8 may be even more of a problem-than a potential change of-9 the slope. 10 MR. JORDAN: Yes. 11 CHAIRMAli ZECH: So, you. recognize that, and I 12 think that's important. I thought it was significant that' 13 you ' pointed -out that the differences among maintenance 14-practices: drove the failure rates. I think that's quite 15 an important conclusion to come to and, so, it seems to me 16' that that would indicate that there we're certainly on the 17 right track and at least the efforts have gone so far to 18 develop this performance indicator. L, j 19 We know how difficult it is, but I think -- I'm i 20 encouraged, as I think my colleagues are, by progress 21 you've made, and I think it's a very significant 22 undertaking. We've got a ways to go, obviously, but 23 certainly you've made an awfully good effort so far, in my l 24 judgment. () l-25 Let me ask you just about a completely separate -(202)234-4433 NEAL R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND AVENUE, N.W., WASHINGTON, D.C. 20005

7;. g - 4 ,, y s z. 1 5 8.. 11 w w ~. n. 11" .subiect. I know we ' rei talking ' !about maintenance' here o o. 0 n2 Ytoday,r asi perf ormance indica' tors,- ~ but I'm interested' in-n n '3-the ' radiation exposure Jperf ormance indicator,- and any <4 progress yoil might have made -in that regard. -Can you give ~ 51 us'a status report on.that? 6. MR.. JORDAN: This.is in; obtaining the ;da ta?.~ m 7 'I'11 let Tom.Novak answer that. 8 CHAIRMAN.ZECH: All right. 9-MR., tiOVAK : -We did meet with INPO, and they.will 10-be submitting-thatiinformation on a quarterly-basis to us,- 11- .so:we'11;be able to include radiation: exposures in our.-- - i l 12. ~ . CHAIRMAN ZECH: So, you'll have a performance I ~13 -indicator' for radiation exposure, which I think is-6-- 14-significant. 15 'MR. tiOVAK : We will', yes. - 16 ' CHAIRMAN ZECH: You will be able to do that and 17 you're: working to get that information. 18 MR. NOVAK: That's correct. ' 1 '9 MR. JORDAti: We will have a longer time delay in f 20 that data. It will be about a quarter behind the rest of j 21 the data, but 22-CHAIRMAN ZECH: But you will have a performance I 23 indicator for radiation exposure, that's what you're i .24 planning on. ( .,..W. MR. JORDAN:

Yes, sir.

Right now, we're 25 0 '(202)234-4433 NEAL'R. GROSS & COMPANY, INC. (202)232-660 h '1323 RHODE ISLAND AVENUE,-N.W., WASHItiGTON, D.C. 20005 F

m.,.. -

7-

h. %.M

~ 59' c e. L~# $1 providing'thatidata.o'nlyfon an1annualibasis. g,

Okay, l

.t ? '2' CHAIRMAN -ZECH: Yes, I: understand that. r. t. .q ? i3" fine. E* l ~

j o

~ 4-Well, let me-just say,;I think we'.ve heard ' an - ,ya .5- . excellent ' presentation this morning. 'I-think ~ all of us 1 6: are encouraged by the. work' that's. going on between L AEOD s l7/ a n d R e s e a r c h.- I-' think 'it 's a'.. teamwork effort. We're a, [,

8

'really -- you know, the performance; indicator program,-'as 9' , we ' ve mentioned before - it'sLbeen ny experience, using !j ~ j m 10-Li ti s in the past, too is that it. takes time to mature. 'll So," I1think we should.not be discouraged by the' fact that L 1' 2. weJdon't have a completeIsolution to.the problem now, tj 13 =I think we should.be encouraged:by.the' fact that 14-we are making progress, and y o u ' v e '. c o m e to. some rather 15- -important, at least1 tentative, : conclusions ~ so far', -it 16 'would seem.'to-me. 1 -j 17 So, .I think it. is' an excellent-ef f ort, and 'I j -l 'l really-doLeommend the staf f f or' working to ' provide this j I ~19 very. important' tool which can be used wi th. o ther-20 ' inspections and management practices, t.o better evaluate j 21-the safety performance of our plants, and that's what i 22 we're trying co do. 23 So, I really do commend you for your efforts, 24 an'd encourage you to continue to refine and develop and k. '25 help mature this indicator program which I do think.has (202)234-4433'NEAL R. GROSS & COMPANY, INC. (202)232-660 1323 RHODE ISLAND-AVENUE, N.W., WASHINGTON, D.C. 20005 l

I'Q';[ 4,] ~ ~ ~- gy - - n ~ 160 A

9j q

..h L1' 'potentialdfor helping us'. measure'the. safety and'.the safety: i :'..,, 4 g S: l 2.L yparameters of.' the. - nuclear power - plants in: L our,1cou'ntry ' S# --3 lSo, again, thanklyou,for?an excellent l presentation.., 4 4 .Are there=any other comments.from my colleagues? '. 5 '(No response.). p t,, . i 6- .I f :' n o t, :thank' you 've ry much. 'We stand: i i -. 7 i' adjourned. ~ 't -8 (Whereupon, !a t 10:48 a.'m., - t h e meeting. was

9' a'djourhed.;)

10

11 12-

- 13 w 11 4 15 16 17' l l l;

18 '.

. lj R 19 io 20-I 21 L E 22 .4 l' l l. 'l ' *Q. 24

r. e.

lb 25' ,(202)234-4433-NEAL'R. GEOSS.& COMPANY, IMC. (202)232-660 0

61 e-1 CERTIFICATE OF TRANSCRIBER ] This is to certify that the attached events of a meeting -) l of the United States Nuclear Regulatory Commission entitled: s TIT 1.E OF MEETING: ' BRIEFING ON STATUS OF PERFORMANCE ' INDICATOR DEVELOPMENT '] PLACE OF MEETING: ROCKVILLE, MARYLAND j l i DATE OF MEETING: MARCH 1, 1989 i were transcribed by me. I further certify that said transcription i . is accurate and complete, to-the best of my ability, and that the transcript is a true and accurate record of the foregoing events. l AG-* A Au / Reporter's name: Phyllis Young l. HEAL R. GROSS COURT R890RTERS AND TRANSCRIBERS 1333 RHOOE l$CAND AVENUE, N.W. (302) 234-4433 WASHP40 TON, D.C. 20005 (202) 232-6600 t

O0 TNE M P n l-O LE E 9 V T8 A 9 E D1 D G1 N H 3 P FC I I ER F R A I O BM SUTAT S c.

c e e. ~ a c_ SD N S E E RT TA N D O I I T D C N N U A F C M E l 2 P E S T C E S T N D Y A O S N N C E E Y R T E T N S E R U F I A A A U M C S C ',(; l1 1

e ru l I iaP M e f r et i r nl u u u ef q l c iane f os a pu m o tn t eodn mc a s Ey d piSl 'o Cr uDa h a q c t Nm eRi e t Pc m Am S nNa Nu T r i d a p N t n e ES E r o i f 3 e s K Ts M c e R p i l d Nu H d y ei O ma t S n isv At L ao W sp I i a l I r P eb p s MS s E pi d M as O ee a R o l t U ed rt v n C ca a T rd e a C n U D A F I ~ ,L

7;j ql l 4, l e y-O s ,dn e eo d s pi e ys t t l oa a a n s nio l t l e n e d at ~ ve e r o c r a dm o h s t e c e n 2 r e l iPp m ev s m t i m S y. ei d y ac I eP t t Er dr oM a per Da oo c p r l m cf d s do S ee nc Om i d a/ T ey ss Cu N sd ua s S E u a ab a se a e c-K a dd t E M c S R d no r 4 s H ec Su d eD O r S dn cR I Ut e a n W P e t I a L s aP e s At P a nN E p u u d C S . M b e e R o n a ic t h t l O nt U e t Rs v n C Ee awT e o o i i f C t L M U D C A F (+ ~

SDNE de R d T ne m N mo O ce I r Try d Ca e m Nm S d a i r T v g ~ Um N o r o Fu E p r K p S M R 5 s H l O a t Ms S hg W i u r I t t L i Ea P sn E r Tt M R e i S h S O wU t Y C e T ru C N U F S A F = Y TEF A S \\

E L d U ~ n R a D s f S e f E r r a o S R u t t O s a O t l i r s o s T a c P i f p e A Cd O n e S c s C t Ri R n N P R R o t ID e s l r u O pt s N n I gt s P8 T n e o r e N I n 8 A I f ar pion E9 C ot S mispe r D 1 o e S ompv E I Cr D opc E l uit co sc G N r n N r m a Ne t p e A I E b ni n a of R l f Am S o e V Ne S cyt ut t e U t s /i n T c oy O t 6 E C a t Ec C nb a g l i t e E min C i TD O eim l N i i x F n bd Nf R m ef F oe ai E v f o E ph po P el t o g u ar S IA a cp D y E ce E e niv dr R Mr C af C eesn P ii a N mct N m A e c swai N A a h e m N epl N b y s y r cf E rSt F u E e S T at T oDu O n r e s st N h t n N osRd iPn E a I ~ A nl o IA h h EPN T N S i M M U a m

P iE S R O G U T A A TC PI L E D iU T E N E R A K J CI T A D NE M AR M NY E iU m OG R T E G J N S R N E C V I A Y U T S L A YO H A C iA I C R M C N F ID S7 O N N D I I S TE E E I I iP E R AT R S AR A C U A U R L E L I R I D E A C iAA R I F NR N M F I E I U V L B IT iE I A A F F L U M N iA U J C 0 5 0 5 O 2 1 1 fq

u ' $l .4 a. m O a U -4 V 4 4 4 4 C -a .a m-R .g g 'n i O C -n n .n O a -n -n .c 4: .5 I 0 4 m } G j -e .e 4 -e g L1J 1 o 4 j T m e m. O -e .s 5 4 C 5 w

j m

O unun+ \\ C 4 sum. e m.a m., +- muu _ C ~ unum -. ~ muu O summ M M -= ~

sumu, m.

m l l %... e' 4

D s o1l N a E I R o 4 T ~ S 03 S 7 E 8 N 02 EV R I 01 A T E C I Y /R E o ' 4 E F T9 R F A E S ' 3 U N Q O E 6 I 8 T C A 0 C ' 2 N ID A N I N 0 ' 1 F E O l T R E N B '4 I M 5 A U 8 M N 0 31 5 4 3 2 1 0 ~ y

HC S A E O C R R IT O P C FN P A S O A DI R S T I E A E P A T D M S B G CI E E L A T L L A S N Y T V I U R H E O T S R T B O N P S O R E O T F E I L R M N Y B T / E C A T T M N N N A N E O E E A T P 0 L S T M 1 R P Y MS A S OI O C N S S O T T N M O A O D C E C D N T E D A S D Z E G Y I N T SI S I L A MT N A R ER R I M E TO E R N SP H O E YE T N G SR O = x

la ic r sn e o m i m ta o c c i d l n l i a 0. r D 4 o f O tu 8 H o A T b T 8 / a A 1 E D 3 g / M n R 3a ir E t e H u a N S v T rd O E o h 1 S c O 1 t e I Y t T L s H 5a A A d T 8u r / q I D N o W e d s e S 1 A c I / I i L e d S a 1 E N u Y r A t L h O S e s A mt V U r I i T N o w u A A a A r l i f l C a L c s f E in T d R T R s h N o R R c iW O 0 r 0 A O O E e eB 5 L T L P R C P = l

7 7 ecnan S e S) n . i' I t \\ j 4 \\ Yn a i Lo M 's' n / i w At s u o Nit n At ~ k s ~ n n ~ Eo U-SC 2 i-1 Ur o ~ At -ll;lllll; ~ 1 Ca Aillllllll x llll ll; m llllllll l c

llllll

o lllll: i llllll Td 4llllllll %d llllll (llllll n n O (I 1llllll \\ 4l;!ll 4a l;lll ll!ll O l/ 1 R R tu 4 o r n a g e i W se J Tu p l.

sec e A v i T t i ~ c t a c A r e p p D s 'e r c e R n p a S E I n E P H e D M tn O T i S a C o O t E m E ID g g S n U H n U i T o A rc T S m se C o f at I E a n W i 3 r R s e 1 r U e e N L cr N nu O y IA ei l I l l T a O F r a A u ef T L t f I f T N ie E u Dh R M A E t R N L O g e O g n n iv C E P do d i R M n r id R n O F E F i R C L O C

mal NhN NCE eye" yMONTH 4 3 2 1 0 g N g O . g IT g A L y E _4 R R s R O O TA C C I D E N T D I O g S N D A A C T 4 R L 4 AD P P1 E N D S E g U SA

• 4 A

B C R* E L SV I P M g O 0 0 0 0 O 4 3 2 1 NPR iND CqOR (v

L TA I A )8 T H w E 4 T 0 D G 8 E N N S G / A I I T D T A sO U e N R E MS WA O )D I O E B 1 E D G O I R A ET T A T H SV S E U TA A E GA O EE S T FX L M UD OID Y OE N A I TT L C E E A N R C D P EO N DA P N EF E EE A CL I A W R N( 5 T EY EI 1 T N PR P E E I XA R T B M M E T E N P P E P N A H U GR X I I I A L S Q NP E I N E TO S Y P O' AR T B D RP N ITN E D A E A A PN L L D E T, OI P E S R N DE 8 C E EL 2 E L R AM WP CP EM FP A O I I I S GU V X A OQ E 0 LE R (E W 1 6 =

8 4 11 8 l i 4 d 0 i 3 n ) _e 7 5 8 8 r 9 Ts o i 2 1 rE r oR o tO cF ta aE B o i1 e cR N O R i d rT l E I T neAtR 0 i 4' R' I aE A-e WO P U6 1 cf Q n gL i 3 A nI a iC l n iR 6 oE 8 e tBM M o n S i 2 O ia C N O ( M T o i 1 I AC I D l N I o i4 LAT 5 O 8 T g 3l 0 O 0 0 o 4 3 2 1 7x

T S C D N E P E G O S E D S I TM T S DD O S C R U E RE R I N .EI G D PZ Y P L F C O FN Y R M N S Nl EO R E I E L I S RC O H OI C A FN I U T T M E I OS A O T F F V L TA L C E Es C W U R E B ES l RP Y G O F E N TS E F F N DC AE O LS R E A ER T C I E D C VU A I C B N R N OO D I L E A A E O S R S N A S I E PE R V F V U P SIT C MR E E 7 L T 1 R E E E N N I C E O E B D L N E S I C T F R E O H A B I A A T N C T D WE R M F I T W F E L I F I A D U U I YI UC L QA A N NE S O RD EP N N C H T I RM O E O E N S S SG I TI I I T T A UN E AT N AN D D DI E I T S LD DE O O N Z T R Ul E D I I A D H H O M R A I N T T R P CI R NI M AA E E OE RI OF N CM M M FR NM CO = l 1

L T A I U C O E H P T S I W D E S T R S A E C T D I S S O S C Y S B E A E D CT D N D EA D E A E N D T D - F O E sT ~N TO A R N E A L C E E T DS P L M M X H G N T 8 E O DG E 1 N O N S C ED AI I SU T D TO U M UJ AO CM SG DC A-A A R E N EL E 6 C G DI DA V F O O R O ON IT E R C P E CO CG I EN ET EN L SI SI RI A G D R D U U R A A O O N D I C A C T C E C (

\\1: l 1 1 2 9 e 2 c n 3 1 a 2 n e r c e o t n n r S r a i n a E E e M lo tn G ia /ll r tn v A M oC Rs ~ n ~ i Ee s' m d d Vo \\ A AC s e 9 Ys 1 k u Ra TC S U 6 D 1 3 N la 1 t 2 6 I s ro n l r r I i / a o r r F E t r sn . E o s p i r C u p e / q O P b E d r a F me e s h / o t n s d O e e n ~ D a c f R iL llll1lllll ll l i

8 3 0 1 2 g h p 2 %C ic n 9. g. is w2 r D o 1 3 nrO S k e /. g n ht le t g h UO s E C M n 6 G %/ o u 0 / c 1 o C As r g P n h Ro C / Ei n tc g VA ise Ae D v 0 Yi 2 t . ~. [ c Rer Tro ~ SC !. 4=E U s D 1' s 5 1 N 2 8 1 I g r n i i a n p ia e r R T / tp e R .p iu q E x

8 s R E n L o i t N c s A e g t I N p n u l TA S O s G s i t n r N e I D E T o i r I A T p D e D c e M A re O n c I RO R C e n a EC O d y s r i r n r s R e u L F w a o t E p s N t n e E L x s s s e I i s F S F Rg h m e F v e a i O En s r O U Li A Y d n s O m y 1 t 2 t y a i d n e e Y a l t r Y C ef m C g s i i l C A r u s a A o e a u T r q R at r c q R p DR U po I r P U S mp o L R C a S l C f O C Cr E S L oe n A E I C i L C r A T S A F A V ,y

~ E U L L A A V N C O e IT v IT i S t I c D O e D N N r AO G o r I E T A c A I D D d M I e V R F t S O O O a l T R F e E r P N IF R d I s A n E E P l a C S 2 2 N EN E se E DA D s B OM O u CR C ac s E.O n E d o S F S U R nit U ec E A A r a C C T P a c m O

0 5 2 7 2 i p r a ic e p% g s n 7 er9 i D a 3 /. r / h g tp e l C n h e e s i Ro n t n c ia .l o u r r ql o P T EA C s / / / r o s Sr n s r o NE i t c Oo r A / Tra m I t s i N t Ce p e AO n s 2 Ed i 6 p e s u .N % Vs q n ' E s1 3 r 1 2 e I m Te o pN 2 O 3 c $n s t 1 CL i e n 9 Ml ia

• l i

2 s Eh M r 1 e P t t Ri n re R W ia M r' h t 2_ O Ot s s e n s [ Ce u v a E C 6 1 t 3 s n 2 i. /I b r a t C F / n. ng i is m ,t d e A D

s R E T L R 9 O 8 P 9 ) 6 E 1 4 R R 0 l O P I 9 P F 8 W 9 G E 8 Y N N 9 C 1 ID S E R R O S O N T C ( F A L Q A T N L V A d O n P O M 2 T O I R R C P O H A E P F T 4 E I 2 R A Y W V U N A IT I T O L P P C U T S W E R N D E I F E N R G P O N O Y C L A IT E L N I N V P G O E S E I C D D B = = = ('

M E ) T T S S Y F Y M S S T E S ILT Y ( IB S T S A Y E S F D L A I R N AT S E A N O s V e T R A R A m T N C OC i t IF FI IAI D e N RN TN c O TG N v I i I IS EE r T Y-MC e C T s K EN N E 5 S CA f 2 FI o U AR AM L R t F S P S u D O E E o M FE F R R s E OT R n e I E U T C L a u i r S RE A P Q r l Y OL E t i I a T E TE R f S AS NR y t C EU T e n Y R TL F a i f I D a O OI S S T r T N A E F PF S I F = AS i

qd.liU!ii\\ I l 1 r SE Y T T I S L I I S 3 BA s IS T L e I A A u r Y V l L S A a i A T f L e I N N E s v U V v i A s E u 5 L s l E n c o n V M N o i t O AS a c I I d T R RG a t 6 2 F o C TO r L g n E A' T e T P A CT d tu S D UN b t RA n O C TL e g SP n S n I R R N i o T s p L i T O OM mU m T E S CO o S or ER C E R H RF R P I (. ~ jl1l l; ll

-ro t S a c E A i T d I A n T S i D E I e V E C c V R n I U a T T O m I C C S r E o A P A fr S T e N O A p R D P m O S G e t H T N s W I y s 7 I T N T s R ? I U S E W O X y L I R t L N E E e f 4 L O E a 8 E I N S s 9 T T O A N D 1 I O D U M R F L L A P P e I r A O X N N P T V V E I F = SS c

1M, fVfVWQVWWW66dWW9VfVfV66W666WWAWW6f'E.MWWW s,

• t t

St. TRANSMITTAL TO:- Document Control Desk, 016 Phillips -i i + p j ADVANCED COPY TO: The Public Occument Roem 3[J N OATE: ~ j 'j FROM: SECY Correspondence & Records Branch L l! Attached are copies.of a Commission meli'4n" transcript and related meeting ll. document (s). They are being forwarded.c er py on the Daily Accession List and Ll placement in the Public Document Room. NJ o'.. distribution is requested or required. t' Meeting

Title:

b4A[ At /4_b+ % ( JExj whdr .l Meeting Date: 3[//77 Open X Closed ?:' i j ftem Description *: Copies 1 Advanced DCS ]: '8 to POR M ~ c -l jl' .1. TRANSCRIPT 1 1 l l : I i u) h, -) \\ l' (/ l l 1 l l 2-1 : .1 l _13 3. i a3.a : 3: 3 4. .l .1 3 : 3 : 3 i. f .] 5. j l: 3 I a l a : 3 i : 1 l J,: g. .3 n.

• POR is advanced one copy of each document, two of each SECY paper.

A fd > C&R Branch files. the original transcript, with attachments, without SECY ul papers. NI N k@@@@@@@bh M @@MM@MMMMMM IN@N}}