ML20056D398
| ML20056D398 | |
| Person / Time | |
|---|---|
| Issue date: | 07/21/1993 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | |
| References | |
| ACRS-T-1967, NUDOCS 9308130119 | |
| Download: ML20056D398 (199) | |
Text
{{#Wiki_filter:[ 1 j g OFFICIAL TRANSCRIPT OF PROCEEDINGS ,rv i pf i [ a-t i l I ggng; Nuclear Regulatory Commission Advisory Committee on Reactor Safeguards
Title:
subcommittee on Computers in l Nuclear Power Plant Operations Docket No. l ? Q l gns Bethesda, Maryland r mm Wednesday, July 21, 1993 FACES: 1 - 197 { ' [1,s / f W ~-. n ;1TC*n a. a s m i N 1 Qv 1 ..) i f 0$ U:'Jb' p Q ANN RIIEY& ASSOCIATES, LTD. I 1612 K St. N.W. Suite 300 i ~ Mshington, D.C 20006 9308130119 930721 ( } 9 39 k PDR ACRS T-1967 PDR + 3 b j\\ _
,-,V.Kif, 6 qg .} f u'q-?f q %,# ne. = ,~ w v n' n ~
- nl k$?p @
q-.. -*^
- i
^ /,, - t <,, 3 e s, \\ x a .. r,. ? "h & Wi:
- LOFFICIAL TRANSCRIW~ ~OF PROCEEDINGS <.' R
^n am r s, A:u ce jf#$y {y 3, ' > 3%^ n r I 4 ygg - r ' 7x; hMh ' ~ i, t x .m -c .,-o y M' b t, o%>., ~9 +, s a h /E '47,P a-s ic s 7 1 < " - g%
- a si S
,a dQkip ~ T Qjo y 3y ? Mn n egh, m9 .O d-r R q". r -.- ' j-h J .( 1, !. ' j
- )
p@% %;~ ? c 5 @ -f6 < % w. m SQ~ g r, r ,4.y.; m.."b ~ 7 W, _. - ggng. ~ o n.., Mg(. l - Advis o ry:JCommittee.on/ReactorLSafeguards? [,&MZ wags Nuclear Regula tory}. Commissi,on.i + ,, m d, J, N. _.; ' f:L b mW5 Q' h,,y T 'i' ~1 }T{g{gj. Subcommitteeon Computerslini M' s' '~, iL4 'i, . Nuclear l Power PlantT0perat'ionsi gg ,r , ~ .,"gW .m gD: A. [ Docket No. _ $~!~AATk w 4. Yg E ', [
- q,
- . v.
W. c.~ w 1 4 9pym; 1%n, ;, ngn ~
- r TMb):::, #
e .c m ' Lyh's 1 s . -19 a ~ 77 $h#g 4 m r P , '9[ { W. 7: .i 3m 'd' s r vn um, s 5 k 3 3< me a Q; ' tqcAnom LBethesda,' Maryland-3.-:ff . Er .l, <*,,i a '} , s _:. p.;' y ~ + pQ +
- ) -
,u [l7 De-We dne sday, July..j21, fl9 9 3. 4. f> v43 , gh gg jJgg,71 m, e g.,,1,s... w ~, +;p.y,. ., t e'q.' i r o s <o ',p 9-3 s.. gr 3.:p g. _ A 4 Q c, r 9;' f f.i ' n i c f -M-M S&****'* l- _ppui - ~ e- -s+" i r m fg j((:%f.%n,tf p,, a j e.# c pfMOO[pr L& a ~ . C; W 6 U s M'e)T'0' - ANN.RILEY& ASSOCIATES., LTD.- ^ S;a ~ p .1612 KSt. RW, Suite 300 yqJ %.f Mshington,DC 20006 9 $A[,1 930e13o119'N0721
- (202) 293-3950 Ms 'g:,
y?' cp M,00 i~i PDR ACRS y$hd4 T-1967 PDR ',l j 7 g '. y&.j%p y u n m[b;1h: 2 . h ,1 i s 1 1 m w* - 4'hb $$ i j 9
l i f. ~ -t [. t PUBLIC NOTICE BY THE l UNITED STATES NUCLEAR REGULATORY COMMISSION i ADVISORY COMMITTEE ON REACTOR SAFEGUARDS i l i i i DATE: "7 l -) l -] i O The contents of this transcript of the proceedings of the United States Nuclear Regulatory Commission's Advisory Committee on' Reactor Safeguards, (date) .l July 21, 1993 as Reported herein, are a record' of the discussions recorded at the meeting held.on the above t date. This transcript has not been reviewed, corrected or edited, and it may contain inaccuracies. ANN RILEY & ASSOCIATES, Ltd. Court Reporters. 1612 K. - Street, N.W., - Suite 300 Washington, D. C. 20C06
l 1 1 UNITED STATES OF AMERICA 2 NUCLEAR REGULATORY COMMISSION 3 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 4 SUBCOMMITTEE ON COMPUTERS IN NUCLEAR' POWER. PLANT. OPERATIONS 5 6 7920 Norfolk Avenue 7 Room 110 8 Bethesda, Maryland-9 10 Wednesday, July 21, 1993 11 12 The subcommittee met, pursuant to notice, at 8:45 13 a.m. 14 15 SUBCOMMITTEE MEMBERS PRESENT: 16 Harold W. Lewis, Chairperson 17 Thomas Kress 18 J. Emmett Wilkens, Jr. 19 Peter Davis 20 21 22 23 24 25 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Slite 300 Washington, D.C. 20006 (202) 293-3950
].. 2 1-NRC STAFF PRESENT: O 2 M. Hon Vagin, NRR/DRCH 3 Cecil Thomas,.NRR i 4 John A. Calvert, NRR/DRIL 1 5 Ray Matthews, NRR/HICB l 6 Paul Loesen, RES/HEB 7 Leo Behracchi, NER/PD3 8 Robert Pulsafu, NRES/HFB i i 9 Robert Brill, NRES/HFB l 10 Franklin Coffman, RES/HFS l 11 C.P. Patel, NRR/DRPW 12 Wayne L. Johnson, NRR/DRPW 13 Johnny L. Mathis, RII/ACRS. 14 Beth A. Wetzel, NRR/DRPW 15 Joel L'. Kramell,'RES/PSR/HFB i I 16 Edward R. Schwelbinz, Region III/DRP t 17 18 19 i 20 21 22 23 24 25 ANN RILEY & ASSOCIATES,,LTD. Court' Reporters 1612 K Street,: N.W., Suite 300 -Washington, D.C. 20006 (202):293-3950- ) .. t
.I 3 ? ~' L 1 ALSO PRESENT: 4 2 Pat Place, SEI, Subcommittee Consultant 'i 3 Dr. William Kerr, Subcommittee Consultant 'f 4 Steve Brewer, AEP' l 5 Bob Carruth, AEP l 6 Daryl Herschberger, KWU/SPC i ~l 7 Bob Fink, MPR Associates 8 Jess Betlack,-MPR f I 9 John Chenord, ARC 10 Ray Torok, EPRI 11 Ricky Mason, CECO j 12 Steve Stimac, CECO 13 Charles Willbanks, NUS ] f 14 R.E. Spence, DOE-15 Sam Horton,-DOE QA Support
- l 16 Dave Modeen, MUMARC 17 George Rudy, NUS
~ 18 David Teague, Winston &'Strawn i 19 1 .? \\ 20 21 22 23 24 25 O l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,.N.W., Suite 300 Washington, D C. 20006 (202) 293-3950 1 . ~ J
'4 l 1 PROCEEDINGS i I . i 2 [8:45 a.m.] 3 MR. LEWIS: Let's begin the meeting. t 4 This is a slight change.of the meeting of the 5 Committee of ACRS on Computers in Nuclear Power Plant .) 6 Operations Subcommittee. I am Hal Lewis, chairman of the 7 subcommittee. The other ACRS members in attendance.are: 8 Ernest Wilkens, Tom Kress, Pete Davis. We also have our 9 consultants Bill Kerr and Pat Place at the table. 10 First of all, even though I am not guilty, I l 11 apologize for the confusion and the. change pace. It was one i t 12 of these things that was predictable, but it wasn't one of 13 these things that was assigned to us, but apparently, even ) 14 though it was predictable, it was not predicted, or the 15 prediction was never put into action. 16 The situation seems to be that-everybody agrees 17 that it is both desirable and inevitable, but there is some. i 18 confusion about these rules under which it can be done by a 19 utility, or specifically whether it can be done under a 20 provision like Rule 50.59 of the regulations. 21 To the extent that it can be done, there is a I 4 22 certain amount of disagreement between the staff and the 23 industry, and probably the subcommittee about what the l 24 threshold is under which a retrofit can be made in Rule } 25 50.59 and under what conditions it needs to be submitted to t ANN RILEY & ASSOCIATES, LTD. I Court Reporters f 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 l (202) 293-3950 j .~
i -) 5 ,s _ the NRC before it is done. That has to do with whether it 1 2 has a substantial effect on safety, whether it affects 3 safety systems. 4 I will reveal at the outset, certainly, the 5 attitude I have. I am beset with attitudes, and one of them 6 is that the search for exact thresholds for any 7 decisionmaking procedure is doomed to failure in the sense t 8 that precise criteria can never be found, although there i 9 often does come a point at which the contestants come to an 10 agreement to decide to let it go with ambiguous language, 11 and then face'the ambiguous language farther down the-i 12 stream. 13 But the constructive interactions between the t 14 staff and the industry in trying to find some agreed 15 threshold is notable, and we are going to hear about where -l t 16 they are today. That is my understanding of our agenda for 17 today. 18 The only logistic thing is we lost a few minutes 19 getting organized, and the other lojistic fact is that the 20 meeting has to end by 3:30, or somebody else has to take it 21 over because I have to get on a plane and go back to I i 22 California, and give a talk tomorrow. i 23 So, with that, I think I will ask, do any of the 24 people.at the table have something to say before we get i 25 cracking? i \\ ANN RILEY & ASSOCIATES, LTD. j Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 i (202) 293-3950
6 1. [No response.] 2. MR. LEWIS: In that case, I forgot to mention that 3 Doug Coe, seated to my right, is the Designated Federal 4 Official for the meeting, and is responsible for everything 5 that happens. He is responsible for my being here. I wish 6 him luck. 7 Without any further ado, I guess Jared Wermiel is 8 going to start for the staff, and.we are all yours. I.can 9 only implore you, despite our hinderance, to try to stick to. 10 the schedule. 11 MR. WERMIEL: I will try to stick to it, Hal. I 12 don't anticipate that I will taking the full time allotted 13 to me. ( 14 MR. LEWIS: That would be even better. .1 15 MR. WERMIEL: Good morning, my name is Jared 16 Wermiel, I am the chief of the Instrumentation and Controls 17 Branch in the Office of Nuclear Reactor Regulation. I'am 18 going to speak, just exactly as Dr. Lewis indicated, on the 19 current status of the interactions between the staff and 20 NUMARC, specifically, on the development of a guidelines 21 document for implementation of retrofits to I&C systems, 22 digital retrofits specifically, under the provisions of 23 10 CFR 50.59. 24 By way of background, the staff recognized -- 25 MR. LEWIS: I love that " genetic" thing. 10 'I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D C. 20006 (202).293-3950
-g i 7 1 MR. WERMIEL: Genetic? I thought I had proofread cO'. -y 2 this. 1 3 [ Laughter.] i 4 MR. WERMIEL: I used spell check and that is a I .i 5 word, and it wouldn't pick it up. Ia 6 MR. LEWIS: Is that software that did that? 7 MR. WERMIEL: Yes, it is. 8 Isn't it, Paul? 9 MR. LOESEN: Yes. 10 MR. WERMIEL: Yes, it is. 11 The staff has recognized.for quite a-few years 12 now, long before I got into the Instrumentation and Controls 13 Branch, that there is a desirability toward implementation { 14 of digital systems in nuclear power plants, and the industry 15 has been doing this at some pace for quite a few years. l 16 As a result of'this recognition and some j 17 inconsistencies that the staff viewed in the process of .j 18 performing these implementations and interactions with 19 industry, it felt it was' appropriate to put forward a 20 guidance document. 21 In August 1992, the staff issued a generic letter 22 in which it proposed an approach towards implementation of' 23 retrofits under 10 CFR 50.59. That document went out for .\\ 24 public comment specifically stating that, because the staff .25 viewed the digital system itself as providing for or t l ANN RILEY & ASSOCIATES, LTD. Court Reporters i 1612 K Street, N.W., Suite 300 i Washington, D.C. 20006 i (202) 293-3950 i j .m.__., .f I
i I 8 i 'N-1
- k'_) -
imposing a kind of a situation that had not been previously j j 2 analyzed, and that specifically dealt with the fact that 3 digital systems employ software that is basically the same l 4 in redundant channels and that that software is subject to a f 5 possible failure. i 6 As a result, since this is an unanalyzed failure 7 mechanism, failure mode or malfunction, as defined under 8 50.59, these kinds of retrofits would need to be reviewed by l 6 9 the staff prior to their implementation. That is what the ~ 10 Generic Letter said at the time. 11 As a result of comments on that approach, which 12 basically indicated that that approach was overly 13 conservative and some additional requests from CRGR to try l 14 to develop an alternative approach through a threshold -- in-t - t 15 other words, try to come up with a means of defining-when J 16 you really would have this unreviewed safety question in an f 17 alternative way -- the staff agreed with the industry to set 1 18 out on a course to come up with a not_so conservative, or a 19 less prescriptive, approach toward implementation of these l 20 retrofits. 21 The staff viewed this at the time as appropriate. 22 because there was a recognition that if we had to review 23 each and every proposed digital upgrade in the future, this 24 might pose an undue burden on both the industry and the NRC ld 25 and may not be appropriate. So we have begun over the last ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 i Washington, D.C. 20006 i '(202) 293-3950 r
f 9 i 1 approximately seven or eight months to work.with NUMARC in 2 earnest to come up with an alternative means of establishing 3 a review criteria, if you will, under 50.59 for these 4 retrofits. 5 NUMARC agreed to take the first crack at this and,. 6 by a letter dated April 8, provided the staff with their .l 7 proposed guidance document. That document basically left.it 8 up to the plants to decide when they would have an 9 unreviewed safety question. Primarily, based on a l 10 determination that they were to make, whether or not I 11 software was really a credible failure. l l 12 We reviewed that document and we have provided 13 comments on it. That document not only dealt with the legal-14 interpretation of 50.59, but also provided some information' ( f 15 on design of digital systems for the industry,.specifically ~! 16 by reference to certain standards and certain additional 17 guidance, and we also looked at that as well. i t 18 MR. DAVIS: Excuse me. The document you are 19 talking about -- I think we have a copy -- there was a 20 statement on page 4-3 that we wanted to ask you about. The 21 statement says, "Since it is considered impossible to prove l 22 that software is error-free, software failure is deemed to 23 be credible." .i 24 The thing that bothered me about the statement is t 25 if you can't prove that something is error-free then it is. l.; O i ( ANN RILEY & ASSOCIATES, LTD. 't Court Reporters ( 1612 X Street, N.W., Suite 300 ~ Washington, D.C. 20006 I (202) 293-3950 l I
10 1 automatically credible that an error will occur. It seems 2 to me like there is an intermediate ground where something 3 can't be proven error-free but a. failure isn't as 4 incredible. You know, it is not black and white. Errors 5 are impossible to prove. It is impossible to prove that an 6 error cannot occur, but it could be such a low probability 7 that it is not worth worrying about. 8 MR. WERMIEL: 'That's correct. If we thought'we 9 could be convinced that a software error was, indeed, of a 10 low enough probability to call it incredible, then we-11 perhaps -- 12 MR. DAVIS: It doesn't have to be impossible? 13 MR. WERMIEL: No, just incredible, just unlikely l ) 14 enough, maybe is the way to say it. 15 MR. DAVIS: That's not what this statement says. 16 MR. WERMIEL: Right. I know. Part of our comment 17 on the document itself was that we thought there might be 18 some misinterpretation within the document, if you used the 19 guidance itself, because it seemed, on the one hand, to be 20 saying, "Well, this is a possibility, yet you ought to 21 decide whether it is credible or not," and, on the other 22 hand say, "You can't really prove whether it is credible or 23 not." 24 From our standpoint, it was simply a matter of 25 saying, " Hey, we think we have enough evidence and enough O l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006-(202) 293-3950
11 1 understanding of these systems to question, at least, 2 2 whether or not it is unlikely enough to develop software 3 that is error-free" and, for that reason, we still believe i 4 that within the threshold that I will talk about you have an 5 unreviewed safety question and you need prior staff approval 6 before you implement your retrofit. i 7 MR. LEWIS: To pursue Pete's point, I wasn't going 8 to bring it up this early, you are drawing a distinction in i 9 your own mind between possible and credible. You said that i 10 several times, and you are interpreting credible as so i 11 unlikely as to not be credible. That means you have hidden 12 somewhere in the recesses of your mind some level of 4 13 reliability and probability of what is credible and some () 14 level that is not, but you are asking the industry to not-l L 15 know what it is, "But trust us, we know what it_is." Is L 16 that the way it is? l 17 MR. WERMIEL: No. I i 18 MR. LEWIS: I know I'm putting it in a prejudicial 19 form. 20 MR. WERMIEL: No, I don't think it is quite that i 21 simple. I think -- and the industry will let you know their I s 22 opinion -- but I think we would agree with the industry that l 23 software is not error-free. Whether or not we would agree' ~ 24 with them on a threshold for review based on that assumption 25 is something that we are dealing with right now. ANN RILEY & ASSOCIATES,-LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 ,,,_m ,.~er
*n--w*
- +
12 i O: But I think we all agree that if we don't have a 1 2 number, we at least qualitatively will acknowledge that iff i 3 there is enough data out there, enough information, enough i 4 experience to tell us that is credible enough that we ought. 5 to at least be concerned for those things that are important i 6 to safety. Those systems that have some nexus to our safety t 7 concern, we ought to be looking at them a little bit more in 8 more detail. 9 MR. LEWIS: My problem is essentially the one that i 10 I said in the introduction, that I am afraid that the search j 11 for precise standards of what is credible and-what is not 12 credible is like the search for the Holy Grail. You can f 1 13 spend your life at it, but you are not going to find it. In-( 14 fact, everyone may agree that that perfect software doesn't '\\ 15 exist, except me, because it is just much too general a 16 comment. l 17 I can write a two-line program for you that I 18 guarantee you has no flaws in it. If you are worried about 19 the Compiler, I will write it in Assembly; if you're worried-20 about Assembler, I will write it in machine language and it 21 will be error-free by any standards. So just to say that 22 software errors are credible does not make them unlikely. 23 One is in the end trying to deal with degrees of likelihood, 24 and 50.59 suffers from the same defect. 25 MR. WERMIEL: Yes, it does. O I ANN RILEY & ASSOCIATES, LTD. Court Reporters' 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 13-1 1 MR. LEWIS: It's not written'by geniuses. I mean, 2 it speaks of when:something increases the safety, every time 3 you test something it decreases the safety. v 4 MR. WERMIEL: You are absolutely right, 50.59, I l 5 believe, interpreted the way the staff would interpret it, i 6 is implemented in accordance with the original version i 7 Generic Letter. "Any malfunction," it says; it doesn't talk l 8 about safety significance of the system you are applying i 9 your modification to at all. It doesn't maKe that argument. l t 10 This idea of trying to come to grips with a i 11 threshold may be contrary to what 50.59 is actually saying. 12 We think, however, we can do it, not necessarily to the i 13 prescription I think you might be thinking we are trying to 14 do it to. I think what we are going to do eventually is [ 15 provide a guideline that we are all going to recognize, both 16 in the industry and in the NRC, are going.to have i 17 exceptions. 18 When those exceptions come up, so long as we can-19 act as rational people and discuss them, I think we can come 20 to agreement that even for things that might appear to be 21 above this threshold -- if we ever agree to it -- we could 22 maybe make a compromise or maybe come to an alternative 23 decision that you can go ahead, you don't need the staff 24 review, with certain understandings. I anticipate that that j 25 is how this is going to come out. I ANN RILEY & ASSOCIATES, LTD. i Court Reporters j 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
14 j 1 MR. LEWIS: Let's move along. 2 MR. KERR: I think implicit in this position is i 3 the assumption that review by the NRC is going to increase-4 the safety of the systea. It seems to me that it would be j 5 helpful to everybody who is looking at this question if we l 6 knew -- and perhaps everybody but me does know -- what you 1 7 were going to look for and what you are going to find 8 acceptable and what you are going to find unacceptable when i 9 do you the review. Does that guidance exist? i 10 MR. WERMIEL: Yes, it does. It exists in the form 11 of a fairly comprehensive safety evaluation that the Staff f 12 did of the Zion Eagle-21 Digital Retrofit. i 13 MR. KERR: I am not talking about the Eagle l 14 retrofit. I am talking about a general set of principles i 15 that says, "Here is-what we are looking for, here is what we j t 16 think we will find acceptable, and here is what we think is 1 1 f 17 not." 18 MR. RUSSELL: This is Bill Russell from the Staff. 19 There clearly is a precedent from reviews that we 20 have conducted to date. However, I would characterize in an 21 activity we have ongoing now there has been an attempt to, 22 in fact, quantify by laying out on paper what the review 23 standards are. We are working on developing a standard 24 review plan that could be used. We are working on 25 developing a process and defining better within that process ON-4 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 -- f
'i f 15 [ l' what would be reviewed and-what our standards.are. That~has 1 2 not been completed. 3 We have been, in fact, doing case-by-case reviews 4 as we go and learning as we go, but we are in the process of i 5 attempting to do that now. It has not been completed. So, [ 6 basically, what exists today would be a compendium of safety \\ 7 evaluations that have been issued for reviews that have been-l 8 completed. We have a number of reviews underway, and we are 9 giving it high priority. We just have found, for example, 10 an issue that came up for the first time that we'had not 11 seen before related to the Diablo Canyon. We are attempting 12 to -- 13 MR. KERR: Is there a schedule for completi;; of 14 this guide? f 15 MR. RUSSELL: We are working on it now. I don't. 'I 4 l 16 know what the actual schedule is. I hope to have guidance ) 17 that would be in a form to be discussed with the committee la and with the industry within six to twelve months. Whether 19 that is achievable or not, that is the kind of timeframe I 20 think we are on. In the meantime, we are continuing'to do, 21 essentially, case-by-case reviews. 22 I would characterize, to put in it context with 23 what Jerry said, we want to identify those things which, 24 based upon the safety importance of the functions, clearly 25 fall into a category that require NRC review and those i ANN RILEY & ASSOCIATES, LTD. Court _ Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
~ t l 16 1 things which fall into a_ category that clearly do not j 2 require NRC review and that we can reach agreement on, and 3 it is probably going to have to be illustrated with some 4 example to work this threshold concept, and then I think 5 there is going to be a gray middle ground. j 6 I would suggest that how wide that middle ground 7 is may take some experience and actual implementation to. l 8 work out, but the concept of a threshold is not a sharp 9 line. l 10 MR. KERR: I am less concerned about the i 11 threshold. My impression is that one of the things that 12 concerns the industry is the uncertainty that now exists 13 when one proposes something as to how long it will take to i () 14 review it, because, in contrast to new plants, in' existing 1 15 plants these sort of changes have to be scheduled. i 16 MR. WERMIEL: Yes. ~ 17 MR. KERR: If they can't be scheduled, it is i 18 almost impossible to implement them. l 19 MR. RUSSELL: We recognize that problem and we are 20 working on it. We have been doing it case-by-case in 1 21 developing a precedent from reviews that have been 22 performed. We are working on a standard review plan to 23 implement schedules. 24 MR. KERR: I guess I don't see how the Staff can 25 review these unless there are some guidelines. The chart O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
l I l i 17 t i i says that each group that does the review starts'from the 2 beginning. 3 MR. RUSSELL: No. That's not what I said. 4 Because we haven't issued a final guideline that.is complete 5 and up-to-date, we have been using standards since about j 6 1980 for reviews that have been done, and we:are using IEC 7 Standards of 880. We are using the same approach we used at j 8 RESARP. There have been issues that have come up onLsome 9 reviews, and each time you see a new issue, we are trying to 10 collect that information, put it together in a standard i 11 review plan approach so that it is clear to everyone. 12 That activity has not been accomplished, but that { 13 is not to say we don't have standards. That is'why I said l (( ) 14 that I believe that the safety evaluations that have been 15 issued constitute, in fact, what we have found acceptable. l 16 We are taking that, plus the information that we have-gained 17 from the new plant reviews and the approaches of what we 18 have discussed internationally, and we are attempting to 19 wrice now a standard review plan of to how to accomplish 20 this review in this area. 21 MR. KERR: Thank you. 22 MR. WERMIEL: Let me make a point, Mr. Kerr. The 23 bottom bullet may also help address the question you are 24 asking. In the course of looking at this guidance document, 25 there is, indeed, reference to a number of standards and O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950.
18 1 other information that we are, again, negotiating with I 2 NUMARC right now as to what is appropriate for the design of: 3 these systems. l 4 So this document itself will also have in it 5 agreement, we hope eventually, between the Staff'and the [ 6 industry on what standards and what specific guidance ought { 7 to be for the design of these' things. This may in a sense 8 be a prelude to eventually incorporating or developing the 9 standard review plan that Mr. Russell has been talking 10 about. There is going to be that kind of information in 11 this document. 12 MR. LEWIS: Could I throw a little more gravel in ) 13 the gears, while I am at it, because much of the I () 14 conversation so far -- well, Bill, for example, referred to i 15 the earlier history as a kind of example of what is l 16 acceptable to the Staff and you, Jared, have spoken about an-17 agreement between the industry and the Staff, those are all 18 good things, we all know that. 19 There is still another level of concern which is l 20 to some extent the responsibility of our subcommittee and 21 committee, which is the effect on the safety of the plants. 22 You know, that goes beyond the question of whether the 23 industry and the Staff come to an agreement You can come 24 to an agreement and can be wrong, and there is a certain 25 probability of that. 4 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite.300 Washington, D.C. 20006 (202) 293-3950
~ 19 1 I wonder whether there'has been much consideration i ~ 2 given to trying'to avoid the kinds of thresholds - I think i i 3 the first version of the Generic Letter -- I think it was-f 4 the " Generic Letter," I may be wrong -- said,."Is the change -I -} 5 safe," and, of course, that is just an invitation to 6 controversy and to foolish, wasteful conversation because i 7 nothing is safe or unsafe. j 8 The probabilities are there,.the uncertainties are ~ i 9 there, the conflicts are there, the honest differences of i 10 opinion are there. If you look, you know, some of the IEEE 11 documents put much more emphasis not on standards,'but on - l i 12 process of the descriptions. j 13 MR. WERMIEL: Yes. i - () 14 MR. LEWIS: I wondered whether the Staff has given ( i 15 anybody -- and the industry, I will ask them the same l i 16 question -- have given much consideration to trying to avoid l 17 this search for is it safe or is the accident credible or j 18 does it affect only safety systems or not safety systems, to 19 scrubbing all that in favor of prescribing a process by 20 which knowledgeable people are required to review the thing I 21 and, in the end, that will be deemed adequate? i 22 You know, that is much easier to implement than 23 what you are trying to do. 24 MR. WERMIEL: We have, indeed. The standard that. I 25 we and the industry are right now focusing on for i ( ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite-300 Washington, D.C. 20006 j (202) 293-3950 .i i
V 20 I i 1 accomplishing that is basically a process-oriented standard, 2 and that is IEEE 7.4.3.2. f f i 3 MR. LEWIS: That describes the process. 1 4' MR. WERMIEL: Correct. 5 MR. LEWIS: But yet, as I read the Draft Generic 6 Letter,-it's full of these thresholds, you know, these 7 marvelous diagrams with diamonds in them, diamonds I can't 1 8 sell on the open market. 1 9 MR. WERMIEL: Can I try something.out? l 10 MR. LEWIS: Try anything, it is your nickel. I 11 MR. WERMIEL: I think the Generic Letter has two 12 focuses, and I don't know if it is appropriate to mix them l 13 or not. The one aspect of the Generic Letter is this -- and I () 14 I will call it the legal aspect -- what is required by the i 15 rules of 50.59. That issues is not,necessarily a-very { 1 16 technical issue, it is really, what-is the Staff's legal -1 17 responsibility as far as review goes and the industry's i 18 legal responsibility as far as the evaluation they do in 19 accordance with 50.59. 20 The other question is, once I have the legal one 21 solved, what is required of me technically to design, 22 develop, build an appropriate digital retrofit. That is 23 also in the document. but I don't know that answering that 24 question necessarily focuses particularly on addressing the 25 legal one. O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 o Washington, D.C. 20006 (202) 293-3950 j . 1
P i 21 I i 1 That, I think, is a distinction that I make, at 2 least, with this guidance document because it does have 3 information in it for designing the system that doesn't I 4 necessarily reflect on how you are to do this evaluation to l 5 legally answer the questions posed by 50.59. 6 MR. LEWIS: There is a general rule about legal j 7 interpretations of ambiguous documents to the extent that 8 50.59 is ambiguous; that is, you can do as you-please as-a f 9 regulatory agency, provided you are not irresponsible, i 10 provided you don't stray too far from what it might have l 1 11 meant. 12 So I have problems with doing too much of the "how l 13 many angels on the head of a pin" stuff. Rule'50.59 is 14 imperfect but leaves a lot of room for you to do sensible t 15 things, and that is good. 16 MR. WERMIEL: All we are trying to do is, to some 17 degree, I guess, define what we would find to be reasonable i 18 in the performance of an evaluation under this guidance. 19 That is all. That is all we are really trying to do. 20 MR. LEWIS: I should let you go on, but Bill wants 21 to make a comment. 22 MR. WERMIEL: Okay. 23 MR. RUSSELL: Let me make two comments to put 24 things in perspective. First, to your earlier point, the 25 staff agrees that a process for development of an integrated 'l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 l Washington, D.C. 20006 (202) 293-3950
m a I -22 1 hardware / software system for the purposes of safety-related l.i - 2 application is important and, in fact, that'is the process L 3 that we are certifying for the design reviews, which we have 4 called design acceptance criteria, which is a phased process 5 which is based upon how you implement from the initial- ) 6 system performance specification through to final 1 7 validation, verification of a digital-control system. l 8 We think that's the right way to go. We have not, 9 at this point, while we are using some elements of that, we 10 have not incorporated all of that. We have some of it in 11 the standards, but we have not essentially licensed a j l_ 12 process, which is what we are proposing to do for advanced 13 plants, so that we do not want to lock in a technology or 14 lock in a particular design and have it become obsolete, j so 15 the process could be repeated and you could introduce new j 16 technology when it becomes availabl'e. 't 17 In the issue that you are discussing with respect 18 to 50.59, this has been an area that for some time the Staff 19 and the industry have recognized, and there has to be a 20 better definition broadly in 50.59. That is why there has 21 been industry activity and there has been staff review and 22 we have a hundred-page-cr-so document that provides 23 background on how to perform 50.59 reviews. It is that 24 document that this information would be supplemented into 25 once agreement is reached. 'O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite'300 L Washington, D.C. 20006 l (202) 293-3950
m ~ i l I l 23 {} 1 I think that we are not going to have a sharp 1 i - 2 definition of a threshold. I think it may be, clearly, if 3 you are going to replace the entire reactor protection-4 system with a digital retrofit, that is something that 5 requires URC review; if you are doing something less, if it. f i 6 is a single channel and it is in a digital display with 7 little safety significance in and of itself, notwithstanding j 8 that it may have digital components in it, that could l 1 9 probably proceed without review. 10 What we want to do is get that narrowed down to l i 11 clearly identify some cases which do not require review and j t 12 to identify those which do and to give some guidance as to j i 13 how you proceed. That is the concept that we are looking at . f~ i -i 14 right now as it relates to the review guidance in the ( 15 Generic Letter. We are clearly saying not all, that was an ~ 16 overstatement that may be legally precise, but, again, we j 17 think we have room to define that and do that through a 18 public process of going out for comment,. addressing the 1 19 comments and stabilizing. l 20 So that is really the long-term approach that we i i 21 are on, and we hope that we can be there within this -i 22 calendar year. The schedule, I believe, as laid out is to j l 23 try to come to agreement on this by the end of this calendar i 24 year. 25 MR. LEWIS: No, I understand that, but when you .l O 'I-ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
~- 24 1 speak of -- and I am a supporter of the DAC process, there is no ambiguity there. If you try to make the distinction 2 3-of replacement of small elements of a system, you run into 4 the problem that, when you look at the history of software' 5 failures, more broadly computer failures -- because I have l 6 never understood this remarkable distinction between 7 software and hardware, it's a package -- when you 1rok at 8 the history, it is always the little things that do'you in. 9 It is not the concept, it is some little thing that has been 10 done wrong that interacts with the rest of the system that 11 in the end produces an accident. 12 I have a personal experience that it is not always. 13 the software. I can't name the company because I will be .j () 14 sued, but I bought a new printer from a well-known company-i 15 in the United States not too long ago that had a defect on 16 arrival. I gave it to their warranty repair people, who 17 returned it months later unrepaired, and the company sent me 18 a brand new one which had the same defect. It turned out it 19 was in their firmware. It was the original firmware. They 20 had sold hundreds of thousands of copies, and it was just my i 21 luck to have discovered it. It had nothing to do with 22 software or anything like that, it just was an obscure 23 problem. I 24 So I worry about using scale as a criterion in any 25 way because I think it was -- to quote a local authority -- O i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
25 1 Ivan Selin, who keeps saying that the devil is in the x(])^ a o 2 detail, but it really is in the software in the computer 3 business. 4 You know, I am using up more of your time than you '{ 5 are, so go ahead. i 6 MR. WERMIEL: I would just make a comment that I 7 agree with you and that is part of the recognition that both -l 8 we and the industry have that we are trying toideal with, to 9 the extent that we can, with the document we are both i 10 pursuing development of. 11 The Staff provided its comments on the NUMARC i 12 document by letter dated June 2, and the specific concern we 13 had was that the document didn't have enough of a threshold 1 ) 14 so that *-he industry could really put its hands around 15 something and say, "Okay, at least at the highest level, I 16 know what I am proposing to do probably will:need to be 17 assessed by the Staff or not." ~18 So we provided a threshold, again, as a first try. 19 That threshold was based on, as I already mentioned, the 20 safety significance of the system to be modified because the 21 Staff felt that it was likely that a retrofit to that system 22 would result in an unreviewed safety question for the 23 reasons that I have previously mentioned. 24 Another important comment that we made, at least 25 in our minds, was, based on our understanding of industry _ ANN RILEY & ASSOCIATES,-LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 26 1 experience, there appears to be some lacking among certain 2 utilities and their plant staffs of an understanding of what 3 it ic they are getting into when they retrofit a digital 4 system and what they will need to know in order to operate 5 it and maintain it properly. We wanted to ensure that that i 6 was at least mentioned in the document and was part of what t 7 was considered when the utility did propose to make such a 8 retrofit. I 9 We also pointed out in o'ir comments that the Staff 10 level of review effort would be reduced for those ;ystems 11 that fell above the threshold if we had previously reviewed 12 and approved the same or a very, very similar type of a 13 _etrofit. We indicated that our focus for situations like i i } 14 that would be only on those aspects of the retrofit that are 15 plant-specific. We wouldn't go back, for examplit, and 16 rereview the software development program for Eagle-21, if 17 the Eagle-21 modification being proposed was the same as one. 18 we already approved. We would only look at the specific 19 aspects of incorporation or implementing it at that 20 particular plant. 21 We also made a point in our comments that we 22 agreed with NUMARC on reference to various industry 23 standards, particularly 7.4.3,2 and we agreed with the use 24 of the broad guidance in-NSAC-125., which is an industry 25 document that-provides guidance, in' general, for i ANN RILEY & ASSOCIATES, LTD. Court Reporters i 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 . I.
27 1 modifications that are made in accordance with 50.59. 1 2 The threshold. This is the threshold as we 3 nroposed it to the industry and, as I indicated, we are 4 still open and still negotiating with them over this, how to i i 5 define it exactly, how to allow for exceptions or specific j 6 deviations under certain circumstances, what exactly we can f i 7 do. t 8 But we clearly felt and we still feel that digital-9 modifications to the reactor trip syster., essential-features i i 10 actuation sy0tems and post-accident monitoring systems that 11 fall within Category 1 -- in other words, the primary 12 parameters re]ied on by the operator following an event -- i 13 would likely fall above the review threshold, because they () 14 would result in an unreviewed safety question. 15 Again, we pointed out that the review process 16 would be streamlined where we have already approved a l 17 retrofit through a topical report or some previous 18 evaluation tLat we had performed, even to these systems. 19 These systemu * .e se ected because they are, in our minds, i 20 those that are mm... satey -significant. I 21 There are modifications and-retrofits that have. 22 been performed to a number of other systems that we.are i 23 aware of that we believe would fall below the threshold, and i 24 it is quite conceivable would not require prior NRC approval 25 using the 50.59 process if those systems were modified, and 1i t ANN RILEY & ASSOCIATES, LTD. l Court Reporters i 1612 K Street, N.W., Suite 300 l Washington, D.C. 20006-i (202) 293-3950 i l
28 l 1 we provided examples of those. 2 Once again, this is still just a proposal and we 3 anticipate continued discussions with NUMARC on this. 4 MR. WILKINS: Mr. Wermiel, let me ask if there is 5 a better word than " threshold" for what you have just 6 described? l l 7 You heard the Chairman's remarks about the 8 futility of attempting to pick precise quantitative ] l 9 thresholds, and I think most people use threshold as if it l 10 is greater than seven, then you do this; if it is less than 11 seven, you do something else; and if it is equal to seven, 12 you measure it again. 13 MR. WERMIEL: Yes, true. ( 14 MR. WILKINS: These are not of that sort at all. I 15 I mean, these thresholds that you have here say to me, at 16 least, if this is a digital retrofit that affects the 17 reactor trip system, then we will review it; if it affects a 18 non-safety system, we will not review it. 19 MR. WERMIEL: Right. 20 MR. WILKINS: Those are not-subject to measuring 21 errors at all. I mean, you know whether it affects the 22 reactor trip system, or it doesn't affect the reactor trip 23 system. It is not fuzzy in thel sense that Dr. Lewis was 24 speaking earlier. 25 If you could find a better word than " threshold," -I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K' Street, N.W., Suite 300 Washington, D.C. 20006 i -(202) 293-3950 i__________1____,______,________,,________,__,_____,______.__________.________,__.,.____________1,_____,__,_-_________.__
v 1 29 } 1 you might avoid some possible confusion, I don't know. 2 MR. WERMIEL: We will consider that. The word I 3 believe.was first proposed by CRGR and we have just sort of 4 stuck with it, but, in a sense, you are right, we are not d 5 . talking about a fixed line at some quantity which if you 6 exceed, you do this; if you don't exceed, you do that. .It-q 7-is not, as you point.out, that cut and dry. j i 8 We tried to identify the line, or the threshold, -l 9 whatever term we use, other ways and were unsuccessful. We 10 just couldn't figure out, from our perspective, any other f ] 11 way of coming to grips with the issue of what it is that 12 . bothers us when a retrofit goes in, other than'through this 13 kind of an approach. l '( 14 MR. WILKINS: I have no problem with the approach. j 15 MR. WERMIEL: Right. I understand. Maybe it 16 isn't a threshold so much as it is a -- 17 MR. RUSSELL: A categorization. l I 18 MR. WERMIEL: -- a categorization or a division, 19 something of that sort. Yes. That is a fair comment. I 20 Where are we going, NUMARC currently is reviewing ) 21 the June comments'that the Staff provided. We anticipate a 22 meeting around the August timeframe to discuss.their 23 response to our comments, and we will continue to deal with '24 whatever issues and concerns come up. 25 Ultimately, our aim is to issue a final Generic ~ O ANN RILEY &. ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
30 1 Letter which ideally would reference the proposed NUMARC or ( 2' industry guidance. document. That is the ultimate aim that 3 we and the industry would like to achieve. 4 As Bill Russell pointed out, there is an ongoing 5 effort with our Office of Research'to develop additional 6 guidance for use by the Staff for consideration of software, 7 and we are also participating and are nearing completion, as 8 a matter of fact, on development with the industry on 9 several standards that we ultimately, again, hope to 10 reference in the standard review plan. These are, we 11 believe, ultimately going to be consensus standards. The 12 industry and the NRC will agree on these. We would hope and i 13 expect that modifications in the future, whether or not the. ( 14 Staff reviews them, regardless of whether we end up i 15 reviewing them or not, would be designed, developed, built, 16 and implemented in accordance with the criteria in these. 17 standards. In that way, we would have some confidence that. 18 the modification is well designed and is an appropriate one i 19 and would cause us less concern. 20 MR. LEWIS: Let me address what may be a 21 politically incorrect issue. When you speak of Staff 22 review, are you talking about in-house review or contractor 23 review or what? 24 MR. WERMIEL: I speak of both. Ultimately, any 25 evaluation we do is a staff evaluation. O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,-N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
~ - ~. t l 31 l ] 11 MR. LEWIS: No, I understand Staff makes these 4 2 decisions. i 3 MR. WERMIEL: We have used contractors in the past l 4 to help us in these reviews, the Zion SER that I' spoke of on i 5 Eagle-21, INEL provided technical assistance to us. INEL is 6 providing technical assistance to us now on the review of a 7 reactor protection system modification for D.C. Cook. We 8 would anticipate continuing to use contractor assistance, as 9 well as using Staff resources in these reviews. ~ 10 MR. LEWIS: Could I try to quantify that? 11 I am really trying to find out where the I 12 responsibilities for safety are being distributed through i 13 this thing and, of course, the Staff has the final { () { 14 responsibility, we understand that. Actually, the 4 15 Commission has the final responsibility. ] 16 MR. WERMIEL: True. 1 17 MR. LEWIS: Where you say INEL is helping,you, in i 18 terms of manhours, are you speaking about ten to one, one'to 19 one, one to ten? + i 20 I won't hold you to number, I just want to'get a l 21 sense of it.
- I s
22 MR. RUSSELL: Having just gone through the 23 budgeting process for the next fiscal year yesterday, we i 24 have about $2 million in technical assistance to support in i 25 the area of digital I&C, both for advanced reactors and i (:) E ANN RILEY & ASSOCIATES, LTD. Court Reporters 1 1612 K Street, N.W., Suite 300 j Washington, D.C. 20006 (202) 293-3950 f
32 1 retrofits.to operating reactors. That is the equivalent.of-2 about, at round numbers, $200,000 per full-time laboratory 3-support. It is about 10 individuals. The branch is about 4 20 professionals. So it is about one-third contract ~ r 5 assistance;.two-thirds, in-house staff. 6 MR. LEWIS: That is more precise than I even asked 7 for,'and I thank you. =t 8 MR. WERMIEL: However, I would put a caveat on 9 that. That tech assistance includes -- a small portion of i 10 that includes the retrofit reviews, the vast majority of 11 that is the advanced reactor work, r 12 MR. RUSSELL: It is also involved withfassisting 13 us on development of the standard review plan. 14 MR. WERMIEL: Right. '15 MR. RUSSELL: So it varies, it is mixed in any_ 16 given functional area. If you talk advanced reactors, if 17 you talk operating reactors, development of standards. The i i 18 mix shifts between those, but it is about one-third 19 contractor /two-thirds in-house of total resources in the 20 digital I&C area. 21 MR. WERMIEL: The current estimate, as I recall,. 22 for FY94 for tech assistance for review of retrofits is less 23 than one staff person, less than one person at a lab. 24 MR. KERR: Well,.when you get a contractor to do a t i 25 review for you, you must give them fairly specific j i ( I ANN RILEY & ASSOCIATES, LTD. Court Reporters j 1612 K Street, N.W., Suite 300 washington, D.C. 20006 (202) 293-3950 i
l 33 l 1 1 instructions as-to what to look for? 0 2 MR. WERMIEL: Yes, we do. j 3 MR. KERR: -Is that generally available so that a 4 licensee knows what it is that you are looking for? 5 MR. WERMIEL: Yes. 6 MR. RUSSELL: All of the standards that are used, 7 the contracting documents, et cetera, are all publicly 8 available. I 9 MR. KERR: No. I am not talking about the 10 standards, I am talking about what you tell them to look for 11 and how to make judgments. 12 MR. WERMIEL: We do tell them what to look for i 13 based on previous reviews and based on our understanding of }_ 14 what the issues are, and we work very, very_ closely.with the 15 contractor. In most cases, what we are asking.the l 16 contractor to do for us is take what we know to be important 17 and apply it to the specific document-that they are 't 18 reviewing, and then we deal with them or we work with them 19 to understand what it is they have found and ask them t 20 questions about the result of their review and then pursue' t i 21 issues with the licensee as appropriate after that; 22 MR. LEWIS: Just to continue for one moment on a 23 politically incorrect track, this is clearly a growth l 24 industry. How is the hiring business going in terms of-l o -25 digital people? j ~. ANN RILEY & ASSOCIATES, LTD. ' Court Reporters 3 1612 K Street, N.W.,' Suite 300 Washington, D.C. 20006 (202) 293-3950 i
34 1 MR. WERMIEL: I am close to' gaining a,new person j 2 on my staff with digital systems design experience in the 3 nuclear industry. I hope he will be coming on within the 4 next couple of months. We are in the process now of' 5 finalizing the offer to him. 6 MR. LEWIS: This is a person with digital design' 7 experience in the nuclear-industry? 8 MR. WERMIEL: Yes, in the. nuclear industry. 9 MR. LEWIS: Who, in turn, came from? I i 10 MR. WERMIEL: He was with a consulting firm who 11 worked for nuclear utilities and previously had worked for 12 Lawrence Livermore National Lab. i 13 MR. LEWIS: I am trying to find out whether it is ( 14 a computer-type taught the nuclear business, or the other ( 15 way around. 16 MR. WERMIEL: Well, his previous work -- as a 17 matter of fact, at one time, he did work for the NRC Staff I 18 back in the '70s. It may be the other way around. 1 19 MR. LEWIS: I see. That answers it,.that answers 20 my question. 2. 21 MR. RUSSELL: Broadly,-I think you-are aware that 22 we are in an overstaffing situation and we are' attempting to-23' reduce total staff to get in line with.the Administration's 24 goals ~for reducing-Federal employment. So the actual' F 25 situation is that, for example, within the technical-staff
- O '
ANN RILEY & ASSOCIATES, LTD. i Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 35 l (} 1 that reports to me I have actually had to downgrade seven 2 Grade Level 15 positions and eleven Grade Level 14 3 positions, and so I am overstaffed in a number of areas, but l t 4 I have been able to keep ones and twos open for individual 5 areas, and I have received support-for keeping open the 6 hiring capability recruiting activities. If you see the i 7 right person with the right skills, we have been given l 8 authorization to pursue that individual. But, in general, i 9 we are in a staffing attrition cutback mode right now. i 10 MR. LEWIS: I know that. There was some great l 11 physicist who once said that you should rejoice at a 10 i 12 percent budget cut because it gives you an excuse to fire 13 the worse 20 percent of your pecple, but I know it is not () 14 that easy. As you know, I have particular interest in this i 15 subject. ] I 16 MR. RUSSELL: The agency has chosen not to use a i 17 reduction in force or involuntary separation, so it is being j 18 done by attrition and reassignment. 19 MR. LEWIS: No, I understand that. That fus always l 20 very hard. We at the university have the same problem, ACRS ( l 21 has the same problem. But, even so, you said when a person l 22 with the right qualifications comes along -- but, on top of 23 that, we are actually searching for such people? 24 MR. RUSSELL: That is correct. 25 MR. WERMIEL: Yes, we are. s l l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 l l .. ~.,. -.. -. ~ _ -
o 36 1 MR. LEWIS: We have made you-last much longer than 2 you were supposed to, but it is our fault. Is that the end 3 of yours? 4 MR. WERMIEL: That is all I had, yes. 5 MR. LEWIS: Let's give the industry a chance. 6 I will repeat to you, as I said to the Staff, that 7 if NUMARC and the Staff come to an agreement, that doesn't 8 convince me that it is the right agreement just per se. 9 MR. PIETRANGELO: I wouldn't be convinced either, 10 Dr. Lewis, just based on that. 11 MR. LEWIS: Hopefully, we are all in this 12 together. 13 MR. PIETRANGELO: Well, good morning. It is a () 14 pleasure to be back here at the ACRS again, and especially t 15 to see Dr. Kerr back. 16 MR. KERR: Should we believe everything else you 17 say? 18 [ Laughter. ] 19 MR. PIETRANGELO: I am going to put in a. plug for 20 another activity we have. You wers talking about 21 " threshold" and the misuse of that term and how it is 22 interpreted, and I will just give you heads up. NUMARC just-23 started a Regulatory Threshold Working Group that is'looking 24 broadly at the applications of PSA technology'to the 25 regulations and the design and operation of plants and also 3 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
1 37 1 how that fits in with the Commission's safety goal policy t 2 implementation, f f 3 I also have this issue where threshold has been'a- [ 4 word used and bandied about. So get used to it, you are? 5 going to hear more of it, and hopefully we will be able to-l 6 define some terms better so that. we-are all speaking from 7 the same page. 8 MR. KRESS: When you think you have an opportunity 'i 9 correct the words PSA to PRA, when you get a chance to. 10 MR. LEWIS: Yes. That is another issue, l 11 When you speak of defining things well enough to 12 all be speaking of the same things, I-don't know about your 13 experience in life, but my experience in life has been that-it is much easier to come to an agreement if you don't 14 15 define the terms very precisely, because then each party to q 16 the agreement interprets it in his own way and you can get j 17 wonderful agreements with people that way, t 18 So your precision of definition and actually 19 getting agreements are antithetical activities, bear it in 20 mind. 1 i 21-MR. PIETRANGELO: Okay. j i .22 MR. KERR: Don't pause, when you get,a. chance keep" ] 23 going. j 24 MR.. PIETRANGELO: Okay. j 25 First, I.just want to start.with, we really like ANN RILEY & ASSOCIATES, LTD.
- Court Reporters 1612 K Street,- N.W.,
Suite 300 Washington, D.C. 20006 (202) 293-3950
38 1 this issue at NUMARC because we think we are addressing it j 2 proactively with the Staff. We said this at the Regulatory j i 2 3 Information Conference. It is a lot better to be able to 4 sit back and rationally try to decide what are the proper l 5 standards and processes for implementing these digital = 6 upgrades at plants, rather than have to go about this after 7 some kind of event associated with a digital upgrade where 8 the atmosphere might be a little bit more charged. So we y 9 think the effort has gone pretty well thus far. . i l 10 We have been working with the Staff in earnest, I i 11 think, since last September, trying to reach some i 12 agreements That is going to continue into the future. I I 13 think what gives us optimism, and I think this has been ) 14 stated by the Staff a number of times and occasions, is that 15 everyone recognizes the benefits of this technology and its. j t 16 potential to not only enhance safety at our plants, but also 17 provide some benefits to the performance at the plants as 1 5 18 well. With that in mind, we are going to continue to work 19 real hard. l 20 We are kind of catching you in the middle of this I 21 process. We are, by no means, at the end of the road yet. j \\ 22 I will get into what our schedule is and what we plan to do_ i 23 in the next couple of months. I am sure we will be back [ i 24 here again, hopefully later.this year, to tell you about i 1 25 where we ended up with this. ANN RILEY & ASSOCIATES, LTD. j Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 --=,, -.-- - c
39 1 To start with a little overview of what I want to 2 talk about today, the background, who is on our committee, 3 what the role of that committee is, and how EPRI and NUMARC 4 are working together on this project. We will talk a little 1 5 bit about the guidance document, the draft guidance, that we 6 sent out this April to industry and NRC for comment. 7 I believe that the copy that you have seen is the j 8 marked-up NRC version. Most of the words we had in the 9 original draft are in there. I think we did send another 10 chart out that wasn't included in the NRC's markup of that, l 11 and hopefully you have had a chance to look at that. l 12 I will summarize the comments we have received 13 from the industry and the NRC. We did get comments from 14 over 30 of our member participant companies on this. I i i 15 think that reflects the interest in the industry on this j 16 issue. Typically, on other issues where we ask our members t 17 for comment, we usually get 20 to 25, and on this one, we 18 got 30 within a fairly short turnaround time. So there is a f 19 lot of interest in the industry on this particular issue. 20 Finally, we will talk about where we are headed f 21 with this revised draft guidance and what our schedule will ( 22 be. 23 We were asked at NUMARC early.last year by EPRI to 24 help provide industry's interaction with NRC on this i 25 particular issue. I think, as most of you know-now, EPRI j I O 1-ANN RILEY & ASSOCIATES, LTD. l Court Reporters j 1612 K Street, N.W., Suite 300 .) Washington, D.C. 20006-' j (202) 293-3950 _j l
40 1 has a large integrated I&C program plan that's real goal-is 2 to facilitate the use of modern digital technology in the 3 industry. 4 It is a much broader program than what we are 5 talking about here today, and they are looking at various ] 6 aspects of digital technology with some vision out to the-7 year 2000 of how a current operating plant would look and 8 how that digital technology would be integrated. That is a l 9 very important industry initiative that is moving forward i 10 and this is kind of one of the first elements, to try to l 11 establish some framework in the licensing arena for those 12 kind of upgrades. 13 We formed the committee in the Spring of last year } 14 to begin development of this guidance. We kind of got 15 sidetracked a little bit in the fall when the draft Generic l i 16 Letter came out. We used that joint committee to help us l 17 develop industry comments on the draft Generic Letter.' 18 Typically, we would form a NUMARC ad hoc advisory committee t 19 to do that kind of function for us, but we already had this 20 committee formed with EPRI that had several of our member I i 21 participants on it already, so we simply used that committee _ l 22 to help us draft comments. It was the same issue. 23 One thing the draft Generic Letter did do in our 24 process of developing comments, I think'we got all parties 25 sensitized to what the issues were. - There was, again, a ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W.,. Suite 300 Washington, D.C. 20006 I (202) 293-3950 i
v l 41 1 good response, I think, to the draft Generic Letter as far 2 as people sending comments directly to NRC. Really, after 3 the Generic Letter and comment process took place, I think i 4 we really got down to the table and started discussing the 5 issues here. 6 In-the primary role, this last bullet-is,_this 7 Committee is trying to develop the industry guidance on 8 licensing these digital upgrades. 9 This next slide is a little picture of how this l 10 fits together here. The top is this committee I have been 11 speaking of and it lists the utilities that are currently on 12 the group. I am not even sure that is complete. I think we 13 have been adding people as interest has grown in this over ) 14 the past several months. EPRI has some other activities 15 that we are trying to tie into this work, the EMI Working ) 16 Group that NRC has had interaction with and a V&V Working 17 Group. 18 At the bottom there, you see the industry 19 standards efforts. One of the purposes of this document is 20 to try to complement and provide a roadmap to a lot of the 21 existing relevant standards. We have tried. There-is, as i 22-you know, an upcoming revision to ANS IEEE P-7.4.3.2. We 23 are trying not to duplicate what is in that document. One l 24 our committee members is from Commonwealth ~ Edison, who is, I. l 25 think, the co-chair of that standards committee that is O l' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300-Washington, D.C. 20006 (202) 293-3950 =
42 1 working on that document. I know NRC has had involvement in 2 that also. We are trying to complement in the licensing 3 arena what is being done.through that' standards effort in 4 the design arena. 5 MR. LEWIS: I have a question related to the ones 6 I asked Staff just to.get some quantification, how big an 7 effort is the EPRI effort for you, for example, and in'any-8 units you like, man-months? 9 MR. PIETRANGELO: I don't know. t 10 Ray, maybe you can help us out.here? f 11 Ray Torok, from EPRI is here, and he can describe 12 that a lot better than I could. 13 MR. TOROK: I am Ray Torok, from EPRI. ) 14 MR. PIETRANGELO: There is a microphone over here, 15 Ray. 16 MR. TOROK: Yes. I am Ray Torok, from EPRI. 17 Are you are referring to the entire I&C upgrade l 18 initiative or just -- t 19 MR. LEWIS: Anything. If you tell me what the 20 numbers go with. 21 MR. TOROK: Just in terms of the overall. effort, a 22 it involves EPRI's Staff, about six or seven people full-23 time with various contractors supporting. So it is a many_ j -24, manyear effort over several years. 25 MR. LEWIS: That is not the whole digital'
- )
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 l (202) 293-3950 1
43 1 business? 2 MR. TOROK: Yes. Now, as far as the licensing 3 guideline effort goes, it is, I suppose, a few manyears of 4 effort. 5 MR. LEWIS: That's fine. Roughly, I wanted to get 6 some feeling for the level of effort involved. That helps a 7 great deal. Thank you. 8 The second question, before you take that one off, 9 is the V&V Working Group. One. thing.I noticed in reading 10 all these proposed generic letters was the absence of 11 anything about V&V. V&V there, does that mean V&V as 12 defined by computer scientists or as defined by IEEE 13-documents which describe the process, which V&V are we 14 talking about here? 15 MR. PIETRANGELO: I believe it is both process and 16 substance. But, again, Ray is in a much better position to 17 answer that. I haven't attended any of that particular 18 working group's meetings. 19 Ray? E 20 MR. TOROK: Tony, your answer is exactly right, it 21 is both. The working group itself involves primarily 22' utility representatives with expertise in this area and it 23 includes application to standards. In some cases, it means 24 interpreting standards for the purposes of the utility use. l' 25 They are working on what they are calling a handbook, which O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 c
44 1 would involve some interpretation of standards, how they 2 apply in utility situations, and best practices,.that kind 3 of thing. So it is really both. l 4 MR. LEWIS: Well, by both, you mean both of those, j l 5 But what the computer scientists would perhaps call formal l 6 V&V doesn't appear in that list? 7 MR. TOROK: No, it does as one of the areas that j 8 they are addressing, and some of the contractors involved in 9 this area are computer scientists with expertise in things 10 like that. 11 MR. LEWIS: I see. Okay, thank you. 12 MR. PIETRANGELO: Any other questions on this? l, i 13 MR. LEWIS: I apologize, but the pursuit of truth f (\\ 14 sometime takes us astray. 15 MR. PIETRANGELO: That's okay. 16 Here, we want to summarize, briefly, what we had 17 in the draft guidance document that went out for comment in 18 April. I think this has already been stated this morning l 19 the real purpose is to establish a stable and predictable 20 licensing framework. It is kind of being.done on a case-t 21 by-case basis up to now, where the last SER set the 22 precedent for the next particular application by some j 23 licensee. I think we would like to see something a little .f 24 bit more stable than that. I 25 You~will hear some presentations later this 1 \\ ANN RILEY & ASSOCIATES, LTD. Court Reporters l 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 i (202) 293-3950
4 'l 45 i i 1 afternoon-from utilities who are wrestling.with that, and I { O 2 think that will give you a better feeling for what is going. i 3 on at the sites with this. i 4 There is a secondary purpose here, and that is, i 5 regardless of whether a proposed digital upgrade results in-l ? 6 a USQ or not, we are trying to establish in this guidance 7 document a process for making sure that the right factors. 8 are considered, it is a roadmap to the relevant standards, a l r 9 and whether prior NRC review or approval is required or not-l F 10 that these upgrades are implemented in the safe manner and l 11 integrated into the current operating plant. 12 These next couple of bullets on intent, we didn't 13 think that we needed a change to the 50.59 regulation or 14 framework to be something special for digital. Maybe we 15 will talk about this a little bit later. 16 Our premise not only in our comments on the draft 17 Generic Letter, but within the intent of this guidance 18 document, is that the licensee has the authority under 50.59 19 to do an evaluation to determine whether there is an i 20 unreviewed safety question or not. l 21 Anything that prt mots or implies how that i 22 evaluation is going to come out I think would require a 23 change to 50.59. We have discussed that at length as a 24 committee, and I think there is a very strong consensus of 25 that position. We don't think it is a legal issue either, ANN'RILEY.& ASSOCIATES, LTD. Court Reporters l 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 l
t I i 46 1 l 1 and it never was. I think we want to get back to the 2 technical parts in.our document and put the legal stuff to i 3 the side. 4 MR. KERR: Excuse me. Go ahead, Pete. 5 MR. DAVIS: If you are contemplating a change, 6 let's say, and you decide that you do meet the requirement 7 for an unreviewed safety question on the basis of 50.59, how 8 much more work is involved in making the change? f i 9 MR. PIETRANGELO: Ideally, there shouldn't be any l 10 more work involved? l 11 MR. DAVIS: But what is the actual case? 12 MR. PIETRANGELO: The actually case is you are l 13 kind of open to a Pandora's Box on the tail-end, if it's a i ) 14 USQ. I know the staff is working on this and we are working 15 on our document, to try to establish what the groundrules l 16 are for those kind of things. I think there is an SER that \\ 17 did set some precedent from the Zion Eagle-21 review. There 18 are. things out there. There are a lot of standards:out i I 19 there. i 20 MR. DAVIS: I had gotten the impression that: 50.59 j 21 is a substantial disincentive to making any changes. You 22 are telling me that that may not be the case? 23 MR. PIETRANGELO: In a utopian world, it wouldn't j 24 be. Unfortunately,-it is to some degree a disincentive. 25 When there is regulatory risk associatedLwith the review,. it. (:) I Idai RILEY &' ASSOCIATES, LTD. l Court Reporters i 1612 K Street, N.W., Suite 300 ( Washington, D.C. 20006 (202) 293-3950 1 ~.... m,. m A
47 1 does provide a disincentive. We are trying to eliminate o 2 that risk. We are not saying that there is not a legitimate I 3 reason for the staff to do the review, but at least, if they 4 do, do the review, you know what to expect. i 5 MR. DAVIS: They even do the review if you don't 6 trigger the 50.59 limits, don't they? 7 MR. PIETRANGELO: There are options to do that. l 8 Yes, that is their prerogative. 9 MR. DAVIS: All right. Thank you. 10 MR. KERR: You say, in the bullet, that you are 11 going to stay within the existing 50.59 framework. Do you j i 12 understand what 50.59 means, or is it just nice to have i 13 something that ambiguous that you stay within? f ) 14 I must say, I don't understand the language in f 15 50.59. 16 MR. PIETRANGEL'O: Some days I think I understand 17 it, and other days, when I hear another issue brought up, it 18 may be less clear. Now, when we talk 50.59, I think you 19 have to look at the next bullet also, NSAC-125. It was an 20 extensive industry effort with EPRI and NUMARC to provide l 21 clarity and guidance on 50.59 that NSAC-125 provides. Just 22 about every licensee has incorporated the practices 23 described in NSAC-125 into their 50.59 review process. 1 1 24 Even though there-is no formal endorsement by NRC 25 of that document, I think it.is recognized that that L O' 7JRJ RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
48 i . f-w 1 document is useful and that licensees are using it. I think -(' i 2 the problems associated, before NSAC-125, was the 3 implementation of the mods under 50.59. We are not hearing t 4 feedback from our membership that there is a problem to be-5 solved out there with this. l 6 MR. KERR: Thank you. i 7 MR. PIETRANGELO: I think Jerry touched on before l I 8 that the other intent is trying to provide a roadmap tofthe 9 relevant standards out there. There are a lot of standards i 10 on digital technology, and it is just the application of 11 which to what situation, I think, that maybe our document l 1 12 will fill some void there. 13 The main issues we touched on in the March draft { t 14 were really -- I would characterize them as the hot button l ) 15 issues that have come up in some of the previous reviews, I 16 and those were as follows. 17 -We did include a chart in there -- I was going to d 18 use a slide to walk you through that chart -- on common-t 19 mode software failures. That is the one_that was sent_out i 20 to you afterwards. We decided not to do that because that I 21 chart will not be in the revised draft guidance, it will;be-22 modified. But I think it is useful to talk a :little bit .l -23 about what our intent was with that chart. 24 When we had our first meeting with the Staff, i 25 after we sent the document out for comment, we scheduled a i ANN RILEY & ASSOCIATES, LTD. ) Court Reporters 1612 K Street, N.W., Suite 300 -Washington, D.C. 20006 [ (202) 293-3950.
e r i 49 j 1 meeting with NRC to try to give them a better understanding i i 2 or heads up before they got into the detailed review of the L 3 document. I remember that meeting very well. We put up l 4 that chart, and that's the only thing we talked about for 'l-5 almost two hours, what do you mean by this in this chart and. -f 6 that. 7 There were three blocks when considering common-8 mode software failures. The first question we asked was, is 9 a common-mode software failure credible for this particular 10 upgrade? i 11 The second question we asked is, what is the i 12 probability of that failure occurring when that function is i 13 needed in its service in the plant? () 14 The third block was whether the existing licensing [ t 15 basis or SAR analysis and SAR bounds, the failure modes'that i 16 you have determined, have they already been analyzed? l 17 There was a lot of discussion on credibility, and 18 I think you saw it reflected in the Staff comment on our i l 19 document. 20 What we are trying to do is get credit for simple 21 modifications. I think, Dr. Lewis, you gave lui example of 22 the three-line code at a previous ACRS meeting. What we are 23 trying to get recognition of there is that there are some 24 very simple mods that are done in our industry, okay, where 25 we believe that a good V&V effort would make the~ potential O I JdRJ RILEY & ASSOCIATES, LTD. Court Reporters i 1612 K Street, 14.W., Suite 300 L Washington, D.C. 20006 (202) 293-3950 1
f 50 'l for a common-mode failure improbable to the point of being w= 2 incredible for us a. simple mod. That is the only point we t 3 were trying to make in that block on the diagram. 4 i 4 MR. LEWIS: Is there anyone who denies that, you i 5 know, that there is code simple enough to set that failure 6 is improbable? { 7 MR. PIETRANGELO: I don't deny it. I don't deny 8 it. But I think the confusion was, we can't guarantee f 9 error-free software. We can't guarantee risk-free plants, 10 either. It is in that context. ] 11 MR. LEWIS:.Sure. I can't guarantee that that 12 ceiling won't fall on you as you say all this, but let's not i t t 13 proceed as if it is going to. () 14 MR. PIETRANGELO: Right. t 15 MR. DAVIS: That makes it credible, then, right? 16 MR. LEWIS: That's right. Well, thatnis'the issue i 17 we keen going around. I 18 MR. PIETRANGELO: That was our intent ' with' that-j 19 block, just to'say, are there things that simple where we 20 think good V&V would preclude the potential for common-mode ] 1 21 failure. 22 The second block dealt l.with probability and the- ~
- 23 example I like to use.to explain our intent here-is, let's-.
24 say, the diesel sequencer, if someone wanted to do a digital upgrade to a 'iesel load sequence, when is'that component-or d 25 ANN RILEY & ASSOCIATES, LTD. Court. Reporters 1612 K Street,'N.W., Suite 300 washington, D.C. 20006 (202) 293-3950 - = =
51 I 1 part of that system needed? 2 It would be needed after you had a large break 3 LOCA coincident with a loss of.all off-site power, and then, 4 on top of that, you would postulate the common-mode failure l 5 in that'particular situation. It-is, in series, looking at 6 the potential for these things happening, t I 7 What we are trying to incorporate, I think, with 8 our upcoming draft is more of a focus on how the upgrade-9 impacts the system operation, when is that function } 10 required. I think that envelopes the safety significance 11 question-maybe better than just trying to establish things 12 up-front because the plants are different and the functions 13 of these things, depending on where you use them in the l( ) 14 system -- 1 1 15 MR. KRESS: So really you are not looking for the 16 probability of failure, you are assigning that a probability 17 of one and seeing what the safety significance of that is? I 18 MR. PIETRANGELO: Well, we are looking at when it 19 would be required to perform its function, not necessarily 20 assigning it a probability of one. ) I 21 MR. KRESS: Well, you are'saying that you are 22 going to postulate it will fail when it is called upon. i 1 23 MR. PIETRANGELO: When it is needed, yes. I mean, 24 that is when it is important. We are going to try to i 25 integrate other factors in that determination, rather than l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 n
52 I 1 postulate a probability of one, what is the performance 2 history of the thing you are trying to install, what is the 3 level of V&V you can do and establish confidence that the 4 thing will perform its intended function, what kind of 5 factory acceptance testing was done, how much verification 6 testing can you do in the field? 7 All we can do on an industrywide, generic level.is 8 give the main factors to consider, and it is a process. We 9 are not going to answer the questions for individual 10 licensees, because the premise is that is their authority 11 under 50.59 to do so. All we can do is lay out the process,. 12 hopefully, and working with NRC try to get this process and 13 the relevant factors to consider that lead people to make I 14 the right decision, rather than predetermine what the right 15 decision is. To me that is a challenge, and we are working 16 hard at it, but that is our intent. 17 MR. DAVIS: I thought the load sequencers were 18 needed any time you had a loss of off-site power, you don't. 19 need a large break LOCA. 20 MR. PIETRANGELO: I don't believe so, at least in 21 the plants that I worked on. 22 MR. DAVIS: What about small break LOCAs? 23-MR. PIETRANGELO: I am not sure. I will have'to l 24-get back to you. 25 MR. DAVIS: Thank you. 1 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612.K Street, N.W., Suite 300 Washington,~D.C. 20006 (202) 293-3950
~_ t e 53 1 MR. PIETRANGELO: The last block that I wanted to 1 2 talk about in that chart was, I believe it says, is it 3 bounded by the current licensing analysis. Really if.you 4 determine that you have some new system-type failure modes, 5 then you wouldn't be bounded. But if it was previously 6 analyzed, and all the plants are different so it is hard, 7 again, to say generically or across the industry who is 8 going to be bounded by what, what new plants may have 9 analyzed for some types of these things, we don't know, but 'I 10 we believe that there should be an' element in that process 11 that says if it has already been analyzed from a system-12 level perspective on its impact, then it is not a USQ. 13 These other issues, primarily we talked about the ) 14 common-mode software failures and EMI. Again, these were 15 kind of the hot button issues that were coming out of the 16 reviews. 17 MR. KRESS: Does the presence of EMI on that list 18 imply that you think there is some serious safety concerns 19 with the EMI? l 20 MR. PIETRANGELO: If it is done wrong, there would 21 be. What we are after, our intent anyway, there is a lot of 22 controversy about this, about site mapping and it is 23 expensive, and a lot of people think that we are just'out l 24' there taking data without'any real purpose. I think our ^ 25 intent is to try to get to some kind of generic bounding. .O ANN RILEY & ASSOCIATES,'LTD. Court Reporters 1612 K Street,'N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
F. >y 54 I n e~' 1 Jevel for the environment.that these things would work in. l i 2 That EPRI/EMI Working Group is focusing on that. They had a 3 meeting with the staff last week,'which I understand went 4 pretty well. 5 There is recognition that we don't have enough 6 data yet to be able to have'enough confidence in where that i 7 bounding level would be. I think there is work underway to-B try to get enough data. Based on the data we have thus far, l i 9 though, and where that level would be establ ished f 10 conservatively, it would' bound the current applications. l 11 That is where we would like to get, agreement on a 3 12 generic level for EMI environment, and then you would be 13 able to select with confidence the test you need to give () 14 yourself assurance that you are not going to have a problem. { l s 15 Clearly, if ic is designed and installed wrong from an~EMI { 16 standpoint, you could have big problems. 17 MR. LEWIS: I confess to never having understood 18 all the emphasis on EMI, because it is a subject for which j 19 the cure, is like aspirin for a headache. The ideology of [ i 20 headaches is extrcmely difficult to understand -- it is one i 21 of the classic difficult medical problems -- but the cure is !l 22 relatively easy with an aspirin. The same thing is true for i 23 EMI. You can gather data out of your kazoo but, in fact, 4 24 the cures for EMI susceptibility are relatively easy to 25 implement and, in fact, ought to be part of good electrical-ANN RILEY & ASSOCIATES, LTD. t Court Reporters 1612 K Street, N.W., Suite 300 I Washington, D.C. 20006 Y (202) 293-3950 l
b A. A-s n 4 .a e 4" as w-
- .A+*
N-spmn-r i 55 De~ 1 practice anywhere. \\ .1 0 2 So I have never understood all this data gathering 3 and controversy about you shield everything, you close the j 4 doors, and inhibit people from using. radios inside,Jand~then F 5 you are in reasonably good shape. + 6 MR. DAVIS: The military has done it. i 7 MR. LEWIS: Pardon? i 8 MR. DAVIS: The military has done it very - l 9 effectively. 10 MR. LE' DIS: Of course, they have. There is no 11 need for a research program. That is the reason I asked the 12 question. 13 Bill? 14 Bill wants to respond. Let's give him a chance'to 15 fight back. 16 MR. RUSSELL: What I would like to suggest is that i 17 we arrange for a-closed briefing on some material regarding i 18 EMI vulnerability in current nuclear power plants to j 19 identify that while it is simple to provide-shielding, 20 wiring does end up acting as antennas, et cetera. So the 21 question of the adequacy of.the implementation of shielding 22 is the issue. We agree that it is essentially a problem of 23 shielding and how you implement that shielding or using 24 hardened components that are not susceptible to EMI. But 25 the issue is, what exists? 1 l ANN RILEY & ASSOCIATES, LTD. Court Reporters 16121K Street, N.W., Suite 300 Washington,-D.C. 20006 (202) 293-3950
i I 56 1 While, in theory, you can say, "Yes, this is a 2 relatively straightforward problem to resolve," it has'not 3 always been resolved well in the past. There has been work l 1 done on that. It is sensitive information, but we could 5 arrange to brief you on that in a closed session. j 6 MR. LEWIS: Bill, I have a strange and 7 indefensible history, but it includes an awful lot of that 8 stuff.. The military interest is in two general areas. One-9 is a jamming environment, which is sort of active EMI, and l 10 the other is a nuclear explosion environment which in 11 principle we are not really concerned.about in this 12 business. You have to be careful about overreading milit'ary 13 documents that stress the difficulties, because they are l } 14 really talking about a different class of problem. If it is 15 sensitive we will go into it at a sensitive level. 16 I remain convinced that the cures are a lot easier 17 to find in this business. Even though I am well aware there 18 are plenty of places with corroded ground leads and things 19 like that -- I know that very well -- they should be fixed 20 anyway, and you don't need research to do it. 21 MR. RUSSELL: We agree. We don't think that the 22 solution is difficult. The issue relates to what exists now 23 and how well have they assessed new designs that.they are 24 putting in for vulnerability because there are EMI sources 25 in power plants. Some of them are stray sources, welding O 1 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i P 57 i 1 activities, things-like that that can go on, that can be of l l 2 concern. 3 MR. LEWIS: I know that. I am only reacting to 4 the fact that in staff lists it always appears as one of the 5 two or three most important subjects. I, frankly, wouldn't-l 6 put it that high on the list, but we will discuss that. 7 MR. WILKENS: Hal, would you like to have such a l ^ 8 sensitive briefing from the staff? 9 MR. LEWIS: Sure. 10 MR. WILKENS: The reason I raise the question is 11 because, Bill, you will recall you offered a somewhat 12 broader, sensitive briefing.. 13 MR. RUSSELL: I would suggest the two could be ) 14 done at the same time. 15 MR. WILKENS: They could be done at the same time. 16 MR. RUSSELL: We will, in fact, brief.on the 17 standards that are being applied and the processes being i 18 used by foreign regulatory authorities, as well as. briefing 19 you on the EMI sensitivity as it relates to what was found 1 20 by some evaluations of existing nuclear power plants. 21 MR. WILKENS: I had asked John Larkins to get l 22 together, or at least to have somebody get together with you j 23 and try to schedule this other one, and if we can include 24 them together, I think that would work out just fine. 25 MR. LEWIS: While you two are negotiating over my ANN RILEY & ASSOCIATES, LTD. t Court Reporters 1612 K Street, N.W., Suite 300 i Washington, D.C. 20006 (202) 293-3950 L i
58 i ~ 1 pay grades, I will go along with whatever you decide. l 2 [ Laughter.) 3 MR. RUSSELL-There was an earlier question that i 4 possibly we can help with, and that is the comments on the i 5 load sequencer. That would actuate anytime there is an i 6 engineered safeguard feature signal present when you have a i 7 loss of power to the bus, so any transient that would cause 8 you to have an ESF actuation, whether it be an overcooling 9 transient, a LOCA, et cetera, anything that would cause,. l t i 10 essentially, a contraction or a loss of inventory in the i 11 primary that would give you an ESF actuation, when that is i r 12 combined with a loss of power on a particular safeguard bus, i t i 13 it would give you the start signal for the diesel and the () 14 load sequencing. 4 15 MR. DAVIS: That is my understanding. ] 16 MR. PIETRANGELO: Thank you. 17 This next slide is one to summarize the comments 18 we received on that draft that we sent out for comment. As 19 I mentioned before, we received about 30 different comment l 20 letters from our members and participant companies. In i 21 general, the guidance was welcome. People recognized that 22 we were serving a function here and trying to get some 23 agreements established in the regulations with regard to 24 these upgrades, so everyone encouraged us to continue to 25 move forward with this. i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 l (202) 293-3950
i i 1 59 1 The second major comment was, our March 1993 draft l 2 tended to focus more on the large integrated system-type l 3 change.like Eagle-21, and the point made in the comments 4 was, "Geez, there are a whole lot of other classes of 3 5 upgrades that maybe would not apply to what you are trying 6 to say here." I think we will see in the next draft -- 7 first of all, we put in an applicability statement that it i 8 will cover both component level upgrades as well as'large-i 9 integrated systems. 10 A third main comment was to try to' focus more on 11 system-level failure modes. A lot of people had trouble l 12 distinguishing in our document between some~ kind of accident 13 sequence or event versus what we were talking about with the 14 failure of the software. A focus on system level would help 15 to make that distinction clear in the guidance. We are 16 defin-aly going to focus more-on that in the upcoming 17 revision. 18 Finally, we didn't have a whole lot in there on 19 commercial grade item dedication, and there was a request to 20 provide more. This is an area that I think we still need to 21 give a lot of thought to. There are some guides out in the 22 industry, not particular to digital, I am not sure in my 23 mind whether there needs to be something specific to digital 24 on this particular issue. This is also mentioned in some of 25 the -- I know the 7.4.3.2 provision has something on CGI O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i I t + 60 1 dedication in there. To the extent we address'this more, I' 2 think we are still debating internally. i 3 Jerry went through the NRC comments this morning. { i 4 I just'want to say we saw a lot of positives in the comments 5 that we received, particularly if something had been r 6 reviewed before that the level of review by the staff would 7 focus more on plant-specific-type issucs associated with. 8 that change. I think we are_already starting to see that a 9 little bit in the industry. That is a welcome change, _ i 10 because we weren't seeing it before. 11 You know, the other comments focused on the need 12 to do a local mapping, to establish the EMI environment. We 13 were a little disappointed in the discouraging tone with i'l ) 14 regard to using the commercial grade items. I. don't think. 15 we want, as an industry, to foreclose on the option of using 16 the technology commercial grade in our plants and drive 17 people out of the marketplace. There is not a hell'of a lot 18 of market for digital in the industry as there is now, and l 19 it is very specialized at this point. 20 I think as this replacement of analog systems and 21 components continues this is going to-accelerate. As an 22 industry, we would like to have the option to use commercial 23 grade items. Now, where you use them and how you use them I 24 think is very important, I think you can use those 25-commercial grade products in a way that would still provide i 1 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
a n 61 1 you reasonable assurance, depending on where and how it is i 2 used, even if you can't get at the source code and all that 3 other business. 4 Finally, there was a comment on functional .j 5 diversity. We did not mention this in our March 1993 draft, 6 an there really wasn't a whole lot that was in the comment 7 besides listing this as a bullet. We are aware that it is t 8 an issue at a plant today that is going through the i 9 licensing of Eagle-21. We have someone from PG&E on the l 10 committee. l l 11 I think this is a generic issue, and we are going 12 to help maybe try to define what is meant here better in the 13 next draft, and then again discuss it with the' staff and see (( ) 14 if we can all reach some common understanding of what that 15 means. Please don't ask me what I think it means, because I j 1 16 don't have a good understanding of it at all right now. j 1 17 Finally, what are we going to do with this next 18 draft. 19 MR. LEWIS: Leave it on. i 20 on the commercial grade thing, there seems to be } ) 21 an undertone through all of this that commercial grade l 22 equipment is sort of de facto inferior in reliability to j 23 really custom-made equipment. Military experience has been 24 not that way, that is -- and, in fact, many people's personal experience -- that when you use commercial grade 25 . O-I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 '(202) 293-3950 l
t 62 j 1 equipment that has been out in the m.llions, it has been T 2 more thoroughly tested than any spec you can ever lay on a j 3 thing. If you use it reasonably, wisely, you are better i 4 off. 5 The kids in our laboratories at my school 6 nowadays, essentially, never build anything because it is .} 7 easier to take a commercial microprocessor and adapt-it to 8 the job that they have to do, and it is a more reliable j -i 9 system in the end because these things have been built in 10 the millions, in some cases hundreds of millions. l 11 MR. PIETRANGELO: Right. q 12 MR. LEWIS: The word " commercial" isn't a dirty -\\ 13 word, that's all I am saying. l) 14 MR. PIETRANGELO: Based on the comments we have l 15 received, we are going to revise the draft. We met with-the 16 Staff to get their comments, and the following day we had a-17 committee meeting where we looked at both the NRC's comments 18 and all the industry comments we received. 19 Our general conclusion was our focus was too 20 narrow in the original draft, where we were focusing more on 1 21 the hot button issues instead of looking at how digital 1 22 technology is integrated within the system of a nuclear R 23 power plant. 24 Our approach in the revised draft is to focus more 25 on the system-level effects. Again, we are going to cover O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 ) (202) 293-3950 i . ~.
63 not only large integrated systems but have guidance that 1 '1 2 applies to component-level upgrades,_but in the analysis of 3 them look how they impact the system and its safety function l l 4 and what effect that has on the plant. That, in essence, is l 5 the tie to the licensing framework under 50.59. ~ 6 Maybe what we have been missing.thus far is a good 7 focus that ties the design, how it is integrated into the r 8 system with those questions asked under 50.59, and what [ 9 things you need to consider in the design, the 10 specification, the implementation, testing and verification. 11 We briefly discussed this yesterday. The full 12 committee was just getting ready for its presentation and 13 putting together some charts on how this fits with that. 14 The bullets we have listed here, I think we are getting more 15 comfortable with how we can address these things within the 16 context of the system. Again, that is your tie to the 50.59 17 process. 18 Again, the intent is, hopefully, with the l 19 agreement of the staff to lay out a process that gives you 20 the factors to consider, what is important to focus on, that i 21 lead you to make the right decision on whether it is a USQ 22 or not, and regardless of whether it is a USQ, whether you l 23 did a good job of looking at all the things you need to look 24 at to implement that mod at your plant. 25 Our purpose hasn't changed. I think our approach ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 j Washington, D.C. 20006 (202) 293-3950
64 1 is going to change a little bit. Because we think the- - (:) 2 document will change substantially, we are going to send it j 3 out for review again, and that is currently scheduled for 4 the first of September. We are meeting in mid-August, and I 9 5 think we will try to get a meeting in with the Staff right 6 after that meeting, or at least early September after we 7 have sent you the thing. It will be a shorter turnaround I 8 time, about a 30-day comment period, because we want.to 9 finish by the end of the year. j 10 That is our schedule, to finalize the document in -j 11 November, and then hold a workshop after that and provide i 12 the industry a detailed understanding of the thought that 13 went into it, the process, and give some. examples. That is I 14 how you establish the threshold. The guidance, the process i 15 guidance, with the premise that the licensee has the right 16 under 50.59 to make that decision with NRC oversight through 17 the normal regulatory channels, that is the way to go on a 18 this, and that is what we are going to try to do. 19 That concludes my prepared remarks. 20 MR. LEWIS: I have sort of scattershot questions. i 21 Just looking through the guidelines document, it has gone i 22 back and forth and I think that is wonderful, but the result 23 is that I don't know the genealogy of any particular words 24 that I am hearing, and that is okay. 25 A couple of places in skimming through it on the t IJUJ RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
p i 65 1 airplane coming in last night, I found myself reading about 2 " reliability," so I looked under definitions and terminology i 3 to find out what " reliability" means, and it isn't in the -i 4 definitions and terminology. It is a subject without a 5 definition which is extremely important. That is not a 6 question, it is just an observation that it would be nice to i 7 put it in. l l 8 I also was interested to notice -- this is also in -{ l 9 the definitions under computer -- it says "See programmable 10 digital computer." I wrote "why" next to it. Why does it 11 have to be programmable to be a digital computer? i 12 The firmware in my printer has a bug in it, and it i 1 13 is not programmable, but it sure.is'a bug and it makes i 14 trouble. I wonder about a lot of these distinctions'being 15 made without people thinking through what they are.
- Again, 16 that is not really a question, it is just an observation, j
17 MR. PIETRANGELO: But I agree with your general ) 18 direction here. We tend to focus on little things on this 19 issue related to software and lose sight of how that thing 20 is used at the plant within the system and what effects it 21 might have it fails. That is the approach we_are going to I 22 take, to try to look at it and its impact on the plant, j 23 rather than every bugaboo that could possibly happen with ) 24 software and digital hardware. 25 MR. LEWIS: Also, on the airplane last night -- O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 29.3-3950
1 P 66 i 1 and it isn't fair to dredge out something that is two year's i 2 old -- but also Doug Coe, to his credit, put together a j i 3 marvelous package for me to read on the airplane. It was l t 4 even light enough to carry on an airplane, which is no small l i 5 achievement. t 6 This is a letter, and I won't even mention who 7 wrote it, but it is from NRC to a certain utility, and it i i 8 sort of keeps mixing up what is a computer, what is a 9 microprocessor, what is software, what is hardware. It I 10 speaks of plant safety analyses that were performed prior to. { i 11 the use of microprocessors. 12 Microprocessors, of course, are a late development 13 in the computer business. Having been personally involved ) 14 in a computer that was built in 1949, which didn't have a ~ t 15 microprocessor in it but it sure had a lot of bugs in it, 16 you know, there is a long' history here. 17 The same letter speaks of microprocessor failures, 18 and actually I know of very few microprocessor failures. I I 19 know lots of computer failures. I believe the last. issue of I 20 Computer, I think, had a list of all the railroad failures 21 that have occurred that they were able to put together. It 22 was a very interesting list because half of them had to do 23 with humans bypassing controls that had been built into-the 24 system, and then, of course, it is called a computer 25 failure. We all, I am sure, get bills for a million dollars-I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
67 I j 1 and'then someone says, "It was the computer." 2 Who else has questions, real questions? l 3 MR. KERR: In your last list, you talk about 4 providing the framework for defense in-depth evaluation. It i 5 is my memory that defense in-depth came into existence' I 6 before there was very much recognition-of capability of 7 doing quantitative reliability evaluations. It would seem 8 to me that today the consideration of defense in-depth ought 9 to take that into account, and the amount of defense in-j 10 depth and how you do it should be based on it. I would i 11 expect that you are doing this anyway. f 12 MR. PIETRANGELO: Yes. The way are going to i 13 structure it is, if you can't assure yourself with'a 14 particular upgrade orican't establish the confidence that j 15 you can't answer those questions no, no, no, on a 50.59 j 16 evaluation, you are in USQ space at that point and you need 17 to look at other systems in the plant from a defense in-i 18 depth perspective for performing those functions, assuming. } l 19 that that upgrade you installed failed. That is what we are ? 20 trying to get at there. 21 MR. KERR: Thank you. I 22 MR. LEWIS: Any other questions? 23 [No response.] 24 MR. LEWIS: Thynk you very much, Tony. l 25 MR. PIETRANGELO: Thank you very much, Dr. Lewis. t l 6 ANN RILEY & ASSOCIATES, LTD. Court Reporters l 1612 K Street, N.W., Suite 300 l Washington, D.C. 20006 (202) 293-3950 l
. ~ _.... _ _ _ _ _.. _ P '3 ~68 f' . j 1 MR.. LEWIS: We are running only 15 minutes'behind ~ 2 schedule, but, by golly, we will take a.15-minute recess at 3 this time, f i 4 [ Recess.] I 5 MR. LEWIS: Let's reconvene our meeting, and we i 6 are going to go on with some industry: perspectives, that'is 7 my understanding. 8 We are all yours. 9 MR. BREWER: My name is Steve Brewer, and I am l 10 Group Manager of Nuclear Safety and Licensing for American 11 Electric Power. I have some brief introductory remarks 12 before our principal speaker takes the stand. q 13 Just by way of orientation, American Electric { 14 Power is the parent company of Indiana and Michigan Power, i l 15 which in turn owns D.C. Cook, a two-unit, pressurized water l 16 reactor in Southwestern Michigan. l 17 MR. LEWIS: Therefore, you recognize-the letter I 18 was reading from earlier. 1 19 MR. BREWER: Yes. t 20 [ Laughter.] 21 MR. BREWER: We began our analog to digital 22 upgrade for portions of the reactor protection system and l 23 some portions of the nonsafety-related systems in 1988. We 24 continued along until late 1991, when we had to redirect our 1 25 project, due to some hardware and some commercial concerns i 4 ANN.RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 i (202) 293-3950
I l 69 1 that were not related to-the digital upgrade. We' restarted l 1 4 2 our engineering and design on the reactor protection system 3 upgrade in early 1992 l f 4 We are currently in the fabrication testing phase l 5 of the equipment and are awaiting NRC review and issuance of i 6 a safety evaluation report for the upgrade. Our current 7 plans are to install both the nonsafety and the safety-l 8 related portions of the upgrade in the 1994 outages. It. 9 would be about February for Unit 1, and August for Unit-2. 10 For our project details, I want to introduce Bill i 11 Sotos, who is the lead engineer, for the upgrade and he will 12 go through in much more detail our system, and where we are 13 headed with the project. ) 14 MR. LEWIS: We are gluttons for detail. 15 MR. BREWER: You are going to get some. 16 MR. SOTOS: I would like to thank you for inviting i 17 us out here. It is a pleasure to be here. I i 18 What I am going to go through, and I do have a lot 19 of material to cover in the time, so'what I am going to go 20 through is to talk about the scope of work basically as 21 background. Hopefully, I will get through that fairly 22 quickly, so that we can get really the focus of-what this 23 meeting is concerning, which are the major tactical issues i 24 and major regulatory issues that we have involved ourselves l 25 with since we got into this project. 3 O i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
70 ] 1 With regards to the scope of work, I want to talk. 2 about overall application and design concepts that we i 3 undertook when we started the project. Then we will break-l 4 it down into both the safety-related portion and the 5 nonsafety-related portion of the work. 6 The safety-related portion involves the reactor 7 protection system, and we used the Foxboro Spec 200 and the 8 Spec 200 micro product, which is the same product that was l'i 9 used at Connecticut Yankee. Then I will talk, in not too i 10 much detail, about the nonsafety-related portion which is 11 the control portion of the job, where we are using a Taylor 12 Mod 30 product. 13 These are the major concepts we wanted to stay i ( 14 with right from the very beginning of the project. We. I 15 didn't want to do any functional changes. We were not going 16 to be make any RPS or ESF logic changes, no changes to the t 17 field instruments or the existing field wiring. We are l 18 going to be using the existing process equipment racks. We 19 are going to be using strictly analog communication between i 20 the modules, and that will come into focus a little better 21 when I get into some of the details. 22 We will be doing extensive acceptance testing 23 prior to installation, which is the phase we are mainly into 24 right now, no technical specification changes, and we want 25 it to comply with existing design basis requirements for a I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202)'293-3950
71 1 plant, for instance, IEEE 279-1971. L 2 We also wanted to attempt to comply with some of 3 the later standards to the greatest extent feasible, at our 4 plant, for instance, IEEE 603-1980. 5 Just to kind of show you the layout of our 6 protection and control scheme, we have got this little 7 schematic here. It probably looks fairly familiar to you. 8 On the safety-related portion, which is everything from this 9 line on over, starting with the field sensors right here, l 4 10 'such as RTDs or transmitters, that sort of thing, we have 11 four separate channels, each have field sensors, like I say, 12 feeding into the process protection racks, such as right 13 here, here, here, and here. In reality, we have four racks () 14 here and four racks in the set, three in this set, and two-1 15 in here. 16 Once those signals are processed, then they go off 17 into Train A or Train B or Train A and Train B of the 18 reactor protection logic cabinets, and also they will come 19 up through isolation off to the control group cabinets or to 20 indicators, hand auto stations, that sort of thing. i 21 The crosshatching indicates the parts of the 22 system that we are changing with our upgrade and, on the l 23 safety-related portion, it is strictly equipment that is 24 within the reactor protection process cabinets. Then on the I 25 nonsafety-related side, it is strictly equipment located in f O f I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 I - (2 02) 293-3950
i 72 i 1 1 the control group process cabinets, as well as hand auto 73.u-2 stations and recorders, i i 3 As far as the safety-related portion, again'we are j i 4 using the Foxboro Spec 200 and Spec 200 micro hardware. We i 5 are replacing Foxboro H-Line equipment. Just to give you an } 6 idea what this stuff looks like, this is an H-Line module. l 7 I can pass it around, if you want to look at it, just to l 8 give you an idea of the level of technology that we 9 presently have in the plant. It is all analog. That i 10 particular device is an alarm unit that does a bistable i 11 function. 12 Then as a contrast for the Spec 200 micro 13 hardware, this is the only device out of the whole system- -i 14 that is a microprocessor-based device. We have about 80 or 15 90 of these within the system itself, and it handles 4 l ~ 16 inputs, analog. inputs, 2 analog outputs, and 2 contact i 17 outputs. It.is only a small portion of the entire-system. 18 MR. LEWIS: I have to say that this old one is my [ 19 kind of circuit. 20 [ Laughter.] t 21 MR. SOTOS: To give you some idea of how this l 22 stuff is actually installed, I have some pictures here. l 23' This is a top and bottom photograph of the existing 'i 24 installation, and then I have got two photographs of what -l 25 the new installation is going to look like in the racks t i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 .(202) 293-3950 i J
t t I 73 1 where they are staged at the factory. 2 As I was saying, this impacts 13 protection 3 cabinets in each unit, and it affects primarily reactor 4 protection emergency safety features and Reg Guide 1.97 5 Category 1 applications. I have some typical loops here 'i 6 listed, but I won't go through those. 7 Let me give you some sense of geography. This is 8 a layout of our control room. Here is the horseshoe where [ 9 the operators are, behind that horseshoe, where the cabinets 10 are located. We have the four sets of reactor protection j 11 cabinets located here, here is Set 1, Set 2, 3, and 4. 12 MR. LEWIS: Forgive me. 13 MR. SOTOS: Yes? ] ) f 14 MR. LEWIS: The U-circuit, which is lighter, I i~ 15 agree, has a prom in it and is the quality control on the l 16 prom in the same package as any software quality control? 17 MR. SOTOS: Yes. 18 MR. LEWIS: What happens to it? 19 Once the thing is in prom, of course, it is firm i 20 but it is not programmable anymore, right~-- it is i 21 programmable but it shouldn't be programmed? 22 MR. SOTOS: It is configurable, and that is the 23 term I like to use, I guess. The software is embedded in 24 the firmware, you are right. j i 25 MR. LEWIS: Yes. } h ANN RILEY & ASSOCIATES, LTD. Court Reporters-1612 K Street, N.W., Suite 300 Washington, D.C. 20006 i (202) 293-3950 l
-i ? 74 1 MR. SOTOS: As a user, there are only limited 2 things that we can program in or configure in, setpoints, i 3 constants, that sort of thing. } 4 MP. LEWIS: I understand. I am just wondering i 5 whether the controls that apply to the really programmable 6 part are also applied to the part that is programmed into i 7 the firmware or whether there is any functional separation? j 8 MR. SOTOS: Yes. Yes, at AEP and out at the site, I l 9 we are still implementing it obviously, but we will be t i 10 having strict configuration management of that data that 11 gets programmed in there. That all falls under our QA 12 program. I 13 MR. LEWIS: But you could reprogram the prom on- ] 14 site? ) 15 MR. SOTOS: No, I can't. .16 MR. LEWIS: You cannot? 17 MR. SOTOS: We cannot reprogram the prom. That 18 has to be the vendor that does that. 19 MR. LEWIS: Okay. Thank you. 20 MR. SOTOS: The safety-related portion of the job, 21 again this is the Spec 200 hardware, the basic system 22 consists of plus or minus 15-volt system power supplies in 23 bulk, 75-volt loop transmitter power supplies and those are .24 necessary because we are a 10 to 50-milliamp-based plant so 25 we need the higher voltage to drive our transmitters, power () ANN RILEY & ASSOCIATES, LTD. Court Reporters i 1612 K Street, N.W., Suite 300 i Washington, D.C. 20006 (202) 293-3950
75 ] 1 distribution modules, we have analog-and contact I/O 2 modules, and then the Spec 200 micro module, which you have 3 being passed around. I will show you how that goes together 4 in a few moments here. 5 .This is a typical rack assembly. I apologize for 6 the quality of-the slide here but it is the best we could do: 7 with a photograph. Basically, it is just a rack, and it is 8 just like the picture you see coming around, a power supply 9 on the bottom in what they call nests are just stacked up 10 there, and then these modules just go in these nests. Then 11 you have a power distribution module in each_ nest as well to 12 distribute the plus or minus 15 volts and the loop power 13 supply voltage. () 14 Within that nest, we mount the modules, as I said, t 15 and they just slip right, There are two screws, one of in. 16 the top and one on the bottom, typically, and then the wire-17 terminations are just terminated using screws and lugs right 18 in the front, so if you ever need to remove any or put them 19 in, you just simply terminate, unscrew, and pop them out. 20 This kind of shows how they slide in and out of the module 21 or in and out of the nest. 22 Just a few pictures of some typical modules. This 23 is a typical I/O module. It is all analog. Nothing really 24 new about that. 'The Spec 200' product itself, the analog 25 version, is about a 20-year old product. It came on the O l' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington,'D.C. 20006 (202) 293-3950
I i 76 { 1 market, I guess, in 1972, so it has been around a while. A 2 The Spec 200 micro product was on the market in 1986, so it j .i 3 also has an extensive user base. j 4 This is a contact output module, and it really is i 5 nothing more than four double pull, double throw relays in j 6 there, which is our interface to the outside world for a 7 trip and alarm functions. 8 Then we get to the digital portion, j 9 microprocessor-based, whatever terms you want to use, which l 10 is, again, the card I am passing around there, that is the l 11 micro card. 12 There are really two versions of that. There is 13 the one you have and then there is an extended version which ) 14 is this one here, and it takes two module slots. The only 15 difference being you have eight additional contact outputs 16 coming off of this side here, and we do use a number of 17 those in our application. The software is the same, the 18 firmware, or whatever term you wish to use. 19 Now, this is a very simple description of what i 20 goes on within the micro card itself. It takes an analog 21 signal in, processes it, does several things here, and then 22 it goes into the block processing. Now, when you configure, 23 you have your choice of up to six blocks you can configure a 24 number of ways. Then once you establish the path, which 25 blocks they go to, it gets processed accordingly and then O-ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 .(202) 293-3950
77 { 1 out, and then to the output end of that card, and then gets 2. spit out also as an analog signal. All this occurs within 3 200 milliseconds, that is the standard control cycle for the 4 process. 5 MR. LEWIS: How many cards of this kind are there?- 6 MR. SOTOS: We have, I think, around 80 or 90 in 7 our application in one unit. 8 MR. LEWIS: What is the level of redundancy among 9 these cards, because in this card, of course, now we begin 10 to get into the question of common-mode failures, which I am. 11 sure the Staff will bring it up we don't. How many of these 12 different channels can be failed by the failure'of a power 13 supply? l ) 14 I didn't even notice there were any electrolytic 1 15 capacitors, but there are, they occasionally blow up 16 violently. How much damage can be done? 17 MR. SOTOS: We don't think any different than our 18 analog system now. The level of redundancy is, essentially, 19 the same. Let's say you had a power supply fail, that power 20 supply is powering and its bused, it is powering the entire 21 protection. set, let's say, four racks. However, there is 22 more than one power supply in there. What we have done is,. 23 we have put one extra power supply in each set, bused those 24 altogether, so that if we had one power supply fail, we 25 would still be operating without any effects. ANN RILEY.& ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202). 293-3950
v 78 r '}~ 1 MR. LEWIS: That is if it fails on the low side. 2 What if it fails on the high side and burns out everything I 3 in its train? f 4 MR. SOTOS: They are all diode auctioneered. 5 MR. LEWIS: Pardon? [ 6 MR. SOTOS: They are diode auctioneered. You say l 7 if the power supply went high? l 8 MR. LEWIS: Failed high, sure Every power supply l i 9 has an internal circuit that in the end regulates the l l 10 voltage, and sometimes they fail by making the voltage too 11 high. .l 12 MR. SOTOS: If the voltage went too high, you j i 13 would conceivably blow some fuses. I don't know how many, j If 14 depending on the level, but we have individual fuses in all' t 15 of these modules and in the power distribution module l 16 itself. The most likely thing that would happen is the fuse 17 would blow in a power distribution module, which are in the l 18 nests, and the worse case is you would' lose everything, I 19 guess, in that whole set. 20 MR. LEWIS: Fuses usually pop on current, not 21 voltage, and damage is often a matter of voltage, not 22 current. I just wondered. You know, there are failures in 23 which power supplies supply too much voltage. 24 MR. SOTOS: Yes. 25 .MR. LEWIS: -I am just sort of groping randomly for. 0~ l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 ' Washington, D.C. 20006 -(202) 293-3950
79 1 common-mode failure mechanisms within the card and then 2 within groups of cards, because I think it is a legitimate l 3 concern. I may be downplaying it in most of my fights-with \\ 4 the Staff, but it is a legitimate concern. ] 5 MR. SOTOS: It could be common-to that particular 6 protection set, but that is really no different than a 7 similar type of failure within a protection set that could j t 8 take out the entire set, and we have that now. Say, you had I 9 a circuit breaker go out, for whatever reason, you would l 10 lose the entire set. That is the worst thing that could i 11 happen. 12 MR. LEWIS: Except that digital electronics are i 13 more susceptible to damage from overvoltage than relays. I '( ) 14 MR. SOTOS: That's correct. You could conceivably .j i-1 15 damage your components. ) 16 MR. LEWIS: That's correct, that is what I am 17 talking about. 18 MR. SOTOS: Right. That could happen. l 19 MR. LEWIS: That's what I am talking about. i 20 MR. SOTOS: That's conceivable. 21 MR. LEWIS: Sc you get a surge down the line that 22 damages everything downstream from it without popping the 23 fuses or by popping the fuse a millisecond too late or j 24 something like that. I 25 MR. SOTOS: Yes, that's conceivable. J i ( ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 j Washington, D.C. 20006 '(202) 293-3950
-l 80 1 MR. LEWIS: Okay. The question is, what is it t 2 that limits that damage, presumably redundancy? 3 MR. SOTOS: Yes, redundancy. You have three other-4 protection sets that should still be performing their 5 function. 6 MR. LEWIS: Which have separate power supplies?' 7 MR. SOTOS: Correct. 8 MR. LEWIS: All right. 9 MR. SOTOS: We mentioned the blocks and how they ) ) 10 are configurable. There are actually, I believe it is 21 11 different control functions that you can configure within 12 these blocks, and I have'them all listed here. We only use 13 at present six of these in our application at Cook. We use 14 the gate function, lead / lag, signal select alarm, which is 15 probably the most common, ramp, and a calculator block to do 16 some of the more sophisticated calculations needed. 17 This is a brief, simplified block diagram of how 18 this all fits together, just in general. You have field 19 inputs coming in via a transmitter or-RTDs. They come into 20 one of these input modules, I talked about, which converts 21 it to a zero to ten-volt signal for the analog signal, and 22 that is what the micro card is based on. 23 You have a zero to ten-volt analog signal going -i 24 into the Spec 200 micro card. It will come in and get 25 processed. It will do it is thing, whatever needs to be ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,.N.W.,. Suite 300 i Washington,'D.C. 20006 -l (202) 293-3950
81 1 done, and then it will be outputted either as a zero to ten-g 2 volt signal, if it is analog, to an analog output module 3 which will then convert it to a 10 to 50 milliamp signal for 4 us, or it will go off as a logic two or ten-level signal to s an output module, the contact module that I had a slide of, 6 for use in trip functions or alarm functions. 7 It is important to note here also that in a lot of 8 our applications when we were going off to maybe the control 9 system or off to an indicator or a recorder, we quite often 10 will go straight from here off to another output module I i 11 which goes off to the indicator, bypassing the 12 microprocessor, because all that microprocessor is doing in 13 most applications in the RPS system is just the alarm or ( 14 trip functions. Quite often, we can just go off to all of .i 1 15 our indicators, just bypassing that thing completely, so it 16 is purely an analog signal. 17 Now, for the nonsafety-related portion of the job,_ 18 and I am not going to spend a lot of time on this, again, we i 19 are replacing that H-Line equipment. We had the same stuff 20 on the control side. That impacts 14 cabinets of similar 21 type like the pictures you have seen. These are some 22 typical loops here that are in the application. 23 I will throw this one up again just to give you an 24 idea where this equipment is located. If you go to the back 25 of the control room, behind some panels, we have got them O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i -.j 82 1 located in the back here, in a row of racks here and here 2 and a couple of oddballs, over there. That is all where I 3 they are located. They are geographically diverse for 4 protection. 'i 5 We are using a Taylor. Mod 30 hardware. 6 Architecturally, it is somewhat different than what I passed i 7 around for this Foxboro, but it is basically the same~ type 8 of equipment. You have power supplies for the system and 9 for the loop transmitters, and then you have analog I/O 10 hardware and then you have what they call math units or j 11 controllers, which is the digital portion of_the system. l 12 Again, it is a lot of analog equipment mingled in with the 13 digital stuff. That is kind of a basic background of the j () 14 equipment we are using out at Cook, or will be using out at l t 15 Cook. ~ i 16 Now I want to get into what the major technical i 17 issues were that we addressed as we have gone through this I 18 job. I kind of broke it down like this, I have 19 qualification which has a number of topics in it, electrical 20 items, equipment performance, equipment failure effects, we i 21 talk about software some, acceptance testing and the things-22 we are doing, and then kind of a catch-all category. 23 Under qualification, a lot of this stuff, I think, f 24 will be things you have seen that really aren't any ] 25 different over the course of years. We had to look at O ANN RILEY_& ASSOCIATES, LTD. I Court Reporters i 1612 K Street, N.W., Suite 300- -I Washington, D.C. 20006 (202) 293-3950 C-'
rl -t 83 i 1 1 temperature and humidity in the control room. We had I - C:). 2 testing for extreme control room conditions per IEEE 3 Standard 323-1974. We also did some supplemental cabinet 4 temperature rise testing, specifically for our application. 5 We used replicate racks or a replicate rack to do that. 6 We had to look at radiation. The Foxboro Spec ~200 { 7 product has a threshold established of ten to the fourth i r 8 rads, that was established by test and analyses. That is-t 9 pretty conservative with our control room conditions, worse 10 case by about a factor of ten. f 11 We did, of course, seismic qualification. We had 12 a whole bunch of testing per IEEE 344-1975 which was generic ) i 13 to the product, plus we did supplemental analyses for our l ) specific application in our particular racks. 14 15 MR. LEWIS: I am having a problem with information 16 absorption, because lists of things looked at don't help a 17 great deal for the subcommittee in terms of knowing what was 18 actually done because anything can be looked at carefully or. i 19 carelessly all the same. I notice your next ten charts are 20 exactly the same, they are lists of things that you looked 1 21 at. I am prepared to stipulate that you looked at a long f 22 list of things, and I don't know quite what to make of it. ~ 23 What have I done wrong? [ '24 MR. SOTOS: I guess I am not sure what level of l 25 detail you wanted to go into. l O IJUJ RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 84 1 MR. LEWIS: Well, you know, reviewing a system of j 2 this complexity is obviously something we can do in a 3 subcommittee meeting. .] 4 On the ot te hand, I am already impressed that it ( 5 is fairly complex and that there are many questions that \\ 6 need to be addressed, and your viewgraphs show me that they l 7 are being addressed. Unless I were to look at the list and 8 think of something that you haven't put on the list, which I 9 am not likely to do on the spot, I am really not sure what I 10 am supposed to be learning. I am not criticizing you, you 11 understand, I am just wondering how attentively I should-i R12 watch the next ten charts. j 13 [ Laughter.] ~ 14 MR. SOTOS: Well, it was our understanding that 15 you wanted to hear about what our experience has been to + 16 date for the project and the issues that we have had to go l t 17 through. 1 i 18 MR. LEWIS: What? l 19 MR. SOTOS: I am sorry? -r 20 MR. LEWIS: Say it again? i 21 MR. SOTOS:.It was our understanding that you. I 22 wanted to hear from us what our experience has been to date 23 on our project and the issues that we have.had to address.- i i 24 MR. LEWIS: Yes. No, that's exactly right. } } 25 MR. SOTOS: That's what I am doing. It probably ? r 1 ~ ANN RILEY & ASSOCIATES, LTD. i Court Reporters 1612 K Street,-N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 -1
85 1 is redundant in a lot of ways to what you have been hearing i 2 before. 3 MR. LEWIS: No, I am not criticizing you. I'am 4 having trouble focusing on what the remaining issues are. 5 MR. KERR: For example, did you find -- 6 MR. LEWIS: You can probably say better than I can 7 what is concerning me, Bill. 8 MR. KERR: Did you find anything in the review 9 process that was unexpected or was particularly important-to 10 safety that you had to correct or change? 11 MR. LEWIS: That's a good way to ask it. i 12 MR. SOTOS: During the review process? 13 MR. KERR: Well, review process, qualification (I ) 14 process, the thing you are describing. 15 MR. SOTOS: Not in the qualification process, we 16 didn't have to make any changes for any of the hardware that 17 we didn't already anticipate. One thing that popped up, and 18 I was going to get to in a little later-slide, was in the 19 power area. We found significant in-rush current was going 20 to be a problem, and so we had to add in-rush current 21 limiting circuitry to the equipment to keep it within bounds 22 of our capability at the plant, that was one thing-. 23 For the most part, we stuck with the vendors' 24 standard hardware, standard software, and standard 25 application methods and techniques for this project. We ANN RILEY & ASSOCIATES, LTD.
- s Court Reporters 1612 K Street, N.W.,
Suite 300 Washington, D.C. 20006 -(202) 293-3950
86 1 haven't had to make any real substantial changes. Virtually 2 all of the investigations, qualification testing, reports, 3 analyses really just confirmed what we had already assumed 4 and, therefore, we didn't have to make many changes in 5 anything. 6 MR. KERR: Well, I guess I don't know what stage 7 this project has reached. Are you operating yet? 8 MR. SOTOS: No. We recently underwent an NRC 9 audit back in may, and we are expecting an SER very soon. 10 The project ~itself is in the preinstallation testing phase. 11 They are undergoing factory-acceptance testing and some site 12 work, as well, and we are expecting to install it'next year ) 13 in our 1994 outages. At present, most of the equipment, j 14 with the exception of the Unit 1 protection hardware is on-15 site now. 16 MR. KERR: We l'1, have you found any of the units-f 17 that were defective in the process of testing?_ 18 MR. SOTOS: Yes. In fact,_when we were doing 19 factory-acceptance testing at Foxboro, we found that we were 20 having some failures of those relays that I showed you in 21 those contact output modules. That turned out to be kind of-1 22 a generic failure. They ended up issuing a Part 21 on it 23 recently, and we had to end up replacing all.of those 24 relays. So we did find.a problem there. We have not found 25 any other significant problems in the testing, other than ~ O ' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,. N.W., Suite 300 Washington, D.C. 20006 (202) _293-3950 l i
i 87 1 minor things, maybe a wire out of place or something like i i 2 that. i 5 3 I guess I didn't talk about this one. r 4 MR. LEWIS: I guess I can do a better job of 5 articulating what was concerning me. I was worried about 6 getting the problems that were overcome up to the top of the 7 ocean so that we could focus on them. If you could do that, 8 then that would be very helpful. l -t 9 MR. SOTOS: Well, I will skip through some of 10 these things, then. These are pretty much routine things i 11 that manufacturers do, electrical isolation, electrostatic 12 discharge, surge withstand capability, lightning effects. 13 They were all tested per the standards I have shown there, f 14 MR. WILKINS: There were no surprises? l I 15 MR. SOTOS: No, none. [ i 16 EMI/RFI, of course, has been a big focus. We 17 evaluated both emissions and susceptibility. We did site I 18 surveys out at the sites, both at power and shutdown. We 19 didn't find any unexpected conditions and we didn't find any i 20 significant differences between units. We performed these 5 21 various tests. We used the MIL Standard 461(c) as our 22 guidance for this, mainly. .i i 23 I think you stated earlier, Mr. Lewis, our same ~ j t 24 position, that we didn't find anything particularly-unusual. j 25 Again, when you do these surveys, they are just snapshots at j 'I IJni RILEY & ASSOCIATES, LTD. Court Reporters l 1612 K Street, N.W., Suite 300 1 Washington, D.C. 20006 o (202) 293-3950 l i
I 88 i 7 1 that particular moment in time. The data could change from i 2 -day-to-day, but it was basically just confirmatory. It gave: 3 us a warm, fuzzy feeling on what our site environment-is for l 4 that. l 5 As far as susceptibility goes, we did testing, 6 again, for MIL Standard 461(c), using these portions of it 7 that were applicable. We found the equipment performance i B acceptable. The susceptibilities we did find were very l 9 minor. We find some output shifts a little bit, maybe a e 10 percent or two, but they were of the same type you would see { 11 with any analog equipment -- in fact, they probably [ 12 performed a little better in some cases -- but all within in i 13 the acceptance criteria we laid down prior to the testing. ) No surprises there, either. 14 15 Under the electrical issues, we did a power study 16 to look these four main topics: power quality, circuit j 17 loading, in-rush current, and inverter aging effects. We-18 found, as we had assumed, that the existing-system would 19 support the new equipment without any problem. As I talked. l 20 to you before, we did have to add in-rush current limiting-21 circuitry to the Foxboro equipment. 22 We also looked at grounding, we looked at ground l 23 faults, lightning strike effects, personnel safety grounds, 24 EMI shielding for sensitive circuits, and ground system 25 design and life expectancy. We used IEEE Standard 1050 as t .i i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612'K Street, N.W., Suite.300 i Washington, D.C. 20006 (202) 293-3950-
l l i I 89 1 guidance for this. Again, we found no surprises. We stuck 0 i. 2 with the vendor reference design requirements for the 1 3 installation. We did decide to make some improvements in i 4 our existing cabinet grounding. We are adding some extra t t 5 things there to ensure we have good solid ground for all the i 6 cabinets. f i 7 MR. KERR: Did you have to make any-changes as a-l 8 result of your lightning strike effects test? 9 MR. SOTOS: No. l l t 10 MR. KERR: Did that assume a direct strike on your j i 11 switchyard, some of your switchyard equipment, or is that 12 incredible with the grounding? i 13 MR. SOTOS: To be honest with you, I don't know () 14 quite what the assumptions were for that particular test. I 15 MR. KERR: I just wondered if-lightning could l 16 strike one of your transformers, for example? l 17 MR. SOTOS: Bob, do you know that? 18 Bob Carruth is my section manager, maybe he can 19 answer that. 20 MR. CARRUTH: The control room is fairly well 21 isolated from any of the power equipment in the plant, 22 except control' power, which itself is isolated. It is.all h 23' in station inverters with no direct tie at all to the_ plant-24 auxiliary system. The kind ofLevent you are talking about' R25 is one where. lightning gets into the generator step-up bank. I 2 ANN RILEY & ASSOCIATES, LTD. I Court Reporters 'i 1612 K Street, N.W., Suite 300 E Washington, D.C. 20006 (202) 293-3950 i
,. = I 90 i 1 You.get a transient on the low-voltage buses which cO-' 2 propagates through the plant in some form. The instances ? 3 where we have seen that in the industry is where the plant i i 4 control systems inverters have a reference to those 5 auxiliary system AC buses, and that is not the case with 1 i 6 D.C. Cook. 7 The study, in general, is a qualitative assessment i 8 based on grounding practices and based on existing lightning i 9 protection in the plant. There was no statistical analysis 10 on potential, maximum, credible lightning shots to the I 11 structure directly or to the substation. The substation is 12 about a half a mile away from the plant, so activity out i 13 there would be of minor consequence. ) 14 MR. KERR: Well, is there any way lightning could 15 strike something closer to the plant and introduce 1 16 transients into the system? I 17 MR. CARRUTH: It could strike in the vicinity of .j 18 the plant. The plant has a fairly uniform grounding system, i 19 it is all basically on one plane. So there is an excellent 20 setting for any current that would result from a lightning 21 strike right directly to the plant. The plant grounding 22 system is very well-emersed, it is below the water-table at 23 that location. 24 MR. KERR: Your ground system is better than those l 25 plants that have had damage done because of lightning I i ANN RILEY & ASSOCIATES, LTD. -j Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 i .'l \\
r 91 7 1 strikes? 2 MR. CARRUTH: We have not seen any extensive 3 damage to any of the protection equlpment at the plant as a 4 result of lightening over the history of the plant, and it 5 about the same class of electronic hardware, from a 6 vulnerability standpoint, in our perspective. 7 Very light duty electronic or low-energy circuits 8 all have a potential for damage from something like this. l 9 We have a very extensive grounding system that is well-6 10 rooted in the ground. All the building steel'is brought j 11 down that, too, and the control room is buried up in the l 12 bowels of the plant, very well-shielded. 13 MR. KERR: Thank you. l t 14 MR. CARRUTH: You are welcome. 15 MR. LEWIS: There was one event not to a nuclear 16 power plant, I recall, it was to somebody's personal l 17 computer, a few years ago, that was reported in which a ) 18 lightning strike hit the step-down transformer outside his 19 house'. I have forgotten whether it was 40 kilovolts down to 20 110 volts or to 220, something like that, but whatever it 21 was,it broke down the insulation between the primary and the 22 secondary. The result is that his house got -- and'I am 23 inventing the number, I don't remember it that accurately'-- 24 40 kilovolts where there should have been 220, and all the 25 light bulbs burmed out of course and is computer turned into O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W,,_ Suite 300 Washington, D.C. 20006 (202) 293-3950
l 92 I 1 a mass of molten junk. j -C) 2 MR. CARRUTH: You could get into that situation if i 3 you have a failure high to low on the transformer out in a i 4 substation in the power plant or in your home. Your home is' ? 5 not designed to take anything like 2.7 kV or 5 kV or 6 whatever happened to come through in volts. 7 The telephone system usually provides some-form of i 8 surge immunity in their carbon blocks, their. clamping device 9 to keep it from coming in that way, but your power equipment 10 in the plant is basically a 600-volt class installation 11 systems. Once you get that kind of voltage in, it can go 12 over, literally, anywhere. Typically, before it does, it 13 manages to pop fuses'on any magnetics like the transformers -('i - 14 in your power supplies, whether it is a computer, light .% ) 15 bulbs, and everything else. 16 MR. LEWIS: It is just a question of what goes 17 first. 18 MR. CARRUTH: Yes, the weakest link in the system, 19 where can it go. Once that goes, if.that hasn't clamped it, 20 it may find another couple of things before it finally gets 21 shorted out enough to open an incoming breaker, blow a fuse, 22 or clear the fault. Some of this is not a direct electrical 23 ' connection. You have got a-flashover from a high voltage I i 24-source to a low voltage source which elevates the-_ low 1 25 voltage source. i a ANN RILEY & ASSOCIATES, LTD. Court Reporters- -l 1612 K Street, N.W., Suite 300 { Washington, D.C. 20006 (202)-293-3950 i
(} 1 MR. LEWIS: Well, in fact, 93 2 if it is sufficiently high voltage any fuse can flashover 3 MR. CARRUTH: doesn't have the ability to iIt may have a clear either i 4 t, if 5 current. nterrupt the follow-through 6 MR. LEWIS: effects of lightning, I am just in the discussion 7 of the the argument 8 sense is to say it has never ha that never makes a lot of 9 ppened. rarely, of course, It happens very and it is an unlikely event 10 something that has the potenti but for 11 to all systems, al to do really major damage you have to think about it 12 MR. CARRUTH: 13 Right. situation since Cook was built We have lived with that 14 there are solid state pr t Most of the electronics in and transistors, o ection systems made up of IC 15
- chips, ability to withstand that kindand these have the same level of 16 17 of abuse.
relays that may be able to take Electromechanical 18 before you flashover and do anup to two-and-a-half kV 19 them in the wrong locationy damage, but then again, you get if 20 happen in some and we have had that substation applications out 21 or if you get lightning into them, in the field 22 relays, too. you will burn out the 23 It is really a question of how much p 24
- provide, how much assurance you gain th rotection you 25 very well-guarded locations in ereby.
These are comparison to most other ANN RILEY & ASSOCIATES LTD. 1612 K Street, Court Reporters Washington, D.C.N.W., Suite 300 , _ ~ ~ - - ' '
' ' yon _w
--20006- - - - -
~ l J 94 l 1 locations around the system where we have solid state 2 . electronics. t 3 MR. LEWIS: But that is the question. The i 1 4 question is, where the guarding is against really unusual 5 but really very high voltage events, and we have to have i 6 something at the in point which will protect on a short 7 enough time scale. 8 Please go on. 9 MR. SOTOS: The next area we evaluated was 10-equipment performance. The two major areas there we looked i 11 at were setpoints and time response. We looked at existing 12 RPS and ESF setpoints, time constants, that sort of thing. j 13 We found we did not have to make any changes in our _ () 14 setpoints, or allowable values, or the time constants, or i 15 anything else. I think it is safe to say that the new 16 hardware that we are installing is inherently more accurate 17 and stable as compared to the' existing analog equipment. 18 As far as time response goes, again, we evaluated ~ ~! 19 our existing time response. requirements out of our tech-j 20 specs against the new equipment and found we did not have to 21-make any. changes there either. -However, we should note that 22 this new equipment does typically perform slower than the 23 analog equipment, due to that microprocessor cycle time. j i 24 which, in our case,-is 200 milliseconds. l l 25 Typically, analog hardware operates where maybe we i O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 lK Street, N.W., Suite 300 Washington, D.C. 20006 -(202) 293-3950 j
95 1 have been seeing 50 milliseconds something like that, so it 2 is a significant difference. But we had adequate margin in 3 all of our test specs to absorb that additional increase in 4 time so we did not have to make.any changes there, as well. 5 MR. LEWIS: What is it that determines the 200-6 millisecond cycle time? 7 MR. SOTOS: Well, the processor has a cycle time, 8 every 200 milliseconds it goes through and does all of its 9 processing. That's just constant. It is every.200 10 milliseconds it runs through its routine. 11 MR. LEWIS: Is that intrinsic to the l 12 microprocessor? 13 MR. SOTOS: To this one, yes, 14 MR. LEWIS: What microprocessor is it? 15 MR. SOTOS: Bob, I think you have it there. 16 MR. LEWIS: Well, 200 milliseconds is slower than 17 my response time. 18 MR. CARRUTH: Well, I think one point that has to 19 be made is that it is running through an algorithm, a series 20 of algorithms, in that 200 milliseconds. That is not the 21 microprocessor's time, it is its program execution time. 22 MR. LEWIS: No. That is what I was. reacting to. 23 He said it was the microprocessor intrinsic time, but it 24 can't be that. 25 MR. CARRUTH: It can't be that, no. It is the ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 96 ~ 1 execution time for-the set of algorithms, or it is path that I 2 it follows before it completes it cycle. Your assurance of e 3 capturing an activity, what the delay is associated with t 4 that activity is nominally 200. 5 MR. LEWIS: If you take 200 milliseconds -- I 6 don't know what microprocessor it is, but it is a rare i 7 microprocessor that needs a clock slower than a megahertz, ~ 8 so you are talking about an algorithm that executes. 200,000 t 'i 9 cycles. That is a fairly complicated algorithm. 10 MR. SOTOS: The clock operates about 5 megahertz. l 11 MR. LEWIS: Five megahertz? l 12 MR. SOTOS: Yes. 13 MR. LEWIS: Then a million cycles in one execution f 14 . cycle, that is a fairly complicated algorithm. 15 MR. CARRUTH: Well, I think that is also an ~ 16 outside number, and there are there other delays in that. i 17 It has a wait period and some other things. J 18 MR. SOTOS: Yes. It is also running diagnostics 19 and everything else every time it goes through this 20 execution cycle. 21 MR. LEWIS: But a million is a large number. 22 MR. SOTOS: Yes. .l 23 MR. LEWIS: We will worry abo'it that later'. 24 MR. CARRUTH: I think the point there is, the 'I i 25 reality of the situation is, by the time you have a product O 1-IJUI RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
l i 97 l ~ g that'is going to run reliably engineered you end upfwith a 1 i -- 2 lot of buffers and a lot of time delays and a lot of margin, 3 and things, and that is what you are seeing with that kind 4 of a published response time. 5 MR. LEWIS: Yes, but you are giving me a million 6 of them, a million cycles, q 7 MR. CARRUTH: We bought it. 8 MR. LEWIS: That's right. Yes, that is.the 9 ultimate answer. You own it, you got it, you deal with it. l 10 MR. CARRUTH: Yes. We own, it we have got it. 11 [ Laughter.] i 12 MR. LEWIS: Please go on, we'will deal with you. 13 MR. SOTOS: The next area we looked at was 1 14 equipment failure effects. We got to looking at failure -t 15 modes and effccts. We did an evaluation using IEEE 1 16 Standard 352 kind of as guidance, but what it really was, 17 was a comparison of old failure modes or the failure modes 18 of the existing equipment versus the new equipment. i 19 We did that on a system-level basis, and that is l 1 20 kind of important to remember as a concept. We did not go 21 down to the component level, it was at the system level, and 22 we did not identify any new failure modes, comparing.old to; 23 new. As confirmation of that, we.are using our factor 24 acceptance testing as the final proof of that conclusion. 25 MR. KERR: Were you surprised that you didn't' O 'l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
98 1 identify any new failure modes? 2 MR. SOTOS: No. At the system level, that was not 3 a surprise. 4 We also looked at reliability again on a system 5 level, and we also used IEEE 352 as our guidance for that. 6 Again, it was a comparison of the old reliability versus 7 new. As we expected, we found the new equipment to be more 8 reliable than the old hardware. 9 At great personal risk, I am going to put this i 10 slide up, talking about reliability of software. Foxboro i 11 did use some techniques to, before release of the product, 12 establish software reliability. I am, by no means, a 13 software expert, but these are some of the techniques that () 14 they did use. ( 15 MR. LEWIS: Shucks, I was. going to ask you to 16 explain each of those. j 17 [ Laughter.] 18 MR. SOTOS: Please don't. 19 Foxboro found the results to be acceptable. I. 20 would like to make the case, however, that these statistical 21 methods really yield pretty soft results. I will. call them 22 soft results. I don't know how much you can hang your hat f 23 on them. The real proof of the pudding is to use actual 24 application experience-data, if that.is available. That is 25 pretty much our case now. We have a product that has been 1 .O { ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202)-293-3950
l i 99 1 on the market and out on the street for a number of years 2 and -- 3 MR. LEWIS: Don't dwell on things you are not 4 going to be able to defend if I push you. 5 MR. SOTOS: I will take the hint. I i 6 [ Laughter. ] I 7 MR. SOTOS: Okay. 8 MR. LEWIS: Also, I am trying to get us to finish 9 on time. 10 MR. SOTOS: I am trying. 11 Diversity, we also evaluated that, that became an i 12 issue. Really, this is driven by concern'over common-mode 13 failures, really hardware or software. We looked at what we. (} 14 call signal path diversity, other people may call it I 15 functional diversity, and just integrated that into'our. i 16 design. i 17 We also addressed equipment diversity, 18 particularly in regards to our AMSAC system. We presently 19 have Spec 200 equipment in our AMSAC system so when we do' [ 20 this change out,.we are going to have to, in order'to meet 21 that diversity requirement, change out our AMSAC system, + 22 which we will be using the Taylor Mod 30 product for that. 23 We already talked a bit about' software. It is not-24 user accessible for our particular product'and only. resides. i 25 as configurable firmware within that module. It is i ANN RILEY & ASSOCIATES, LTD. .l Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 q (202) 293-3950 .j i
. =.. ~ i i f 100 1 configured first using a personal computer, using menu-2 driven software. A V&V was performed, and it was documented 3 per the ANSI 7.4.3.2 guidance. We did so some additional f 4 software retesting or Foxboro actually did per our request. i 5 to satisfy those requirements. Again, no surprises there, i 1 6 no changes, no modifications to the software. 7 Configuration management has become a recent' 8 concern to the Staff but has always been for us.. We will 9 just make the case that strict administrative control of the -i 10 software at the vendor, in our case, and of the data out at j 11 the utility is very important, as is access to configuration 12 tools such as the PC, that should be strictly controlled, as 1 i 13
- well, (t )
14 MR. LEWIS: Does strict administrative control 15 mean only one person can do it or 17 people? 16 MR. SOTOS: No, it means you have it very 17 proceduralized, it is in your QA program, and you' document j 18 things using documents within your QA programs, using 19 control documents. 20 MR. LEWIS: Does any modification of~the software 21 go through the same V&V, whatever you meant by that, as.if J 22 the thing were being installed new? l 23 MR. SOTOS: The software, any modifications we 24 have done at the vendor, which would be under their-QA 25 program,-which would go through that V&V process. O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 m_... .,m_, _ ..,,..___.,.___I
i, 101. g-- 1 MR. LEWIS: That then raises a question that has 2 come up to me many times, because all through this 3 presentation you have spoken of specifications laid down by 4 the vendor, environmental specifications and that sort of 5 thing. What about the next step. Who is watching the 6 vendor? 7 MR. SOTOS: Well, the specifications were 8 initially laid out by us in our specification to the vendor, 9 and then they have to comply with it. 10 MR. LEWIS: But the vendor is, in turn, supplying 11 the equipment to you with a list, an environmental envelope, 12 for example, of some kind. 13 MR. SOTOS: Yes. ) 14 MR. LEWIS: Or did you prescribe that to'him and 15 he claims he met it? 16 MR. SOTOS: We prescribed it to him. 17 MR. LEWIS: He claims he met it? 18 MR. SOTOS: Yes. Then we verify that against the 19 documentation provided. 20 MR. LEWIS: You verified it, or you checked his 21 documentation? 22 MR. SOTOS: We check his documentation. 23 MR. LEWIS: So the assurance that he'actually 24 -- I am not saying he'didn't, but the assurance that he 25 actually met -- that your' specs were reasonable because, of ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) ~ 293-3950
-i 102 1 course, you laid them down before you saw the stuff, and 5 i ~ that he actually met the specs is in his honor? 2 3 MR. SOTOS: No. That is for us, that is our 4 responsibility to make sure that is done. 5 MR. LEWIS: You said you checked the paper? 6 MR. SOTOS: Yes. i 7 MR. LEWIS: Did you check the testing, audit the .i 8 testing, repeat some of the tests? l 9 MR. SOTOS: No. 10 MR. LEWIS: What makes you sure he wasn't faking a 11 test? 12 MR. SOTOS: That is through our QA program, and j i 13 their vendor surveillance of Foxboro to make sure that they ) 14 are complying with the programs that they are supposed to be .I 15 doing. If they are doing that, then the documents we should 16 be getting from them should be on the level. We do go back, i 17 our QA does, and we will go back occasionally to audit the. 18 vendor to make sure that there is that paper trail, that 19 traceability. 20 MR. LEWIS: I am not suggesting for a moment'that i 21 anybody is dissembling here, but one has to'be careful. I .i 22 have forgotten where it was that they had this wonderful i 23 event at which some concrete through-bolts had heads ~and 24 tails but no middle. .I have forgotten what plant that was j i 25 at. But people will go to extraordinary lengths to cut i I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202)-293-3950
(: t 103 r 1 costs in production, and somebody has got to watch out for 2 these things. 3 MR. SOTOS: Well, we do, particularly on this l 4 project, maintain very close vendor surveillance, both wi.tv 5 our QA folks as well as in engineering. 6 MR. LEWIS: I see, okay. l 7 MR. PLACE: I just have a quick question with 8 respect to this. I 9 MR. SOTOS: Yes? 10 MR. PLACE: If I have any PC with Foxboro y 11 software, could I modify your system, assuming I can get i 12 into your plant, or is it tied to a specific PC that you can 13 keep under lock and key? () 14 MR. SOTOS: Well, there is an interface device i 15 that has a key-lock on it. If you had that, but that is 16 also generic to Foxboro, if'you have that and the PC,. if you 17 could physically get to the equipment, yes, you could get to 'I 18 it and change it. You couldn't change'the software, you 19 could only change the configuration data. 20 As far as acceptance testing, we have several 21 aspects. We have got factory acceptance testing out at the j 22 Foxboro folks. We will be testing a number of those things, i i 23-or all of those things-there on my list. I am not going to 24 go through them in the interest of time. We will be using l 25 detailed procedures that the vendor developed and that we O I JJM RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
? l 104 l ) 1 have reviewed and approved, and we are_ testing 100 percent l O f 2 of the equipment prior to its shipment. One other thing, we 3 are also simulating all inputs and outputs when-we do this l 4 testing. It is to as closely resemble as we can our D.C. } 5 Cook inputs and outputs. 1 6 On-site, once the hardware gets there, we have a-7 number of tests that we are doing. The primary one will be -[ 8 hardware integration testing, and don't confuse that with an 9 integration testing between software and hardware. -What we l 10 are doing here is, we are taking our racks of Foxboro f i 11 equipment, staging it up with our racks of Taylor Mod 30 12 equipment, tieing them together to make sure that we don't. f 13 have any strange things going on between the two of them, () 14 since there is communication between the two when the things ( t 15 are installed. i ~ i 16 Post-installation testing, we will be looking at I 17 electrical verification, the calibration, functional j i 18 performance / control actions, make sure those are okay; j i 19 dynamic performance; response time; and, again, any. 20 interaction with interfacing equipment. Those are the ~ 21 primary things we will be looking at. l 22 A couple of other odds and ends. Training, we l T 23 have formal training for. engineers and technicians using j 'i 24 replicate equipment. We will also be doing_ formal training 25 for operators in a plant' simulator, which is going to be l I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street--N.W., Suite 300 Washington, D.C. 20006 (202). 293-3950.
i i 105 1 modified very soon to reflect the new equipment. { 2 We had one other little oddball thing pop up i 3 during our initial reviews. If you remember when we passed J 4 the module around, there was a little battery.in there. 5 That thing is used to maintain memory if you have a power-6 down situation. There was concern raised if there would be f 7 any heating effects caused by short circuits, primarily. We i 8 did some testing to determine what those effects might be. i 9 We really found no significant problem. They do heat up to f 10 some extent, but they don't do any kind of damage, so.it 11 became a nonproblem. 12 As far as major regulatory issues, there is really 13 just one area that was really giving us fits -- not fits, I () 14 shouldn't say that -- just really has been kind of difficult f -t 15 for us, especially in the early going, and that is lack of' i 16 applicable standards in a couple of areas. I think we are 17 pretty much aware of what those are. 18 In the EMI/RFI area, yes, there are standards that-19 are out there, and I have listed some of them here. There 20 is the one we use primarily, MIL Standard 461(c) and we have 21 also got an IEC series, as well as a couple of IEEE 22 standards and PMC 22.1-1978, which some equipment 23 manufacturers use. 24 None of these documents really addresses EMI/RFI 25 in the particular way'we do in nuclear power, not all of 1 l ANN RILEY & ASSOCIATES, LTD. i Court Reporters j 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
4 i 106 (} 1 them address all of the various areas. The reason being i 2 mainly, I think, because they really weren't intended for i 3 qualification use, to use the term loosely. They were made 4 primarily as guidance for equipment manufacturers or for 5 people designing installations at their facilities. 6 What we have had to do as utilities, we have had 7 to determine what the appropriate standards are, which is 8 something we did way back in the 1988-89 timeframe, where we -l 9 pretty much centered in on MIL Standard 461(c). At present 10 there really isn't any regulatory guidance that endorses any i 11 one particular standard, primarily I think because of these + 12 reasons. That is a difficulty we have had to deal with so 13 far. _ () 14 The other area is in software V&V. We have been 15 using ANSI-7.4.3.2-1982, and I know there is a draft on the 16 street for a later version that should help, but that was 17 written primarily for large computer systems, using newly-18 developed, custom-made, user-accessible software. For our 19 particular class of equipment, as you have seen, we have 20 just got small microprocessor-based instruments. They are 21 using nonuser accessible really stock software and it has an i 22 extensive experience base. You have got to' kind of stretch 23 to make our application fit 7.4.3.2, so we had some 24 difficult there. 5 25 Those are the primary two main issues that we had i ANN RILEY & ASSOCIATES, LTD. I Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 i (202) 293-3950
i 107 j 1 some problems with. Otherwise, I think everything has ) 1 2 pretty much gone as expected and proceeding.with the 3 project, like I said, we are expecting an SER soon and we 4 will be installing this equipment in our 1994 outages. 5 That is the conclusion of my. prepared remarks. I. 6 don't know if you have any more questions, but if'not, I 7 will sit down. 8 MR. LEWIS: Any further questions? 9 [No response.] - 1 10 MR. LEWIS: We have been asking you questions as 11 we went along, and I have to give you a commendation of some-12 kind for finishing on time, in spite of our harassment. But. 13 I think even though it is a minute early, let's adjourn for (( ). 14 lunch, and come back at 12:30. t 15 [Whereupon, at 11:29 p.m., a luncheon recess was 16 taken.] 17 'I i 18 ~; 19 20 d 21 j 22 23 i 24 25 O ANN RILEY & ASSOCIATES, LTD. .i ' Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 .(202) '293-3950 -l
4 L f 108 1 AFTERNOON SESSION 2 [12 : 3 3 p. m. ) 3 MR. LEWIS: Let's reconvene. Before the break I 4 was bawled out by one of my friends for calling a luncheon l 5 adjournment instead of a luncheon recess. I, therefore, 6 retract the first term and will now resume after our- 'I 7 luncheon recess. 8 We are going to hear about Zion; is that correct? 9 MR. MASON: Correct. 10 MR. LEWIS: We are in your hands. I will try to 11 keep us on schedule. 12 [ Slides.] 13 MR. MASON: Thank you. Good afternoon. Welcome 14 back from lunch. Hopefully, I can provide an interesting 15 talk to keep you awake after a good lunch. 16 I am Rick Mason, with Commonwealth Edison. The 17 slide shows Ricky but my friends call me Rick. I hope that 18 most of you guys are my friends, at least even after today. ) 19 With me today is Steve Stimac. He is our licensing i 20 administrator. He has just recently been promoted to -j i 21 regulatory assurance supervisor. He can help me answer.any ] 1 22 questions that stumble me. 23 I am going to talk a little bit today about our 24 experience with our Eagle-21 installation and then continue 25 on with some experience sne have with the current ANN RILEY & ASSOCIATES,'LTD. Court Reporters 1612 K Street, N.W., Suite 300 q Washington, D.C. 20006 'l (202) 293-3950
~ -. ~ 109 1 modification that we are doing to our diesel generator j i 2 control system. 3 [ Slides.] 4 MR. MASON: It's kind of an overview, and then I t 5 will briefly touch on what I intend to talk about today. .I 6 will talk a little bit about Eagle-21, what the project is, 7 and kind of give you an overview of what the system is. I 8 will talk a little bit about some licensing experience that 9 we have had. Most of you are familiar with our SER. I will 10 talk a little bit about the audit that we went through, and 11 some of the things that are involved in that. ) 12 Then, I will shift to a discussion on our i 13 emergency diesel generator upgrades. I will talk a little () 14 bit, again, about what the project is. I will touch on some i 15 initial considerations that we had when we were developing 16 the conceptual design. Then, I will discuss where we stand 17 today with our plans for that upgrade. At the end, I will 18 try to draw some conclusions and summaries. 19 [ Slides.] 20 MR. MASON: I will start out talking about Eagle-21 21. In this discussion, I will touch just real briefly on 22 some of the design features and the benefits. I will 23-probably skip a system overview, because I don't think most-24 people want to see how-Eagle-is built. We will get down 25 into the meat of the discussion here, under licensing-l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
I i 110 [ i f 1 challenges. I will talk about initial 50'59 approach, our 1 2 reconsideration, and then spend a lot of time here in our l 3 NRC audit summary. I will talk about-some of the details of 4 the audit. 5 Just to kind of give you an overview of what t 6 Eagle-21 is and why we chose to go with that product. 7 Eagle-21 is a microprocessor based functional replacement 8 for an existing analog protection system. In our case it 9 was Westinghouse 7100, but it hcs been used to replace 10 Foxboro-H line at some other plants. 11 The hardware is designed to fit into the existing 12 racks. That, obviously, saves on construction and 13 i.tallation time. You just rip the internals of one rack (<f 14 out and put'the new Eagle-21 hardware in. It makes it 15 pretty simple. Again, the modular design allows you to 16 install major pieces of the system,-so that I don't have to ) 17 wire each circuit board once it's installed. I can install i 18-power supplies, sub-assemblies and pre fab wiring harnesses. 19 It all goes together. It's been previously tested at the 20 factory, so all we have to do is confirm is that we have it 21 assembled correctly. 22 Since it does go in the existing racks, that means l 23 we don't have'to move any field cablas. Again, all that 24 does is eliminate installation time and potential errors 25 during construction. l O 1 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 i (202) '293-3950
111-1 At Zion, we did a consolidation. We shrunk 16 l 2 racks spread out across four protection sets,-down into ten. 3 We did that for different reasons. The driving reason, I 4 guess, was our inverter capacity. We identified the fact 5 that Eagle drew just a little more power than existing 6 analog equipment. When we added up all the power 7 requirements we couldn't stay within our limits. 8 We took advantage of digital technology and shrunk-9 the number to the physical number of cabinets that we 10 utilize to actually perform the protection functions. One 11 important thing to keep in mind here is the fact that we 12 didn't rearrange any loops or cross protection set i 13 boundaries or anything like that. If a particular cabinet ) 14 was still in protection it one it stayed in protection set ~ 15 one. We didn't change any of the loop layout. 4 i JU5 Another highlight is the fact that we did install 11 7 some rack mounted man-machine interface carts. 18 Historically, the plants that had done Eagle before us had a 19 portable rolling cart. It's a test cart that is made to 20 roll up to the rack you are going to test, plug it in. It's 21 comprised of a test screen CRT, a printer and a couple of 1 22 other things. It allows the instrument maintenance guys to 23 do surveillance testing. It allows us and maintenance to do 24 trouble shooting. 25 We have extremely tight limit on space on our L ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950-
1 112 1 equipment room. We felt like it would be nice to utilize 2 for these racks that I opened up, to mount this test 3 equipment so it wasn't rolling around the aisle all the i 4 time. We did that. 'i 5 [ Slides.] 6 MR. MASON: Touch on these just real quick. Some f 7 of the benefits we expect to gain from Eagle-21 was, we felt 8 like that it would provide a reliable operation over the 9 next 20 years. We feel like that's important, because if we 10 decide at Zion Station to approach plant life extension we 11 feel like this equipment will already be in place to support 12 any kind of life extension or anything like that. 13 We expected man-hour savings, mostly from our _(i) 14 instrument maintenance surveillance testing. With the old 15 system the IM had to go out and -- I am sure you are 16 familiar with how they calibrate loop. You tweak.a channel 17 and go back and re-tweak what else feeds it, and you finally 18 get it all where it's just right. With this, Eagle-21 does 19 a continuous self-calibration. He doesn't have to deal with [ 20 the analog to digital conversion right up front. i 21 The surveillance testing is all automated also. 22 The instrument maintenance tech walks up, selects which i 23 channel he wants to test, basically pushes to, sits back and i 24 puts his feet up and when the printer prints out the results 25 he verifies the results are within tolerances and he's done. 1 i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 .h
l l l 113-1 It's pretty straightforward. i 2 'The machine has self-diagnostics. This, we think, 3 is a great benefit. You don't have to wait for a failure or 4 for a channel to trip before you know'that you have a 5 problem. This will detect drifting transmitters or numerous 6 other items inside the machine. 7 We also have a redundant sensor algorithm.
- Again, 8
Mike had just mentioned that it detects drift of RTD's. 9 MR. LEWIS: What did you say, RSA sensor? 10 MR. MASON: Redundant sensor algorithm. 11 MR. LEWIS: Thank you. It means something else in 12 the computer business. 13 MR. MASON: Here, it's redundant sensor algorithm. ) 14 It takes inputs from -- 15 MR. LEWIS: While I have you stopped, the 16 continuous self-calibration, that's against some standard 17 that is built into the system? 18 MR. MASON: Yes. 19 MR. LEWIS: Voltage standard or what? 20 MR. MASON: Both. Voltage standard and a time 21 base standard. Every time the instrument maintenance guys 22 go through a surveillance test the first thing they are 23 asked to do is verify -- the machine says I am outputting 24 one volt. We may use a calibrated meter to a national I' 25 standard, NIST standard, and we tell the machine no, you are O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612.K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 l
~ t 114 1 really outputting-1.1. volts. The machine will readjust q 2 itself. It comes out and says now I am outputting one. We i 3 say.yes, you are right. 5 4 MR. LEWIS: The time base standard can be measured-5 against the standards that are in the air, so that's okay. 6 The voltage standard is a built in voltage standard. I am l 7-just wondering about the potential for common mode failure' 8 if the standard goes bonkers. 9 MR. MASON: I guess the standard that I am l 10 referencing is the NIST standards of one volt equals one j 11 volt. We use calibrated volt meters that are calibrated 12 yearly, to detect what one volt is. Then, we'use those I i 13 meters to calibrate the machine on a quarterly cycle. () 14 MR. LEWIS: When you say continuous self-il i 15 calibration, that's against a one volt standard somewhere in 16 the machine. i 17 MR. MASON: Yes. 1 .i' 18 MR. LEWIS: The one volt standard that is in the 19 machine is either a battery or an IC or something like-that. 20 I' don't know what it is. Whatever it is, if it does i 21L something bad does that mean that everything within sight l 22 goes bad with it? 23 MR. MASON: No. 24 MR. LEWIS: Why not? 25 MR. MASON: All it would do would be to give you a O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
.= l-i 115 l /~T 1 trouble alarm that tells you that you have a drifting k_/ i 2 channel. We get the trouble alarm, go into the_ rack and 3 investigate what the cause of the trouble alarm is. 4 MR. LEWIS: I am trying to understand what happens 5 if everything drifts at once, because the calibration 6 standard -- 7 MR. MASON: In a single channel or an entire 8 protection set. t 9 MR. LEWIS: It depends on how many things are j i 10 dependent on this standard. l t 11 MR. MASON: Okay. r 12 MR. LEWIS: Should I know the answer to this 13 question without asking it? () 14 MR. MASON: Maybe not. 1 1 15 MR. STIMAC: Rick, I think the answer may be as l 16 simple as understanding that although the equipment has } 17 these capabilities, we didn't change any of our existing i 18 technical specification surveillance requirements to take i 19 advantage of these. These are an added benefit. I 20 So, you would still continue to have technician { 21 surveillance go in, in addition to self-calibration to j 22 identify any significant problem where you had a number of 23 channels that drifted out. 24 MR. LEWIS: If the standard drifts then everything 25 that depended on that standard will drift in real time. o 't ~ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
1 116 i 1 MR. KERR: Do you have one standard for each-j O'. 2 channel? 3 [ Slides.] 4 MR. MASON: Here, this is what I was going to show j 5 you. Here is kind of the physical layout. What you want to l l 6 see is down low. This is basically how Eagle-21 fits into 7 the plant. It fits into these four protection sets. These 8 basically represent your four protection channels. Each one 9 of these protection sets is made up of independent racks. ( ) 10 There are four in here, four in here, three and three. 11 Eagle-21 fits into each one of these racks. So, I .i 12 have four Eagle-21 in here, four in here, three in here and l 13 three in here. 14 MR. LEWIS: Each of those has its own standard. 15 MR. MASON: Each has its own standard. Each i 16 machine runs totally independent of the others. They don't 1 17 communicate. There is absolutely no interconnection with i 18 the exception of the power feed that feeds them from the i 19 same instrument bus. 20 MR. LEWIS: Presumably, there's a three out of 21 four or two out of four, or some kind of logic. 22 MR. MASON: Sure. These all feed out into~the 23 logic trains. 'l 24 MR. LEWIS: I understand. 25 MR. MASON: The single machine failure, yes,:would i \\ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Etreet, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
1 1 117 .t J~ 1 drift all.the channels in that rack. j i 2 .MR. LEWIS: I understand. 3 [ Slides.) 4 MR. MASON: Eagle-21 provides a platform for. i 5 potential future upgrades. Some of the other utilities that _i l 6 have used Eagle-21 have performed some of these upgrades. t 7 It allows to do RTD bypass elimination and a couple of l _1 8 others very easily. It also provides a capability for test I 9 in bypass without lifting leads or installing temporary 10 jumpers. We felt like that is a significant benefit, so I i 11 when we are in testing you don't end up in a one out of i 1 12 three coincidence versus a two out of three of your 13_ remaining channels. ) 14 I guess one point is, we did not implement this at 15 this point. We had a problem with some test requirements on 16 our relay protection systems, so.we didn't implement that. 17 But it does give us the capability. 1 18 I want to talk just real briefly, and I guess I 19 don't want to try to get into a 50.59 argument at this 20 point. I want to talk about our experience and explain how I i 21 we got to where we are. We knew that there was some concern e 22 on NRC's part about 50.59 and digital applications under 23 that. We realized that there is no formal guidance given by 24 the Commission. on how you should or should not proceed \\ 25 according to 50.59 on digital upgrades. i I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 i
~ i i l 118 1 When I did my 50.59, I interpret 50.59 from a very 2 system level, more of a system level standpoint than from a l 3 component level. If you interpret the way that some peoplc 4 want us to, any time that I change a resister type or change 5 gate valve to a globe valve you have in effect introduced a-6 new failure. I did my 50.59 from a very high level system 7 level standpoint, and drew the conclusion that failure modes 8 at a system level were the same. 9 Eagle has dead man timers and things like that to 10 ensure that the channels go to a trip state -- I should say 11 a preferred state which is mostly tripped'-- in the case of -l 12 a microprocessor failure. I felt like the failure modes of 1 13 Eagle were the same as the existing analog system. r ) 14 We portrayed that direction to the NRC, and they 15 came back and suggested pretty strongly that we don't 16 proceed in that direction. There's a couple of factors that 17 I think led into that. We were on the watch list at Zion, i 18 and a couple of other things. I guess we took their 19 direction and, in response, we did submit a license 20 amendment request. 21 We added a definition into our technical 22 specifications. That allowed the Commission to review our 23 project. The Commission asked us to respond to a number-of 24 questions. Our initial request for information consisted of 25 roughly somewhere between 16 and 20 pages, of about 300 1 I I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 i . ~... -..
1 119 1 questions. We prepared a pretty. detailed license amendment } 2 request, submitted that, and then that led us into the rest i 3 of our schedule which I will talk about a little bit later, j 4 At that point we did schedule a technical NRC audit'to be [ 5 held here, in Rockville. 6 I will talk just'a little-bit about the audit. -It i 7 was a full, five day audit.- It started Monday morning 8 bright and early, and ended Friday evening. We had six l 9 support people. I had members from Westinghouse, I had 10 Commonwealth Edison people and a couple of architect 11 designers with me. There were six fulltime people and, as I 12 say here, numerous others via phone. Everybody from our 13 corporate telecommunications department to answer microwave I () 14 questions. We had Westinghouse software designers answering 15 software questions. We got very good support from inside i 16 our company and inside Westinghouse and others that we 17 talked to. 18 The NRC consisted of five auditors. Those f l 19 auditors were made up of two NRC people and'three j 20 contractors. Each one of the auditors had his own agenda. 21 In other words, I had one guy that looked at software 22 qualifications for five days. That's'all he did. The 23 audits that we have historically seen was kind of like this; 24 everybody sits at the same table, software gets up and 25 presents, the guys ask the questions that they have. The 1 ANN RILEY & ASSOCIATES, LTD. 1 3 Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 -(202) 293-3950 j i .m.
120-g 1 next guy'gets up and presents, he asks the questions, and 2 you move on. 3 It's not the way this worked. This worked where I 4 had five guys in five different meeting rooms, and all we 5 did was answer each one of those guys' questions for five 6 days. We did a lot of jumping back and forth between. rooms, 7 trying to scramble to get information. 8 These are the five major areas that we had guys 9 interested in. I had one auditor that was very interested l 10 in software qualification. He did thread path audits. He. 11 did software V&V plan reviews, everything to do with I 12 software. I had another individual that was interested l -i i 13 pretty much only in hardware design and equipment 1 () 14 qualification. He did nothing but ask seismic questions'and i 15 humidity and temperature questions for five days. i 16 MR. LEWIS: Before you go too far, just backing j i 17 off for one section to software qualification. Did the guy 18 actually go through by line of code. I 19 MR. MASON: Yes. j 20 MR. LEWIS: How many lines are we talking about l f 21 here? 22 MR. MASON: Good question. I don't know.
- Zita, 23 do you have any idea?
24 MS. YURKO: My name is Zita Yurko, with 25 Westinghouse. I was part of the audit review team that we o ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 ] (202) 293-3950 i 't E'
I i 1l 121 } 1 had down here with this Eagle-21. The software auditor was 2 'primarily interested in one of our algorithms which we use 3 for protection function. Of how many lineslof code', I am I 4 not certain. Thread path did end up.taking several days, in: 5 which he compared lines of code, checked lines of code with 6 design specifications and design processes to ensure that' 7 the V&V process met everything. 8 MR. LEWIS: That was an audit of the procedure. 9 He did the thread paths, but did he get down to the line of l l 10 code level? I 11 MS. YURKO: He basically did review the source 12 code listings, yes, j 13 MR. LEWIS: For that particular algorithm? '( 14 MS. YURKO: For that particular algorithm. i 15 MR. LEWIS: Then the question is, how many lines 16 in that algorithm? 17 MS. YURKO: I can't answer that right now, sorry. 18 MR. LEWIS: That's okay, thank you. 19 MR. MASON: Our third auditor was interested j 20 mostly in plant procedures; how we were going to implement 21 Eagle at the plant. From a training standpoint, our 22 training procedures, our calibration procedures, our 23 testing. They were interested in plant start up tests and-f 24 things like that, i 25 Another individual that was interested almost l [ t ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 [ Washington, D.C. 20006 i (202) 293-3950 3
I 122~ I solely in electromagnetic and radio-frequency interference. 2 Along those lines he asked a lot of power systems questions; t 3 bus loading, auxillary power system, compatibility, l 4 grounding, ground system, ground buses, lightening 5 protection, those types of power oriented questions. 6 The fifth auditor was involved mostly asking 7 functional diversity. I have it listed here'as functional t 8 diversity.versus equipment diversity. Our perception was -- 9 right or wrong -- our perception was that they were getting 10 confused between functional diversity and equipment 11 diversity. What seemed to start the question was the fact 12 that we had done some rack consolidation. i 13 We went from 16 to ten. In other words, in 14 protection set I went from five racks down into four, doing l 15 the same function. That seemed to trip him up and started-l 16 him asking the diversity question. We got the impression i 17 that they misinterpreted the word " functional diversity" and 18 was getting it confused with redundancy..Be that as it may, l 19 that was an issue. -1 20 MR. LEWIS: Is there an assumption, that 21 functional diversity and equipment diversity are good things l 22 and the question is how well you have implemented them? 23 MR. MASON: The assumption is that they are good, ~ 24 and their assumption is that they are required. We agree -i 25 with that. Their question was, had we degraded functional -l t i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,'N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950-i i
123 1 diversity by reducing the number of racks that we were 2 performing the protective function in. 3 Also, taking it a step farther, instead of having 4 independent channel separation now through hardwired 5 modules, I now had multiple channels being processed in a 6 single microprocessor or computer. Their concern was that 7 now if the single computer fails I have now failed eight 8 channels instead of just one. 9 MR. LEWIS: That's redundancy, not diversity. 10 MR. MASON: I agree. That's what we perceived 11 their questions as, they misunderstood the definition of the 12 two. i 13 MR. LEWIS: I am challenging the unspoken ) 14 assumption that diversity is per se a good thing. The 15 example I always use is an airplane with a prop engine on. l 16 one side and a jet engine on the other, which is diverse is 17 all get out but not a very sensible thing to do. That 18 presumption is built in there, the presumption that 19 diversity is a good thing, even if it's confused with 20 redundancy. 21 MR. MASON: Yes. I guess Westinghouse plants are 22 designed with some amount of diversity built in, steam flow 23 channels might farther back up trip to steam level or water i 24 level. l~ 25 We didn't feel like we jeopardized any of that l' LO l 5007 RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N-W., Suite 300 Washington, D.C. 20006 (202) 293-3950
124 [ 1 because, again, we didn't' arrange channels and we didn't i .O-t 2 rearrange loops. Even if you postulate a single 3 microprocessor failure, even though you take eight channels j i 4 down, we could still before Eagle postulate a single { t 5 instrument bus failure which takes a whole protection set-i 6 down. t i 7 So, we didn't feel like we had in any way i 8 jeopardized any kind of redundancy or diversity that had 9 been previously built into the plant. t 10 MR. LEWIS: I am just raising the question of 11 whether it isn't good for safety, to jeopardize redundancy 12 and diversity in some cases. That's a subject we won't 13 resolve here, so let's go on. i 14 MR. MASON: Good. 15 [ Slides.] i 16 MR. MASON: As a result of the audit there were ~~ i 17 basically five -- I will summarize them into five major 18 items. They are listed here. The first question was the-19 EMI/RFI qualification adequacy. I will touch on'all five of 20 these, and then I will come back and discuss each one. 21 The second was concurrent, multiple MMI use. The 22' third one, they had some software walk through items that 23 they had some concerns about. There was a rack blower i 24 maintenance issue that was open. Finally, this functional l 25 diversity, equipment diversity discussion that I don't think o -ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 o a
125 L 1 1 today, is resolved. I 0 5 2 I will touch on these just one at a time real 3 quick, to let you understand what we perceive _the issues i 4 were. For EMI/RFI qualification I had done -- let me back i 5 up. We had realized that EMI/RFI could contribute to the [ 6 reliability of a computer. If you walk into a room and key 7 a radio, obviously, you could cause not only the computer 8 but any analog system to fail or give you erroneous results. 9 I had some individuals inside our company that do 10 this for a living. They are the guys that come out if you 11 call the utility and say your TV buzzes whenever-you turn 12 your blender on, they are the guys that out and find out 13 why. I had these guys come in and map our rooms, to try to- ) 14 identify any harsh areas or harsh conditions where.we are 15 going to be installing the equipment. 16 They came out and didn't find anything. So, based l l 17 on engineering judgment we said because of the controls we 18 already had in place and the area is a radio exclusion area, 19 we felt like that there was no reason-to concern ourselves 20 with furthering EMI/RFI issues. 'l 21 The Commission contractor came back and said let t 22 me see your equipment that you did your map with and let me- -23 see your test procedure, your mapping procedure. We hadn't-- i 24 done a very good job of actually developing a procedure. We i 25 sent him the specs on the equipment that we used to map it. 1 l 1007 RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 126 1 It wasn't very accurate. It's not a quarter of a million i 2 dollar piece of equipment. It's a $5,000.00 piece of 3 equipment. 4 I guess the results had some uncertainty 5 associated with them. We realized that but we discounted 6 it, because our results were so low compared to where the 7 tolerance level of the equipment was even at errors, it 8 didn't even get close. That didn't satisfy the auditor, so 9 we were forced to go back and write a very detailed mapping 10 procedure, and spend $40,000.00 to hire a contractor to come 11 in and actually perform a very detailed map. I 12 MR. LEWIS: Did he say why he wasn't satisfied? l t 13 MR. MASON: Yes. He said that the equipment we I 14 used had a very high inaccuracy associated with it. t 15 Therefore, how could be believe the results that the 16 equipment produced. 17 MR. LEWIS: The way you used your hands, it looked 18 as if you were several orders of magnitude below the 19 threshold for the equipment vulnerabilities. 20 MR. MASON: Correct. That's what we believed, 21 yes. .I 22 MR. LEWIS: He thought that your equipment may 23 have been inaccurate by several Ord:rs of magnitude? 24 MR. MASON: No. On the order of 20 percent. 25 MR. LEWIS: Then, I don't quite understand his O (, ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,.N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 ] v e -- e s o-w, r-
~ ~... 1 -1 127 1 rationale. That's not my job. I 2 MR. MASON: We didn't either, but that didn't seem i 3 to matter. We spent the $40,000.00 and gave him a very I 4 detailed map and a very detailed test procedure, producing ( 5 some very detailed results with some very expensive 6 equipment. The issue was put to bed. 7 MR. LEWIS: Did you say that he was a contractor? { 8 MR. MASON: Yes, sir. ) 9 MR. LEWIS: So, he was representing the NRC in i 10 this matter. i 11 MR. MASON: Yes, sir. 12' MR. LEWIS: You didn't choose to appeal to the 13 NRC? Again, forget it. Go on. 14 [ Laughter.] 15 MR. MASON: Yes we did, but to no avail. 16 MR. LEWIS: I see. That's interesting. 17 MR. MASON: Another issue that was outstanding 18 after the audit was multiple man-machine interface use. We 19 had proposed the installation of multiple test carts. Like 20 I said, all the plants up to us had only used one test cart 21 per unit. So, this was something different than the 22 Commission had seen in the past, and they started asking a-23 lot of questions about how do you know that you are not 24 going to degrade multiple protection channels at the same 25 time. How do you know you won't have more than one channel i l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 q (202) 293-3950
I 128 I 1 in test at the same time, and those types of questions. L 2 We have.had to go back and prepare a write up. .I l 3 think once we explained to them a little bit more about the i i 4 architecture of the system and how this device actually fits. l I 5 into the Eagle-21 system, these concerns were put to bed 6 pretty easy. 1 7 Software walk through items. Basically, we can .l 1 8 summarize it by saying that they identified some misspelled j 9 words in some comment sections of the code, totally non-i 10 executable code. They said that based on the fact that you { 11 have misspelled words in your code how do we know that 12 doesn't imply a breakdown in the V&V process. -j 13 We had to go back and re-justify the entire V&V } process and program based on a couple of misspelled words in 14 J 15 the comment sections. We did that. Again, I know we did a 1 16 good enough job that we put this issue to bed. 17 On the rack blower maintenance, this was just a 18 simple misunderstanding of how we were going to handle j 19 maintenance on the cooling system inside the cabinets. That-I 20 one was extremely easy to resolve. This bottom one we + r 21 talked about previously, and I imagine we will continue to \\ 22 talk.about for the next year or two, until this one gets'to 23 bed. This one still is an issue by the way, at Diablo 24 Canyon in their Eagle audit. This is still an ongoing. 25 concern. l 4 l ANN RILEY & ASSOCIATES,.LTD. Court Reporters 4 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
129 i 1 [ Slides.] 2 MR. MASON: As a follow up to our audit, these are l 1 3 the four things that we basically committed to do. l 4 Obviously, we submitted a written report that responded to l 5 every concern that the Commission had. We did perform 6 additionally, EMI/RFI mapping. The key word here is 7 additional, f 8 We provided a detailed defense in depth analysis. f 9 I will come back to that one in a second. We did commit to I l 10 provide a start up test report which produced all the 11 results of our start up testing. We also committed to t 12 provide a periodic performance report. This was pretty much 13 consistent with what the previous Eagle plants had done. () 14 The Commission was concerned about not only the 15 installation and operation but the performance.of the ~ 16 equipment after it was installed. We agreed to provide i 17 performance reports for the first operating _ cycle of both t 18 units. 19 If I may, I will back up to the defense in depth 20 analysis. That analysis was done, to try to address a 21 concern that the Commission had on common mode software 22 failures. The postulation was that all ten Eagle racks -- i 23 every rack in the unit -- dies in an unknown state, non-24 detectible. We were asked to provide a discussion and a l 25 justification which would indicate that we could detect and 1 i ANN RILEY & ASSOCIATES,-LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
J .a -2.-
- - tl LL a E-
_&A*a_ i l i 130 i ' ~ -- 1 mitigate the accidents in our FSAR. -i =' 2 We spent a lot of time and money on this 3 particular defense in depth analysis. We had a senior i 4 reactor operation that just come off shift. She spent weeks i i 5 at the simulator, running simulations of line breaks with no' 6 reactor protection. We spent a tremendous amount of time 1 7 and effort. Zita was instrumental in putting this defense l 8 in depth together, also. We provided-that, and the 9 Commission was happy with our results. j 10 As a result of all of these items we did get a l 11 favorable SER. From what I can understand that SER is l 12 viewed as a guide now for future installations, which makes l 13 me pretty happy. If I can save somebody else a lot of work. 14 I think that's great. 15 [ Slides.] ) 16 MR. MASON: I won't spend long on this' slide. I i 17 just wanted to give you a feeling for the kind of timeframe 18 that we worked under. We notified the Commission in 19 September I believe, that we were going to approach this 20 under 50.59. We gave them some information on the mod in 21 October. In November we scheduled a technical meeting out l l 22 here to talk about our approach. As a result of discussions l 23-in the meantime, we decided to proceed with a license 24 amendment request. We did that December 26th. I remember 25 that day, the day after Christmas. O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 j (202) 293-3950 'l
i 'l 131 i 1 In January, the Federal Register notice, a O 2 proposed finding of no significant hazards. This was a very 3 important step. We knew that there weren't going to be any-4 intervenors that would slow our schedule down. i 5 We were scheduled -- to put this in perspective -- l 6 we were scheduled to start installation in March. We were 7 aware that Commission reviews typically took six months. i 8 They would like more than that, nine months, if they could I 9 get it. We didn't start until October. We knew we were 10 going to be hurting puppies. We started the technical audit 11 in February and concluded that. We started actual physical 12 installation in March. We were still getting requests for 13 .information while we were doing installation. ) 14 We had a plant tour. We had a very high level 15 meeting out here to talk about open items and how we were 16 going to proceed. Finally, June 9th, we did receive an 17 approved SER. I think one point to understand is that the 18 Commission, once we got to this point here, they did a very 19 good job at expediting our review. Like we said, i i 20 historically we had expected six months. They managed to 21 shrink this into three or four months, at least enough to 22 give us a good feeling that we weren't going to have any 23 major stumbling blocks. .i 24 '[ Slides.] _ l 25 MR. MASON: Just a couple of conclusions that I l ) ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W.f-Suite 300 Washington, D.C. 20006 (202) 293-3950
132 1 will draw based on Eagle before I get into our diesel [,_ \\~ 2 generator discussion. I think that one of the things that 3 we learned is that you have to talk early with the NRC. 4 When you do that, try to discuss any undocumented concerns 5 they may have. We were aware that they had somegconcerns 6 about 50.59. We didn't talk with them early enough, it's 7 that simple. 8 If you are going to entertain an audit -- 9 actually, no matter what you do, you have to do your 10 modification in a very detailed and precise manner, and 11 document it. I think it's important -- this bullet and the 12 last bullet kind of go together -- it is important to 13 understand that we don't change the way we do business just i 14 because we have an unreviewed safety question or just I ('J) \\- 15 because we are expecting an NRC audit. 16 My modification was done. When we got 17 notification of the audit, all I did was pack my files in a 18 suitcase and jumped on a plane and came out. We didn't do a 19 single extra calculation. I didn't do a single extra 20 drawing. I didn't do anything additional to prepare for 21 this audit. 22 I think all of this, the results are that our 23 project was installed on time in the first outage and 24 actually ahead of schedule on the second outage, based on 25 mostly some installation lessons learned. O~~ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i i 133 f i 1 1 [ Slides.] 2 MR. MASON: Let me talk real quick about our i 3 diesel generator controls. I am going to shift gears here a ~ 4 little bit. Because of increasing attention to diesel 5 generator reliability and some new regulatory requirements, 6 we are going to be required to monitor diesel generator 7 reliability. 8 We at Zion, have five diesel generators, two that j 9 support each unit and one is a common diesel that swings _to I i 10 either unit. That common diesel is a killer, because tech 11 specs -- if it goes out tech specs on both units are f 12 affected. We had had some instances where that old diesel 13 had taken both units down. We felt like it was important to-14 address not only that engine but the entire lot of five. t 15 To do that, our engines were old. We felt like we [ 16 had to replace existing pneumatic control system. They were [ 17 installed _20 years ago with the engines, back when pneumatic 18 controls was about the only thing you had. We were having 19 problems with some of the pneumatics. They leaked or they i 20 were very difficult to trouble shoot, and things like that. 21 We also had some control relays that we were -l 22 having problems with. They were a plug and socket type 23 relay. We were having bad connections in sockets. We were 24 seeing some other problems. We felt like we needed to 25 replace those. s i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washingt'on, D.C. 20006 (202) 293-3950
t 134 =! 1 In addition to just replacing the actuation + 2 components of the system we felt like we needed to add some r 3 process monitoring capability. That capability would allow 4 us to monitor fuel oil pressure, lube oili pressure, jacket i 5 cooling water temperature, those type of things. By storing 6 that information and then doing performance tracking or ] 7 trending we could hopefully do a little bit better job at 8 preventive maintenance. 9 These were the basic goals or intent when we j 10 started with the modification. Obviously, to accomplish i 11 these we decided to install a new control system and a new 12 monitoring system. Those could be -- at this point it could 13 have been any_ combination. One machine could do both. We. () 14 could split it into two machines, however you wanted to do 4 15 it. l 16 We knew that our performance monitoring machine l 17 had to be relatively powerful. We were going to be 18 installing 80 or more transmitters or sensors on the engine i 19 to monitor different process parameters. That was going to 20 require a very high volume of data processing and storage 21 capability. All of these considerations went into our 22 conceptual design of the modification. 23 Just so you kir.d of have a feeling for our 1 24 schedule, we are scheduled to install three engines, control i 25 systems, this fall, starting in October. Then, the'other l. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
-. ~, .,.~ 'I 135 l t 1 two engines in the subsequent outages. 2 I guess one other clarification I need to make is 3 that we are not replacing our sequencer. This is only the .i 4 control, the actuating circuitry of the engine itself. The 5 sequencer are still the way they are. 6 MR. LEWIS: Before you take it off, you stated 7 that viewgraph -- your second line actually -- it says I 8 reliability to meet NRC requirements. You said new 9 requirements when you said it. Which requirements are we + 10 talking about? i 1 11 MR. MASON: I believe there's a Reg Guide 1.108 12 that's coming out that talks about diesel generator + 13 reliability. Steve, are you familiar with the actual 14 number? t 15 MR. LEWIS: It's an upcoming reg guide that you 1 16 are talking about. i 17 MR. MASON: That's my understanding. 18 MR. LEWIS: A reg guide, of course, is not a 19 requirement. j 20 MR. MASON:
- Correct, 21 MR. LEWIS:
If it's upcoming, it's especially not 22 a requirement. r 23 MR. STIMAC: Dr. Lewis, as a point of 24 clarification. We have had some traditional-problems with 25 our diesel reliability at Zion Station. So, beyond whatever (1) Plai RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W.,~ 20006 Suite 300 Washington, D.C. (202) 293-3950 i i -f
136 j 'l new requirements there were, we were very cognizant of the 2 need to improve the reliability of the machines to meet 3 existing requirements, including our target reliabilities 4 for station blackout, the number of initiatives like that. 5 We have had problems with them over the course of 6 the years, and we recognized the need to improve l 7 reliability. Interestingly, at one point in time -- this 8 was several years ago -- we did an analysis of the failures j 9 that had occurred, trying to attribute them to certain I 10 elements of the diesels. We looked from as far reaching as ( t 11 replacing all of.the diesels entirely. 12 We found approximately 50 percent of the failures 13 could have been prevented or were attributable in some way, l ) 14 to the' control system. A lot of them were_ difficult to 15 diagnose because of'the antiquity, if you will, of the 16 monitoring system. It's a good business decision when you 17 have diesels in your tech specs that can cause you to shut 18 down the power plant also. I 19 The new requirements and in fact the reg guide, 20 there have been some changes in how that is going to be 21 handled anyway. So, I would say existing requirements. 22 MR. LEWIS: I was only reacting to that because 23-obviously -- and I am not against improving the reliability 24 of diesel. generators, for God sake -- I also have grave 25 reservations about the ability to monitor the reliability of i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite'300 Washington, D.C. 20006 (202) 293-3950
~ P 137. ~ 1 them. -The new reg guide is heavy on the monitoring the 2 reliability as is the station blackout rule. It's a 3 different subject. That's why I asked the question. 4 MR. MASON: Thank you, Steve. 5 [ Slides.] d 6 MR. MASON: During the conceptual design phase of j 7 the mod -- let me back up. From here on, I am going to talk l i 8 mostly in the next couple of. slides mostly about the safety 9 related actuation control circuitry of-the engine itself. I i 10 will ignore the monitoring portion which is non-safety. 11 We had a couple of different ways we could i 12 approach the control system. One of them was the use of 13 programmable logic controller. That.was the first thing 1 ) 14 that came to mind. Programmable' logic controllers are 15 widely used. They are commercial pro' duct. We talked about 16 this, this morning a little bit. I.would much rather have 17 commercial product that has 15,000. devices on the market 18 than I would a special custom built product that only has v 19 five. 20 So, we knew programmable logic controllers were i 21 out there. They are very cost-effective for our 22 application. As a matter of fact, they are designed 23 specifically for the type of application we were going to i 24 use them for. They are designed to replace relay logic 25 control systems. t ANN RILEY &-ASSOCIATES, LTD. l Court Reporters { 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 l '(202) 293-3950
e l 138 '( 1 However, a drawback, we recognized that it would I 2 require commercial-dedication of that product. Another j 3 option we had was to use an Eagle-21 system. We knew that 4 it's a proven product. It is licensable. But Eagle is not 5 designed to do relay logic. It's designed to do more of an j 6 analog loop conversion and set point calculation and those 7 type of things. It was going to take a substantial 8 modification to the software and hardware to be able to do l 9 what we wanted it to do. That was going to be very i i 10 expensive. 11 We determined that Eagle-21 was not cost-effective ~ f 12 for our application. The other option was just to replace 13 or existing relays with a new relay system. We can identify l () 14 relays that we know perform better than the ones that we ( ) 15 have. We can get rid of relays that have sockets, that 16 would eliminate one problem that we have today. I mention l i 17 here that relays are widely used. They are very cost-l 18 effective and' easy to install. 19 However, the mechanical operation was part of the 20 problem that we had now. We have contacts that get dirty i 21 and things like that. We wanted to get rid of that if we 1 22 could. Relays provided absolutely no kind of communications-23 capability. In other words, they couldn't talk to the j 24 control room computer or wherever we wanted them to talk to f i 25 and communicate, their diagnostic status or anything like j l 1 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,.N. W., Suite 300 l, Washington, D.C. 20006~ (202) 293-3950
139 1 that. Communication was going to be very cumbersome if we 2 decided to implement it. 3 These were basically the three things we looked at l 4 4 and considered to replace our control system. i 5 MR. LEWIS: Would it be fair or. unfair, to say' 6 that the only argument against using programmable logic' ' !i 7 controllers was that you would have to convince the NRC that 8 they were reliable? j 9 MR. MASON: I think that's 80 percent-true. There 10 are a couple of other small problems; spare parts -- 11 MR. LEWIS: I will buy 80 percent any day. l i 12 MR. MASON: I would say 80 percent, yes. That' led l i 13 very well into my next slide. We did decide that a PLC was j (f 14 probably the best choice for our application. 4' 15 MR. LEWIS: Let the record show, I have not seen 16 your next slide yet. l 17 [ Laughter.] l 18 [ Slides.] l 19 MR. MASON: For some of the reasons I mentioned on 20 the previous slide, we felt like a PLC was perfect for what 21 we needed this to do. However, we knew that when we entered 22 into licensing space we were going to have a very difficult 23 time producing formal design documentation on the PLC. As a' q i 24 matter of fact, we even at one point investigated using a 25 NSSS supplier in conjunction with'a major PLC manufacturer,- O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 140 1 to provide a qualified PLC. 8 2 That major PLC manufacturer declined to 3 participate because of the requirements that were going to 4 be imposed on him. A lot of commercial vendors, they make 5 their livelihood out of designing the best product and. 6 keeping that product a secret from their competitors. This } 7 vendor did not want to expose his product to the kind of 8 scrutiny and detailed audit that was going to be required. j 9 We also realized that they don't do a lot of I 10 formal product testing. They do it, but it's not very well l 1 11 documented. It's not necessarily very thorough. They use a 12 lot of on the job experience, if you will, to use their 13 customer base to help them figure out what's wrong with I () 14 their machines. i 15 Realize also, that their product has.to be good or l ~ 16 they go bankrupt. Even though it's not documented, that l 4 17 doesn't mean that it isn't a good quality product. 18 I mentioned this earlier. These two bullets kind l 19 of go together. A commercial vendor, since heLis business 20 driven, will typically buy the cheapest product he-can get. 21 That goes everywhere, from cases down to resisters and 22 capacitors and circuit boards. As long as that component 1 23 meets his design specifications, whoever is cheapest today I 24 he will buy from. We knew that was going-to be a1 problem in t 25 safety related Class IE space, was getting any kind of- -I 4 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612'K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
-.as s_. i 141 ~ j 1 consistent component traceability. 2 Again, that would lead in the future into r 3 procuring spare parts. Some of the options are you buy ten l 1 4 extra PLC's today, qualified. You hope those last you long l 5 enough. Another option is, you pay somebody to qualify them t 6 later when you need them. It's kind of a muddy mess to get l 'f 7 into. 8 These are some of the pretty serious drawbacks 't t 9 that we saw in a PLC. We took those and thought that maybe 10 we can address some of these. We will see. Then we started 11 looking at previous SER's that had been generated for 12 commercial products. 'There's a Woodward 501 governor that i 13 was installed at a utility and some load sequencer. Let me () 14 clarify what I am about to say as perception on my part. I !~ 15 read the SER's, and helped make this decision. i 16 I perceived that by the words the NRC wrote, that I 17 tl.,v wer'n't real happy with commercial dedication, 18 especially in the case of the 501. It seemed very apparent l 19 that they did approve its use. However, they left a huge 20 door open for enforcement action on commercial dedication, 21 should they choose to do so later. 22 The words that were used in some of the SER's 23 combined with this, combined with some of the questions that 24 we were asked on Eagle, it.didn't give me a good and warm 25 comfortable feeling that we could do any kind of dedication I ANN RILEY & ASSOCIATES, LTD.. 'I Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
142 ] 1 in licensing process in the' kind of timeframe that we needed 2 to do it, and in a cost-effective manner. 3 We looked at the Eagle-21 questions that we had 4 been asked, and that's where I got some of these bullets. I 5 knew that we couldn't provide any kind of design documents. 6 I knew that we couldn't produce testing documentation. All 7 of this, together, didn't leave me a real good fuzzy 8 feeling. 9 MR. LEWIS: I hate to keep interrupting you. 10 These issues of the usefulness, I don't know why people use 11 the word dedication, the usefulness of commercial equipment 12 to serve essentially low demand users of electronics, these 13 are not problems which are unique to the nuclear business. (() 14 In particular, the Defense Department has been worrying that ( 15 problem for a long time. 16 There was, as I recall two or three years ago, a 17 major Defense Science Board study on exactly this question 18 which must -- I haven't read the report so forgive me -- 19 must have gone through this entire list and produced a lot 20 of wisdom. Do you refer to it? 21 MR. KERR: Excuse me. Would you get that 22 microphone a little closer to you. I can't hear you, Hal. 23 MR. LEWIS: You don't want to hear me. I really 24 do apologize. Anyway, the Defense _ Science Board and 25 probably other groups within the Defense Department, have O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D. C -. 20006 (202) 293-3950
=i 143 I ()' 1 looked at this because their Mil specs are another matter. \\s/. -2 There was a lot of fuss I remember during Vietnam, that the ) 3 equipment used in Vietnam still had to meet the minus 55 4 degree centigrade requirements on temperature reliability. 5 In fact, commercial radios were widely used in Vietnam and l 6 -worked better than the military radios. ] 7 But the'DSB within probably two or three. years ago. l e 8 did have a major study on.this subject, the extent to which 9 commercial equipment which is manufactured in large i i 10 quantity, tested in a variety of environments just because 11 it goes out there handled by dunder heads and all those good j 12 things, may be cost beneficial. It went through all these f 13 things. ) 14 You, presumably, didn't refer to this. I don't .1 15 know whether the staff did or not. I guess not. It's worth I 16 looking at the DSB report. In fairness, I haven't'seen it 17 but I know the guy who chaired it, and he's a competent guy. ~ I 18 MR. MASON: All right. Thank you. ) i 19 MR. LEWIS: It's unclassified, I think. ) ~ 20 [ Slides.] a 21 MR. MASON: Taking all these things into ).i 22 consideration that brought us'to a decision point..As I j 'l 23 talked about earlier we still felt like'a PLC was probably 1 24 .the best for our application. Big word, however. Based on 25. cur experience during Eagle-21 the burden: associated with () I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington,'D.C. 20006 (202) 293-3950 J
1 144 l 1 the qualification'of commercial PLC to meet NRC expectation -l 2 -- I think the key word here is expectations. I didn't put l 4 i 3 requirements there for a particular reason. 4 .To meet NRC expectations would'not be practical'or l.i 5 cost effective for CECO or vendor. We didn't want to take f 6 the challenge on ourself. We had vendors decline to j 7 participate. We chose not to do it. t 8 That brings us to where we are at today. i 9 MR. DAVIS: Excuse me, before you go on. Did you { 10 attempt to explore the use of the PLC with the NRC-on an 11 informal basis up front or was this entirely your own 12 perception of what would happen? 13 MR. MASON: Perception. There were.no formal l ( 14 communications or even informal, really, between us and the t' 15 Commission. 16 MR. DAVIS: Thank you. 1 1 17 MR. MASON: Realize, some of these decisions were I 18 being made even as we went through our Eagle-21 audit. I 19 There were some informal discussions maybe, between myself 20 and some of our~ auditors and things like that,;but I would 21 have to answer that' question'no. 22 MR. DAVIS: Thank you. 23 MR. MASON: You mentioned a key word, perception. 24 I was taught in a management training' class,.that perception ~ 25 is reality. If a customer thinks that his-electric bill is. O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 l Washington, D.C. 20006 (202) 293-3950 j a -- --..l
145 i 1 too h',a no matter what I tell him,_he's still going to 2 think his electric bill is_too high. I think the key word j i 3 is perception. 4 Based on all those decisions and the information i 5 that we had at the time, this is where we ended up. I am 6 installing a brand new relay based control system. I am 7 installing a brand new cabinet that has a number of brand s 8 new relays in it. Obviously, we feel like those relays are 9 an improvement over what we have now or we wouldn't be 10 spending the money. 11 It's not as good as I think it could have been, if. j 12 we would have used some other equipment. 13 We weren't totally discouraged away from using ) 14 digital control systems. We realize that digital-control 1 15 systems are very powerful. They have great data i 16 manipulation capabilities and storage and things like that. l i 17 So, what we did decide to do was on the monitoring side i l 18 which is the non-safety data acquisition side, I did use a i 19 distributed control system. That system will be used to 20 gather information from all these process sensors, store it,- f 21 trend it, display it and things like that. -i 22 We are taking advantage of some modern techno3cgy-23 here. We are using redundant fiber optic data highways, i 24 running throughout the plant. We.are using sun micro t 25 computer workstations which are very high powered, high l C:) i' ANN RILEY & ASSOCIATES, LTD. Court Reporters I 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202)-293-3950 l i
5 146 1 speed machines. We are using optical disks, to provide data i 2 storage and retrieval for trending purposes. We are also 3 using this distributor control system to monitor relay 4 positions. 5 I actually hardwired out contact positions into l 6 digital inputs into this DCS, so that will aid us in our 7 trouble shooting effort. We will now be able, by our l 8 sequence of events recorder, be able to tell which relay 9 actuated before which relay and in what sequence. It should 10 help us in trouble shooting quite a bit. j 11 One huge advantage that we have in using a DCS, is f 12 the fact that it allows process parameters to be displayed 13 in a graphical format. We are installing a CRT, a 19 inch -l ( 14 CRT in each diesel generator room. The operator will be l 15 able to walk in and see graphics that are designed to look l 16 like P&ID's. They will have pump status indicators, tank 17 levels, all indicated by graphics, colors, flashing if they 18 go into alarm and those types of things. 19 At a quick glance he can look on the CRT if 20 everything is green and he's happy. If he sees a tank level 21 flashing red that means it's too high or too low. So, we 22 feel like that's going to be a great aid to the operator, 23 and help him be able to operate the engines and be much more -i 24 efficient at his job. 25 MR. LEWIS: One of the great breakthroughs in I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington,.D.C. 20006 (202) 293-3950
l i 147 l aircraft instrumentation -- I remember once and forgot what 1 2 airplane it was -- somebody had the ingenious idea of 3 setting the flight instruments on the panel in such a way l 4 -that the normal position for each needle was horizontal. t 5 Therefore, you could scan the row of needles and if you Ei 6 didn't see a straight line something was wrong. It was very 7 easy to acquire that kind of information,- even if you have i 8 100 dials there. { 9 The real reason I interrupted you was that this i 10 list of things that I have in front of me -- and I really i 11 know no more than appears on the list -- says that Turkey _ 12 Point digitized their EDG sequencer in 1991 successfully. l 13 MR. MASON: That's right. They thread their SER-14 last night. 15 MR. LEWIS: Does that have any relation to what 16 you are doing? i i 17 MR. MASON: Their job was totally different. The 18 scope of their job was different. 19 MR. LEWIS: It was t 20 MR. MASON: Yes. i 21-MR. LEWIS: I really don't know what their job l j 22 was. 23 MR. FmSON: The scope of their job was different. 24 However, it is comparable, in that they did use a'PLC, 1 25 commercially available product. ~! I ANN'RILEY & ASSOCIATES, LTD. i Court Reporters l 1612 K Street,.N.W., Suite 300 I Washington, D.C. 20006 i (202) 293-3950 l i
148 1 MR. LEWIS: They did use PLC's? 4 ~ 2 MR. MASON: Yes. 3 MR. LEWIS: And, had no trouble with it? 4 MR. MASON: I wouldn't say that. 5 MR. LEWIS: Okay, thank you. 6 MR. MASON: I see that I am running short on time. 7 I will skip a little bit of the presentation material-on 8 some layout of our system and go directly to the last slide 9 that you have. 10 (Slides.] 11 MR. MASON: I am going to close at least my formal 12 prepared comments with three basic bullets that I think-we 13 feel are important. As identified this morning by D.C. .( )- 14 Cook, there is very little formal guidance on the I 15 requirements to license of digital system today. 16 There is~a lot of old standards. There are'a lot 17 of documents that are ten and 15 years old, that don't 18 necessarily apply to today's equipment. Because of this and 19 because we are getting-smarter -- we are asking better '20 questions -- we know how machines work better. We know a 21 lot-of things that we didn't ten years ago. We are asking-22 different questions. 23 The answers to those ~ questions aren't fua these old 24 requirements. However, we are being held responsible to-25 answer those. So, as a result of this first-bullet there-is O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202)- 293-3950
=. ) 149 1 a significant uncertainty that exists with trying to license ' O' 2 a digital upgrade today. l 3 This leads to the final bullet, where I think-it's 4 important to understand that the industry today is facing a ] 5 new challenge. We now have competition. We have 6 independent power producers out there that can slap a .j 7 combustion turbine in and produce power in six months. That 8 didn't used to happen. We now have to become very good at i 9 running our plants economically-and safely. I don't want to f 10 imply that economics would ever get in the way of safety; it i 11 won't. i 12 However, there comes a point of diminishing 13 returns, where you have to make an intelligent decision on ) 14 what equipment you install in your plants and what your i 15 expected payback is. So, future industry decisions, I 16 guarantee, will be made based on economic justification. 17 Any u certain and fluctuating licensing 18 requirements will discourage the use of the best equipment 19 for the application, as I think is evidenced in our mod. 20 Facts aside, maybe perception is maybe the key word. We' 21 didn't perceive that it was going to be possible to license 22 a PLC, so we chose to go a different direction. l 23 That concludes my formal prepared presentation.
- I n
24 would be glad to entertain any questions or comments. ] 25 MR. LEWIS: Questions. O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K-Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
l 150 1 MR. DAVIS: 'Did you do a 50.59 determination on 2 these diesel modifications? 3 MR. MASON: Yes. 4 MR. DAVIS: You determined there was no unreviewedi [l 5 safety question? j ^ 6 MR. MASON: Yes. 7 MR. DAVIS: The staff agreed with that, or didn't 8 challenge it. 9 MR. MASON: The staff has not reviewed that as far l 10 as I know. I 11 MR. DAVIS: Your last sentence there is a little .l 1 12 bit provoking, I think. I don't think that you can make { 13 that sweeping statement, that this always happens. l (I 14 MR. MASON: Which statement? i 15 MR. DAVIS: The last one. I think it's possible 16 that it could discourage the use but that seems to say that I 17 it does it every time, that fluctuating license requirements 18 always discourage the use of the best equipment-for the 19 application. I don't think that's true every time, 20 certainly. 21 MR. MASON: I wouldn't argue with-that'. That's 22 probably true. For our case it-was true. 23 MR. STIMAC: Perhaps we could say;may discourage. i 24 MR. MASON: May; discourage. 25 MR. LEWIS: I should think you would rejoice about 4 ANN RILEY & ASSOCIATES,-LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 .(202) 293-3950
151 l ~ 1 uncertain and fluctuating licensing requirements gives free 2 reign to your own initiative and inventiveness. 3 MR. DAVIS: Provides flexibility, right? 4 MR. MASON: I won't comment on that. I wish that i t 5 was true. j 6 MR. LEWIS: Are there other questions? Pete, were 7 you finished? 8 MR. DAVIS: Yes, thank you. i 9 MR. LEWIS: Thank you very.much. ] 10 MR. MASON: Thank you. t 11 MR. LEWIS: We will press on. According to my i 12 schedule Pat Place is now going to give us his perspectives 13 on the proposed guidelines; is that right? () 14 MR. PLACE: Indeed, I am. -4 15 MR. LEWIS: Can you do it in 55 minutes and get us i 16 back on schedule, Pat? i 17 MR. PLACE: I will try. As it happens, it may 18 come as a surprise, the SEI does have some bureaucracy. I 19 apologize to the Committee'that I have no prepared slides.: J 20 This is in part due to the difficulty of getting approvals 21 in the short time that I have available for this. I stand i 22 up here to give my comments. That way,.you-can see who I .) 1 23 am. You can throw your rotten fruits here. .I will say, j 24 that I stand behind my comments. 25 We have.seen two and one-half of the versions!of I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 1 Washington, D.C. 20006 (202) 293-3950 -j
152 (J 1 the guidelines; a version from NUMARC; a commented on 2 version from the NRC; and then, the outline of a new version 3 which is coming out from NUMARC. I am grateful to the 4 people who sent me the outline last week. It gave me some 5 chance to read what it is that they are planning to do. I t 6 think there is still some good things in there, and there 7 are still some things that need to be done. 8 There are a lot of topics that have surfaced 9 reading these documents and it's difficult to present - a 10 coherent picture of the whole, because there's a whole bunch 11 of stuff in there. I have lots of small comments which I 12 think will provoke discussion rather than -- I hope they L 13 will provoke discussion, amongst those concerned and perhaps I 14 give the Committee some view from the software person's view 15 of what all this stuff is meaning. I mean, I am a software 16 developer and engineer, call it what you will, and not a 17 nuclear engineer. But I am beginning to understand some of 18 your concerns. 19 one of the major topics in the guidelines is that 20 of a threshold. This is clearly a point of contention. 21 NUMARC, in their original discussion, don't have the concept 22 of a threshold. The NRC additions add them'in. In the ~ 23 latest document that I saw-from the NUMARC side it seems 24 that they are once again moving away from the notion of a 25 threshold. I can understand this. ~ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612'K. Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950-o
153 1 First of all, these are really difficult to deal i 2 with in the sense that they will prejudge whether something l 3 is a non-reviewed safety question or requires NRC approval. 4 The NRC approach is to say here are certain systems, that if f 5 you are touching these systems -- if you are making upgrade 6 in these systems, then we want to improve this which sounds 7 like you are prejudging the event, 8 on the other hand, if you don't take this. measure 9 which is by location in terms of the software, what can you 10 do. Can you say if we make a change that is of this 11 magnitude then we pass some threshold. The difficulty is, 12 what does it mean to make a change of some magnitude. I 13 think we have no real way of measuring function and no real 14 way of measuring divergence from function. It becomes F f-15 really difficult to say I am making a one percent change to 16 this software. Maybe we could measure that in terms of l 17 lines of code but that isn't always a safe way to do things. 18 I would like to remind the Committee of the 1991 19 outage of the AT&T's long distance signaling, which was a i 20 two line change after they had done all of their V&V. They 21 had done all of their testing, they had their. equipment they 22 believed safe in their terms correct and reliable, and then 1 23 somebody made this final addition after the 12. week process l 24 that they went.through to do their upgrade. 25 Anybody who tried to make a long distance phone O I ANN RILEY & ASSOCIATES, LTD. Court Reporters i 1612.K Street, N.W., Suite 300 ~I Washington, D.C.'20006 I -(202) 293-3950' 1 i
154 j i call with AT&T that day will recall the chaos and the i 2 impossibility of doing so. j l 3 Threshold is a good idea, but I am not sure how 4 you apply it other than in terms of location. That goes 5 back to the industry's objections to prejudgment. The other l 6 problem is, given that most designs or many designs will 7 differ there will be different pieces of any individual 8 upgrade which you will have to say yes, that is part of our 9 sacred software or threshold software; that, if you touch 10 that you are going to need staff approval. I suspect t 11 there's going to be -- if such an approach is taken, there l 12 will be a lot of effort involved in determining what is the 13 sacred software or the threshold software. ) 14 Another really big question that comes up-in the 15 guidelines is that of common mode failure. It's clearly an 16 important issue. I have just a couple of comments with l r 17 respect'to that. 18 Analog equipment suffers from common mode failure 19 as much as digital equipment suffers from common mode ] l 20 failure. I think you need to distinguish between' design l l 21 time failure and execution time failure. Clearly, a design 22 time failure which is what we try to guard against in terms ) i 23 of common mode failure analysis in large part, it says that j .24 we have made a mistake. It says we have made a bug in our 25 design, and the implementation in hardware -- as an analog i LO: ANN RILEY & ASSOCIATES, LTD. i . Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 ')
- l
.2 155 1 piece of hardware, as a digital piece of hardware or 2 software, is all going to suffer from that bug. If it's 3 ever exercised we could have some~ serious consequences. 4 In terms of design time failures, common mode 5 failure is a problem no matter what you are implementing in 6 it. t 7 In terms of execution time failure, then you have~ 8 a difference. Clearly because the replication of the 9 hardware elements or the physical elements, those are likely i 10 to fail in their execution differently. A piece wears out,. 11 something drifts too far. Whereas with the software, it 12 will all fail the same way. That's because it's going to 13 fail because of a design time bug. I think you need to sort ) 14 of somewhere think about the distinction between design time 15 and run time. 16 One of the approaches to run time -- of mitigating 17 the problems of run time fault in software is that of 18 diversity. It's not clear, whether diversity really helps. 1: 19 I am going to come back to this issue. 20' The other problem with respect to software;is that-d 21 there is a tendency to concentrate function in a single 22-microprocessor or collection of. microprocessors. _This is j 23 going to increase your probability of design time failure --1 f 24 common mode failure if your processor fails. That's-25 obvious. I actually like the look_of the Foxboro ( ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W.,' Suite 300 Washington, D.C. 20006 (202) 293-3950 y,
156 1 instrumentation where you have all these hundreds of 2 modules, each of which can fail independently of each other. 3 To some extent you are not concentrating all of 4 this software function into a single unit. It's still quite i 5 diverse. f 1 4 6 The NRC talks about reducing the amount of review j 7 that might be necessary when you change or modifying a 8 previously approved design. I think this is something that I 9 is slightly dangerous. I go back to the notion of the AT&T i i 10 signaling system, where you had a previous design -- it had 11 been approved and somebody made a modification, and we had 12 that chaos. 13 There have been other cases where the same thing () 14 happens in terms of the local Bell offices also, in 1991. l i 15 They had the same problem. Their suppliers upgraded. The 1 16 signaling system and a number of the local Bell offices.went 17 out around about the same time. I know Bell of PA in 18 Pittsburgh was unable for a day to get-any calls through 19 locally. These sorts of problems where you modify an 20 existing design and don't review it as carefully'as.you 21 previously did, look like they could be danger spots. 22 When you change the context that a system exists 23 in you are changing the assumptions it was written, under 24 which it was approved. I think you run into the danger 25 again, of making essentially a design change even though'you i i ~ l ANN RILEY & ASSOCIATES, LTD. 4 Court Reporters 1612 K Street, N.W., Suite 300 l Washington, D.C. 20006 ~l -(202) 293-3950
157- { 1 are taking an existing piece of software that'you proved in. 2 one application, if you move it into another application I. 3 think you run into some potential danger unless you check. 4 all of the assumptions for the new system against thoseaof-S the old system. 7 6 One of the things that I liked about the 7 guidelines is that they started to say some things about the ~ 8 design, the specification and implementation process. I. 9 think one of our earlier speakers today described it as'all 10 the legal stuff and all the stuff that tells you what'to do t 11 As a software person I cannot help you a whole lot with.the I 12 legal stuff, though reading 50.59 and readingLthe questions t 13 based on the 125 standard, I have some comments. ={ j) 14 The design time stuff, specification time stuff-t i 15 and implementation time stt ff really; interested me. I am i 16 assured by.the standard 2.5, the. outline of the new one, 17 that there will be more emphasis on'some of that 18 information. I think that's good. Unfortunately, as-an' 19 outline, it's very difficult to do. We are going to beef up 20 this section and that's a good idea, but unless you know how I 21 .they are going to beef it up it's a bit tricky to say i -22 whether that is a good way of beefingLup the section on l 23 configuration control or whatever. 24 I was a little disappointed in the; standards that 25 are being referenced. The-standards that-I looked at were LO ANN RILEY & ASSOCIATES, LTD. i Court Reporters 1612 K Street, N.W., Suite 300 , Washington, D.C. 20006 (202); 293-3950 1
t i 158 1 the ANSI standards, 1016 and some other number. They are [ 2 the ones on software requirement specification and software-3 design descriptions. t 4 To my mind, these standards are very weak l 5 standards. They describe the types of sections that should 6 be in your documents. They make a small amount of-l 7 reference to the way you should be representing informatici 5 8 in those sections, but they are weak. As an industry ? 9 interested in safety it would be better to look at stronger 10 standards. I am a fan of 0055 and 0056. I believe that t i 11 those standards say something very strong in terms of I 12 techniques and technicalities, and will lead to safer j 13 systems. ( 14 They mandate the use of mathematics to describe ( 15 your systems and to continue to design your systems as 16 opposed to natural language. I think you described earlier, 17 the notion of how wonderful it was to have all these 18 standards written or this stuff written in natural language 19 that was as ambiguous as possible so that you could get as l 20 much agreement. That's probably not what you want in terms l 21 of safety. I think you have to be warned there. 22 The discussion of failure mode analysis, I think, 23 is really good. I think that's the right way to go. I-t 24 think failure mode analysis of the software is, perhaps, 25 going to be the approach that will pay off the best. Rather l ANN RILEY & ASSOCIATES, LTD. l Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 I f = m -m-e w w a re - - * - +
- +w ee
.e w- -m--- .re-a,
l ? 159 1 than aim for error free software or perfectly correctly 2 software, let's look for safe software. Those two are i 3 different. 4 If we can demonstrate that there are unsafe states 4 5 that could axist in our software or that an unsafe state 6 cannot be reached in our software, then we know whether cn: i 7 not we could approve it. 4 8 The last speaker talked about' commercial item j 9 dedication. This is clearly a problem. It was a problem to 10 me as I was reviewing these documents, in that the different l 1 11 perceptions of what review can and should be done -- I don't-12 need to say anything more than the previous speaker-said, i 13 He said it all; that there is a real problem and if there is () 14 a perception that the staff won't accept commercial items on i 15 trust based on the fact that they are out there, then they 16 won't get used. The perception is that that will lead them l i 17 into unreviewed safety questions or the failure'to get i 18 there, a good evaluation report, so they won't do this. 19 MR. KERR: You don't think some sort of way of 20 sprinkling holy water on components might do the trick? 21 MR. PLACE: I wish it would. I suspect you would 't i 22 see a number of them would go up in smoke though because H 23 they are not all error free. As I was speaking just before i 24 we started this afternoon's session, I have yet to find a 25 commercial' component that does not have some errors in it. i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 =..
160 No matter how r.any times it is being used -- the particular 1 2 example I have is a compiler on an AT&T UNIX machine. 3 That's.an old compiler. It's based on AT&T's first UNIX 4 compiler, PCC. 5 A couple of years ago I move some software to it 6 and found a bug that, when I reported it to the AT&T people 7 they said that's interesting, we didn't know that one. This 8 was after a number of years of people using this same 9 compiler. I wasn't doing anything particularly difficult in 10 my code. It was just something that this particular 11 compiler on this particular system failed. 12 MR. LEWIS: I often use the example of a compiler 13 bug I had a while back which only failed every 512th time it ()- 14 was used to compile something. It was a terrible bug. I 15 MR. PLACE: That's actually -- we can talk about 16 compiler bugs or bugs in programs -- 17 MR. LEWIS: Your general point is well taken. 18 There is, of course nothing without-bugs, including -- 19 MR. PLACE: Indeed. All you have to go for is ) 20 what is the best thing we can do. 1 21 MR. LEWIS: For that reason you haveito quantify ~ ] 22 what you are doing. Of course, what.I was joking about i 23 going for vague standards, I wasn't serious. 24 MR. PLACE: I understood that, too. The issue is 25 that if something is-ambiguous you can get agreement.when I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 -.. -.-..=...
161 f( 1 people are agreeing to different things. I think that is 2 the important issue, that when you agree to something you 3 better both be agreeing to the same thing. I 4 MR. LEWIS: If the degree of freedom that involves 5 contestants thinking they are agreeing to different things 1 6 didn't exist, chere would be no agreements. 7 MR. PLACE: I don't think I am going to add to ~! 8 that one. 9 MR. LEWIS: Please go on. We are on software. 10 MR. PLACE: Right. There is certainly a question 11 with commercial items. How do you analyze them, how much l 12 can you analyze them, especially if vendors aren't prepared 13 to provide any further information. That does lead to some () 14 notion of concern. 1 I 15 As I said, I am really going to stand here and 16 raise questions rather than be able to give you any good i 17 answers, I an afraid. ( 18 This brings us back in a sense, to diversity. In 1 19 some senses diversity has been a holy grail for some time, i 20 and people use redundancy -- that's one form of diversity -- 21 and they use functional diversity where you actually use 22 different software running on different processors, to; 23 ensure that maybe you-have some increase in safety. 24 There is reference to that within the guidelines. 25 My recollection is that that's an NRC addition. I--am not I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington,.D.C. 20006 (202) 293-3950
i 162 f~. 1 going to stand firm with that. 2 Nancy Levinson talking to you now, will tell you 3 that diversity doesn't help. She has done the experiments 4 to prove it, or at least to demonstrate'it. I recommend, 5 that people go back and re-read some of her experiments with i '1 i 6 respect to diversity. I am inclined to believe what she's 7 saying. i 8 I think there is a form of diversity that can be i 9 helpful though. Hal, you raised the point about the plane-10 with the jet engine and the propeller engine. Yes, you have 11 diversity in implementation but not diversity in function. l 12 You have the same function being employed by each engine. I 13 think more interesting is the sort of work that-Lui Sha is ) 14 doing at the SEI, where you have a control system which may 't 15 be very complex and controls some system by a complex set of-16 controllers. 17 There are a number of ways to implement that, You 'f 18 can implement it with the full accurate, as accurate as 19 possible and as stable as possible, control algorithm which, v 20 because of the complexity of the algorithm you can't be 21 convinced is safe or will it always operate and will always 22 work, and doesn't.have bugs. Equally well, you can take j I 23 very. simplified set of control laws and you can implement 24 those, and you.can put a lot of effort into the simplified 25 control laws into'the implementation of them, and say yes we 1 O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
163 l 1 can' guarantee that this simplified set is correct. It won't + 2 give me the accuracy and stability of the complex set. 3-You can now use the very simple algorithm to bound 4 the complex algorithm. Indeed, Lui is doing that and has. j l 5 some experiments with some really neat little toys with~some 6 very unstable systems, lots of vertical holes that are 7 hinged at the bottom that he can keep upright for as long as 8 the simple control system is running. In~ fact every time to I J 9 this date that he has shown me this experiment his simple 10 controller has never failed to run. Every time the complex 11 controller runs it brings the thing back to the point it's 12 supposed to be because the simple algorithm has some drift. { 13 The last time I spoke to Lui he wasn't quite sure why there l 14 was drift in his simple algorithm. f 15 The point there is, you have real diversity. You l I 16 have diversity of function and diversity of intent. He's 17 not intending to have two really complex systems doing the 18 same thing. He's got one very simple system bounding the '19 complex system. I think that is interesting, in' terms of _j 20 diversity. 1 21 The general diversity that Nancy has~ checked, I am 22 prepared to believe her numbers, given that she has had 23 enough of the people involved in diversity chasing after. 24 her. Her work stands out. I 1 25 MR. LEWIS: My memory -- correct me if I am wrong l i I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300. Washington, D.C. 20006 (202) 293-3950
_~. 1641 1 -- is that what her experiments showed in software diversity 2 was that for any problem'there are one or two tricky points
- j 3
that people tend always, no matter who they are, come to 4 these tricky points and tend to make the same mistakes at 5 these tricky points. Therefore, the diversity involved in 6 having different groups write independently is just not. i 7 there, even when you'think it is. 8 MR. PLACE: Right. That was the conclusion that I j 9 drew from that. t 10 MR. LEWIS: That's my memory. 11 MR. PLACE: Right. i 12 MR. LEWIS: The other point, the simple system, I 13 am a physicist. We tell all our students to do sanity () 14 checks on every complex calculation. Think of what it is 15 like in the extreme limit. If the strength of gravity goes 16 up to infinity the thing shouldn't fly off into the 17 stratosphere and things like that. That's the same'as your 18 extreme case, your. simple versus complex system. -j 19 A simple proximate system is a sanity check on a 20 complex calculation. 't t 21 MR. PLACE: In Lui's case it's more than a sanity i 22 check. It's enough to maintain the system-in the state it j 23' should be in. i 24 MR. LEWIS: That's even more so. j t 25 MR. PLACE: It's not enough to maybe keep-the f ANN RILEY & ASSOCIATES, LTD. Court Reporters -1612 K Street, N.W., Suite 300 .j Washington, D.C. 20006 -(202) 293-3950 t J
i 1 165.. {} 1 system operating as efficiently as possible. 1 2 MR. LEWIS: I understand. 3 MR. PLACE: As I said, in one of his experiments ) 4 there is a drift -- there is a little engine -- this sort of 5 vertical pole is attached on to a little trolley on wheels-6 which moves back and forth -- i 7 MR. PLACE: Everyone has written that program. 8 Even I have written that program. 9 MR. PLACE: His simple algorithm has the drift on l 10 the trolley whereas the complex one keeps the trolley f 11 exactly dead center of the track,'which is where it's 12 supposed to be. 13 MR. LEWIS: Mine just runs back and forth, but it 14 keeps it erect. ( ~ 15 MR. PLACE: He's able to' keep it -- when the 16 complex algorithm runs he keeps it center of the track and 17 erect. i 18 MR. LEWIS: Mine doesn't. It runs it around but 19 it keeps it up. -That does raise a question that came into. e i 20 my mind during some.of the talk. I think when we were j R21 talking about the pid controllers, when the pid controllers l 22 showed up -- and you were speaking of keeping things within 'f 23 limits. I wondered why nobody has ever proposed fuzzy-logic l 24 in this business, because in a' sense that's.a great 25 simplification. l 5 1001 RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street,.N.W., Suite 300 Washington,.D.C. 20006 (202) 293-3950--
4-A ? 166 1 I am not a fuzzy logic buff. Usually, when people O1 2 see the words pid, they instantly respond by saying you can 3 do it with fuzzy logic and fewer lines.- It's more reliable 4 and works just fine and so forth. Any comment on that? 5 MR. PLACE: No. I will pass on that one. Another ] 6 whole issue that I am going to pass on with respect to the 7 guidelines and make it now, is on the EMI. I am a software j r 8 person. This is a discussion that is beyond me. 9 MR. LEWIS: Why don't you do something innovative .f 10 then, and talk about only the things that aren't beyond you - l 1 i 11 MR. PLACE: I am going to try to keep it that way. l 12 Again, with respect to commercial item dedication or use 13 --whatever word you use -- again, it goes back to your i i ) 14 example of your firm ware. It's really just software you 15 can't change easily. l 16 There is this issue of, you would like to be able 17 to check the software that is in your system, all of it, and 18 that does include the firm ware. Again, that raises this 19 issue of just how easy is it going to be to check some of 1 20 the software, be it firm ware or whatever. 21 I did actually digress briefly into some of the 22 legal stuff with respect to 50.59, when does it apply. It 1 23 seems like it applies pretty well every time you are going 24 to do a digital upgrade, that you have to do this review in 25 some sense or run the 50.59. If I read it correctly it said ANN RILEY & ASSOCIATES, LTD. I Court Reporters f 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950' 1 i e
i 167 1 when you change your method of performing the function of a 0; 2 structure system or component described in the software 3 3 safety analysis report. i 4 It seems to me that changing from an analog to j 5 digital system is in fact changing the mode of performing or 6 the method of performing the function. It's a legal j i 7 question. You may have different interpretations, and I'am 8 not going to try'and suggest that you should interpret it as 9 strictly as I would. That was my reading. 10 With respect to 13 CFR 50.59, the fifth section of 11 the document as it currently stands is a set of seven 12 questions which, according to the NUMARC comments which were-i 13 struck out by the NRC, if you answered yes to any of these ) 14 questions you_may have an unreviewed safety question. The 15 NRC says if you answer yes to any of these questions you do j 16 have an unreviewed safety question. 17 I thought it's time to read 50.59. It seemed like 18 the seven questions that you are looking at -- this is just 19 my perception in reading the words -- are merely an 20 expansion of the three clauses of 50.59. It was surprising 21 to me that somebody could make that expansion and then say 22 if you answer yes to any of these questions you only may 3 23 have an unreviewed safety question. t l 24 I throw that out and suggest that somebody was 25 doing the right job in terms of catching that and striking ANN RILEY & ASSOCIATES, LTD. Court Reporters { 1612 K Street, N.W., Suite 300 Washington, D.C. 20006-(202) 293-3950 l l =
i 168 l 1 it and'saying no, if you answer yes you are back in 50.59 + 2 role. Of course, from what I have heard today I understand 3 that you interpret 50.59 in certain ways. Maybe to speed 1 i 4 the review process. Perhaps that was an interpretation that 5 I missed. 6 Under the additional guidance there was discussion 7 of the automatic self-testing and diagnosis. I think this I 8 raises a thorny issue. It's something that you would really l l 9 like to do. It's wonderful to know that if my system is j 10 going wrong -- indeed there are commercial vendors out there l 11 producing highly reliable systems that do not fail without-12 them knowing why it failed be it the hardware or software, i 13 because of the way they are using self-testing and self-14 checking. ( 15 The problem that faces the safety community is l t 16 that that addition of self-testing of diagnosis and-indeed 17 of security controls which you probably need in these-l 18 systems, complicates the software. This makes it harder to 19 analyze because you have a lot of extra stuff in there. It 'l 20 increases the probability of unexpected interactions because 21 there are more things going within your system, more l 22 opportunity for unexpected interactions which could be bad l l 23 interactions. Indeed, the self-checks can be wrong. j i 24 Nancy Levinson and John Knight did another j 25 experiment on self-checking code and they had a bunch of
- t I
ANN RILEY & ASSOCIATES, LTD. j Court Reporters i 1612 K Street, N.W., Suite 300 l Washington, D.C. 20006 (202) 293-3950 ]
i 169 ( 1 students write checks before they had seen the code and { r 't 2 after they had seen the code. The bulk'of the students were 3 unable to detect the errors that Nancy and John had assumed i 4 or determined were in fact in the code. l 5 I think this comes back to the same issue with i 6 respect to diversity. It's a case'of, if the interpretation i 7 of the specification is wrong then it doesn't matter how l many self-checks you are going to put in, because you are 8 9 not going to be checking for the right things. I 10 I am severely concerned by self-testing and f 11 diagnosis software checks being put into the code. One of~ 12 the things that 7 found horrifying was a couple of years ago l l 13 at the December Conference when an AT&T representative said () 14 how many lines of code were in a number five ESS. There i 15 were four million lines of code, two million of which are 16 self-testing and diagnosis. 17 Half of the code that is running in these systems 18 is there for checking purposes. That may well be good, but. 19 it's problematic if you are one of these people trying to I 20 analyze this. You have to see if this is a self-check and 5 21 then does the check do the right checks, and is it 22 interfering with any of the function of the' system. I am a 23 firm believer in keeping it simple and keeping it as simple 1 24 as'possible. 25 I am sorry for the licensees, but if that means 1 4 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 I ~
0 4 .a 170 'l-pulling something.out of a rack and putting it someplace 2 else or pulling a test cart along that I can use to do some q 3 really simple tests on my function, I would rather do'that 4 from a safety viewpoint thin complicate this software'and 5 have the much harder problem of determining whether the [ t 6 software is correct. 7 MR. LEWIS: I understand your point, and let's j 8 address it for a moment. At the lowest possible level then' 9 you are against parity bits. 10 MR. PLACE: No. 11 MR. LEWIS: We have an inconsistency here. Would 12 yua like to resolve it? i 13 MR. PLACE: It is not an easy problem'to answer. l 14 I am against something like AT&T's two million lines of 15 code. 16 MR. LEWIS: I would have escalated up to all sorts l l 17 of error correcting codes if you would have said no. I f 18 understand that. 19 MR. PLACE: I knew that.at some point I would have l 20 to jump over a threshold and say yes. 21 MR. LEWIS: You have to draw the line somewhere'. 22 Where is there a line in the sand. 5 - 23 MR. PLACE: I think you have to draw some lines. 24' Clearly, you want to be able to put in some sort of 25 capability for testing. You have to be careful how you do l ANN RILEY & ASSOCIATES, LTD. Court Reporters '1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
171 i:i 1 that. You have to be sure-that you don't rely on self-2 testing and diagnosis as your main mode of determining l 3 whether in fact this piece of software or this system is in t 4 fact in error. 5 MR. LEWIS: As you know, most errors in the public-l i 6 codes that people use in banks and stores and offices, are i i 7 typing entry errors, human entry errors. There is a i 8 tremendous amount of diversity and of error correction built. 9 into the coding for those things, just because that's the l 10 most common kind of errors. 11 Presumably you would agree, that self-testing for 12 the most likely errors makes a lot of sense. It. buys you 13 more than it loses. If it's put in for its own sake by ) 14 computer scientists, that is another matter. 15 MR. PLACE: That's clearly another. matter. There ~ 16 needs to be a happy median. I don't think I can stand up 17 and tell you what I believe it is, without reviewing the 18 specific system and saying for that system I think this sort l t 19 of check seems sane and this sort of check seems insane. 20 MR. LEWIS: To use the old Bernard Shaw joke, once 21 you have agreed to parity you know the principle'is 22 established. f 23 MR. PLACE: I will reverse myself. I will throw 24 out parity. Maybe not. Clearly, we are worried about 25 software failure. If we are going to count. software ANN RILEY & ASSOCIATES, LTD. Court Reporters' 1612 K Street,.N.W., Suite 300-Washington,.D.C. 20006 (202) 293-3950 l
i 172' { 3 1 components into a system we have to worry about the i 2 reliability of them. l 3 We have no really agreed upon method -- in fact, I i 4 am not sure of any method that is good for assessing the ( 5 probability of failure in software. I don't know of any way 6 I can come along and say here is a piece of software. It's 7 this many lines big and therefore it has this much 8 probability and this complexity of function, it's got this i 9 probability of failure. I don't know how I do that. 10 I don't know of any reasonable way to test for l 11 high levels of reliability. If any item in fact needs to be l 12 highly reliable I don't know how I can test for that, 1 13 because I don't have the physical time. Again, at the 1991 () 14 ACM conference Ricky Butler presented a paper on this based 15 on his work at NASA. 16 If I recall his conclusions once you get to 17 testing ten to the minus four or ten to the minus five of 18 reliability levels, you are running into more-time than is 19 plausible for doing these tests. 20 MR. LEWIS: To use your own terms, if you are 21 talking about reliability in execution as distinguished from 22 reliability in design -- 23 MR. PLACE: Right. He's talking about reliability 24 essentially in execution. Sometimes that will throw out a \\ 25 design time bug. O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 ~ . (202) ~ 293-3950
173 1' MR.-LEWIS: I understand that. In fact, my i 2 understanding is that most software problems are not 3 problems -- you have distinguished between run time and ~ 4 design time and writing time. I have forgotten your 5 terminology. 6 Most of the bugs I know are not execution level 7 bugs, not run time bugs. They are design bugs that you 8 discover at run time. 9 MR. PLACE: Right. Again, there are multiple 10 level of bugs that you can introduce in the process of going 11 from some notion of what it is you want to'do, to the 12 executing code. There is a design bug where you introduce a 13 serious problem, and then there are implementation bugs from 1() 14 that final design specs into your implementation you may 'i 15 introduce further problems. Those are usually easy to 16 recover. 17 MR. KERR: Back to your. problem of assigning 18 probability to software reliability or assigning a number. 19 I just finished a book written by a well known author which 20 has a short supplement on probability. I learned that the 21 probability of something like this can be described by what 22 the experts think it to be. 23 MR. PLACE: I am prepared-to state that if you 24 want to assign a reliability factor you can do so to any 25 piece of software. Stick your finger in the air and see 10 l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i j i 174 I I 1 what the answer is, or make it what you need it to be. 1 2 You can maybe see this piece of software is likely 4 3 to be more reliable than that piece of software, _if you [ 4 believe that the process by which the software was developed j 5 was one that induces reliability so that it went through i 6 good reviews. This comes back to the issue of should the l i 7 guidelines talk about process. I believe they should. 8 Should they reference documents that are stronger than the 9 ones that they currently do, I believe they should. f 10 I really do think that the IEEE standards, 1016 . i 11 that they reference in the existing guidelines, are very 12 weak. I was disappointed in them. 13 There's the notion of error free software versus i ) 14 safe software. These are distinct concepts. Software 15 doesn't have to be error free to be safe. We.can examine t i 16 software for error freeness. We have software fault tree j i 17 analysis, which is another excellent tool for analyzing l i 18 code. I 19 It comes back to one of the issues, compiler bugs 20 aside, what level do we trust. Do we trust my programming j 21 language. Do we trust my implementation. Do we trust the 22 microprocessor. You have to pick a level at which you trust 23 your system and say assuming that that conforms to my views 24 that it works the way I expect it to work, then I can i 25 analyze at this level. i ANN RILEY & ASSOCIATES, LTD. i o Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 e . ~, - - -s%--am--., v ... w e , 3 -,,-yz -w.c -tr%. -w e- ~p r
175 1 I am a great fan of software fault tree analysis. 2 I wish Professor Levinson were here because we sometimes i 3 disagree on this issue. I think she did the right thing 4 with them. 5 I think I talked briefly about the notion of i 6 probabilities for software. I think you can assign some j 7 sort of scale, very loose scale, with respect to software as 8 to whether you believe it's more or less reliable based on 9 the process that you go through. The stronger the process l 10 the more the reviews, the less likely it is to have errors. 11 Be warned, a software walk through was described 12 comicly but accurately by one of our colleagues at the SEI, 13 as the process where one designer shares his misconceptions ) 14 with the rest. You have to be very critical, clearly, in .I 15 the review process. It would be wrong to state that the 16 review process in fact produces correct code. j 17 In the same section he talked about formal 18 verification. That is the demonstration that an incorrect 19 program satisfies an incorrect proof. You have to be 20 careful there. All you are doing is some other way of 21 describing the function of the system. An incorrect proof b 22 is easy to achieve, very easy to achieve. It may agree with 23 your incorrect program. It's not a catch all. It's a way H 24 of increasing your confidence. 25 My conclusions with respect to the guidelines'was . O 1 ANN RILEY & ASSOCIATES, LTD. Court Reporters j 1612 K Street, N.W., Suite ~300 R Washington, D.C. 20006 i (202) 293-3950
l 176 ] 1 that there weren't enough guidelines as yet, that help a 2 developer in the creation of safe systems or of systems that 3 will smoothly pass through the review process. There will 4 be times when people will want to make an upgrade that will i 5 require a review. Again, I like the last. speaker's comment, 6 that he didn't have to do anything else between his design 7 and the requirement of the NRC audit other than stick all l 8 the relevant papers into the briefcase and bring it out 9 here. That seemed to be a good state to be in, that he was j 10 at least satisfied with the review process. To strengthen 11 the review process he will have to do something more next-12 time. 3 13 You do need stronger guidance on techniques. I am ) 14 sorry to say this again, but the ANSI standards are too 15 weak. They talk around the problem without saying anything j 16 substantive. I like MOD 0055. It says something i 17 substantive. It's unfortunate, that it remain be tested I 18 that people can develop to that standard. It s a safe f 19 standard. It's a standard that we can strive toward. 20 The ANSI standards form a base level which is 21 below something that one would consider acceptable for a 22 safe system. l 23 I thought the staff additions on training for { 24 operation of maintenance were actually really good ideas. r 25 You introduce a new type of system potentially with some new ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite.300 Washington, D.C. 20006 (202) 293-3950 l
177 1 read outs and new ways of maintaining it, and that's clearly. L 2 something important and should be in the guidelines. My .] 3 understanding is that that will in fact be the case. 4 I will be pleasantly surprised if the EPRI V&V 5 document has real formal verification in the sense ^that 6 computer scientists mean formal verification. It remains to j 7 be seen, what is in there. I offer the warning perhaps 8 needlessly, but I offer the warning that.it's somewhat 9 foolish to sign off on one set of guidelines that refer to 10 other documents when those don't yet exist or are in 11 preparation. i 12 You are signing off on something that is a moving 13 target. At least it sounds like it's a step in the right 14 direction. ( 15 MR. LEWIS: With the help of good software we know i 16 how to aim at moving targets. 17 MR. PLACE: Sometimes we even hit them. 18 MR. LEWIS: We learned how, when it mattered. ] 19 MR. PLACE: Indeed. I think that essentially 20 concludes the remarks that I had, based on the guidelines. 21 I am happy to stand and take any further questions that 22 people want to throw at me. Maybe I have upset everyone + 23 here. 24 MR. KERR: In your earlier comments I thought you 25 were reading from 10 CFR 50.59, and I couldn't understand O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 178 I where you got the language and I still don't. What I read 2 here says proposed change, test or experiments will be i 3 deemed to involve ur. reviewed safety question if; one, the 4 probability of occurrence or the_ consequences are and 5 accident, malfunction important to safety. j i 6 MR. PLACE: Perhaps I misspoke. I understand what 7 I said, and your question. There is a section -- 3.1 in the l l 8 guidelines -- that says when does 10 CFR 50.59 apply. When-9 should I use 10 CFR 50.59 to determine whether or not what I 10 have is an unreviewed safety question. l i 11 MR. KERR: I think 50.59 is much less' specific. I 12 just want to make sure I understood what you were reading 13 from. () 14 MR. PLACE: In terms of the distributed guidelines 1 15 there's a section 3.1 ~ 16 MR. KERR: As long as you are' reading that I have 17 no problem. 18 MR. PLACE: It just says in here that as discussed 19 in 125, 10 CFR 50.59 requires safety evaluations only for 20 changes to a facility that affect the design, function, or 21 method of performing the function of the structure, system J 22 or component described in the safety analysis report. I 23 read this as describing I am making a change.and should 24 apply to 50.59 questions to the change I am making. 25 It doesn't say what those questions should be. O-ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 i
179 1 1 MR. KERR: Thank you. 2 MR. DAVIS-I have a quick question. Is it 3 conceivable to you that after all of the proper V&V is done [ 4 and-the software is installed, that some kind of virus could i 5 strike all of the software simultaneously based on how many 6 times it's been exercised or some aging effect or some 7 temperature effect or something like that, that would be a 8 cause for a concern over common cause failure. 9 MR. PLACE: It's plausible. This comes into the I 10 issue of credible or incredible. i 11 MR. DAVIS: Exactly. 12 MR. PLACE: Some years ago Ken Thompson wrote a i 13 report on a Trojan horse that could have been introduced -() 14 into a C-compiler. It is an excellent report, how to put -- { 15 if you haven't read this, this is for UNIX types of systems. 16 His notion was that whenever you detect you are compiling 17 the program that allows people to log in, you put in a i 18 Trojan horse that the compiler inserts into the compiled i 19 code. 20 People never see this because they are compiling 21 their code. As they compile their log in code it emits this 22 extra Trojan horse that captures the passwords and installs j 23 them somewhere. He went one step further. He then changes I 24 his compiler so that the compiler inserts that Trojan horse 25 into the compiler and then throws away the source of the i ANN RILEY & ASSOCIATES, LTD. i Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 I
t i i i 180 1 compiler that does that. 2 Now, every compiler that he produces subsequently 3 is affected. He assures everyone that he has never done 4 this. r 5 MR. LEWIS: That was beautiful stuff though. It 6 was very nice. 7 MR. PLACE: If you are concerned at'that' level. 8 which I think is unlikely -- I think it's a very unlikely: 9 event -- yes, it's conceivable. I find it unlikely, that 10 somebody could introduce function into something that I 11 could review that says if my temperature is sustained over a 12 certain temperature for a certain level of time, that 13 suddenly my software will all fail. ) 14 That, I consider to be unlikely, assuming that the 15 software goes through a good review process. .You do have to 3 16 worry if you are taking firmware on trust. Your vendors may i 17 be holding you up. That's an' issue -- I am not trying to. 18 blame vendors of hardware or software or fi'rmware. I am 19 just saying it's plausible but I'think unlikely. 20 MR.. LEWIS: There's still.the point that to have 21 that something introduced into the compiler will never been 22 seen by anyone is a good point. 23 MR. PLACE: Right. 24 MR. LEWIS: But you have to have access to the l 25 compiler. 4 O l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington,.D.C. 20006 (202) 293-3950
? 181-( 1 MR. PLACE: I think you actually have to be a 2 remarkably smart programmer, to be able to do everything 3 that Ken Thompson described. i l 4 MR. LEWIS: Indeed, but he is. 5 MR. PLACE: Indeed, he is. 6 MR. LEWIS: Thank you very much, Pat. I believe 7 we are on our last leg. We are going to switch. gears now. 8 If our transcriber still has the stamina to last through. 9 this one, we are switching to the environmental program now i 10 and how it's.being organized. i i 11 MR. VAGINS: I will try to speed this along. l 12 MR. LEWIS: Spectacular. You will become very r 13 popular. ~ ) 14 MR. VAGINS: Just for those who do not know me, I 15 am Milt Vagins. I am the Chief of the Electrical, 16 Mechanical' Engineering Branch in the Division of Engineering f 17 in Research. I am going to talk Just a little bit about the i 18 update on the environmental qualification study for digital 19 I&C hardware. I am not going to talk about software. There 20
- bs no sof tware involved.
It is strictly nuts and bolts 21 hardware. 22 MR. LEWIS: Nuts, bolts and chips, right? 23 MR. VAGINS: Yes. 24 [ Slides.] 25 MR. VAGINS: Basically, the presentation I am ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C.'20006 (202) 293-3950
182 1 going _to follow is four points: previous directions with ~ 2 ACRS, a very quick research update; a review of the major 3 milestones and schedules; and the response to three specific 4 ACRS concerns and questions which recommendations have 5 raised in the last two meetings. 6 Briefly, just the last several meetings, we met 7 with the Subcommittee on June 16, 1992. We had an in depth l l 8 review, where we discussed the technical regulatory issues, 9 the user needs, the research approach methodology, the 10 details of the research program, and milestones and i 11 schedules. 12 At that time everything was all right except that I 13 ACRS Subcommittee recommendations said include the effects ) 14 of smoke as a stressor on I&C hardware. I believe Mr. 15 Michelson raised that issue specifically. EIt's true, we did-16 not have that spelled out as a specific concern. t 17 MR. LEWIS: To be very, very precise, 18 Subcommittee's do not make recommendations. It is not an 19 ACRS Subcommittee recommendation. 20 MR. VAGINS: An individual recommendation. 21 MR. LEWIS: There was an individual who made that-t 22 recommendation. 23 MR. VAGINS: Right. 24 MR. LEWIS: I am responsible for the product of' 'I 25 this Subcommittee. e ANN RILEY & ASSOCIATES, LTD. ~ Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 t ~. -. - -. -
i 183 1 MR. VAGINS: I took that' individual as a J 2 recommendation since nobody.seems to object. Again, then .3 you will see -- 4 MR. LEWIS: Don't peddle it as a Subcommittee 5 recommendation. 6 MR. VAGINS: All right. The main Committee met on f 7 October 9, 1992. Here, we only were back up. NRR made the a basic presentation. -They discussed the research program, 9 they discussed their user needs, their safety concerns,. 10 their environmental issues, their stressors, EMI/RFI, 11 milestones and schedules. 12 Forget about the beginning. Individuals recommend l 'l 13 that we reassess stressors and rank according to relative 1 l 14 safety significance, learn from the experience related.to ] 15 EMI/RFI from the U.S. Military. I 16 MR.. LEWIS: We have to be very careful. This-one, ] 17 you 'should believe. That was an ACRS recommendation. You 18 are exactly backwards. 19 MR. VAGINS: People have'said that to me many, i 20 many times. 21 MR. LEWIS: You are consistent. 22 MR. VAGINS: Yes, I am. 23 [ Slides.] 24 MR. VAGINS: This is a summary of the NRR comments 25 on October 9th. NRR concurs with the research effort. NRR -l ANN RILEY & ASSOCIATES, LTD. Court Reporters -1612 K. Street, N.W., Suite 300 Washington, D.C. 20006 i (202) 293-3950
g 184 1 at.that time was' issuing an update memorandum to include 2 additional environmental stressors. The only user request 3 we.had. physically in hand at the time dealt with the 4 EMI/RFI. They had told it was coming. It just took a while 5 to get off their desk. 6 It was updated on February 22, 1993, so we have 7 the updated user request. NRR supports the development of 8 uniform guidelines for qualification of digital I&C hardware 9 to ensure regulatory stability in the process. You heard 10 several speakers today, who said their primary' problem was 11 lack of regulatory stability. What are the guidelines. 12 What are the requirements. What do they have to do. We 13 are trying to do that for the NRC. 14 MR. LEWIS: Speaking only for myself and therefore 15 not an individual comment, regulatory stability takes second 16 place to safety ~in my book. 17 MR. VAGINS: Absolutely. There is no question 18 about that. I take that as a given. 19 MR. LEWIS: Except people keep saying as you just 20 said, regulatory stability is the primary objective. It '21 isn't, really. It would be good. 22 MR VAGINS: Accepted. Just a real quick update 23 since the last time we spoke. We have developed a 24 methodology for identifying functional environmental issues 25 that apply to I&C as differentiated now from analog O 4 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K-Street, N.W., Suite 300 Washington, D.C. 20006 '(202) 293-3950
5 185 1 equipment. 2 We have developed environmental stressor templates f 3 for Westinghouse AP-600, General Electric ABWR and SBWR, and 4 the ABB CE System 80 Plus. We have issued for internal 5 review -- it has not gone anyplace but inside a few people 6 in the NRC -- draft NUREG CR-5904, environmental 5 7 qualifications and functional issues for protection systems, 8 just the protection systems, in advanced reactors. 9 We have studied issues related to fiber optic 10 technologies and reactor I&C systems. That has been issued: 11 in a Oak Ridge Letter report, TR 9313. We have surveyed 12 some experience of U.S. Military with EMI/RFI testing. We-13 have begun an in depth survey of European experience'with 14 digital protection systems. 15 MR. KERR: Just out-of curiosity -- l 16 MR. LEWISI For the military, you have read i 17 reports?' 18 MR. VAGINS: We have read available, non-19 classified documents. 20 MR. LEWIS: You didn't speak to anyone. 21 MR. VAGINS: We are getting there. To speak you 22 talk about classified. I 4 23 MR. LEWIS: I understand. 24 MR. KERR: Just out of curiosity, why does one 25 separate environmental qualification from other ~ l ANN RILEY & ASSOCIATES, LTD. i ' Court Reporters-1612 K Street, N.W., Suite 300 -Washington, D.C. 20006 (202) 293-3950
a 186 f 1 qualifications. There are a lot of important qualifications 2 which equipment should have, and it would seem to me that 3 environmental qualifications are not necessarily any more 4 important than any of a variety of others. 4 5 MR. VAGINS: I agree with you. I am not quite 6 sure -- we were tasked with the environmental. 7 MR. KERR: If you just look at one set of i 8 qualifications there is no chance that one will conflict 9 with the other if you don't look at them as a package? 10 MR. VAGINS: We are not looking at functional 11 qualifications in the sense of software. We are not looking 12 at -- we are looking at operational hardware qualified for 13 our environment. (G) 14 MR. KERR: What you are looking at is a way of 15 specifying equipment or systems that will have some 16 characteristics. I would just worry a little bit about 17 concentrating on one facet of this problem. Maybe it's not 18 an issue. I just leave it at that. 19 MR. VAGINS: We will come back to that if 20 necessary. I will continue with the research update. 21 Progress has been made in defining electromagnetic field 22 envelopes with digital I&C systems in nuclear power plants. 23 This is,_ again, published in an Oak Ridge letter report, the 24 same one that I mentioned before. The other one is 93/13. 25 This one is 93/3. There's two different letters. I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
187 1 We developed technical basis for regulatory 2 guidance on susceptibility of digital systems to EMI and RFI 3 and that's in a draft NUREG which is now being issued for 4 internal review, CR-5941. 5 MR. KERR: If I go back to the first bullet, what 6 am I supposed to learn from that; that you haven't forgotten 7 about it? 8 MR. VAGINS: Yes, I suppose. We made some 9 progress. In other words, this is just an update from the 10 last time that we spoke. 11 MR. KERR: All right. 12 MR. VAGINS: We also had a task on isolation 13 devices. We have completed testing of ten isolation devices p} 14 recently. The test results would be completed this week or (r_ 15 next week. We expect a NUREG CR out in December. This is 16 the evaluation of ten different isolation devices subject to 17 not only maximum credible fault but slow burn questions and 18 others that were raised. 19 [ Slides.] 20 MR. VAGINS: Just a reminder of how we are dealing 21 with the template, this is -- in the previous handouts I 22 gave you quite a bit of information. There were five of 23 tbese sheets. This is only one sheet, that lays out a 24 representative coolant flow stream in advanced light water 25 reactor. We talk about functions performed in software and /() ~ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
188 ~ 1 where it is done in various places in a reactor building, 2 reactor penetration and outside. We talk about.various 3 --on this level of stressors, and some of the things that .l 4 have to be done. This is just a way to qualify them. f 5 If you look down this list you will see down here, 6 typical stressors, temperature radiation, moisture, I 7 vibration, maintenance, installation, smoke. We did not 8 have smoke the last time. 9 MR. LEWIS: Before you go on, I am just looking at 10 one number at random. Under EMI/RFI on that last thing it 1 11 says 30 megahertz to 500 megahertz and field intensity of 20 12 volts per meter. 13 MR. VAGINS: That's right. l () 14 MR. LEWIS: Twenty volts per meter is a fairly ] t 15 high field intensity. Where did that come from? 16 MR. VAGINS: Paul, could you help me with that. 17 Paul Ewing, from' Oak Ridge, is one of my contractors. f 18 MR. EWING: The 20 volt per meter level actually 19 came out of the ANSI standards. I think it's C-63.12. 20 MR. LEWIS: Out of the ANSI standards? f 21 MR. EWING: Dut of the ANSI standard. 22 MR. LEWIS: ANSI standard, for what? l 23 MR. EWING: It's an ANSI standard for general'EMI i 1 24 testing of equipment. t i 25 MR. LEWIS: Don't they depend on.the environment? I l ANN RILEY & ASSOCIATES, LTD. l Court Reporters 1612 K' Street, N.W., Suite 300 . Washington, D.C. 20006 i (202) 293-3950 {
1 i 189 .O 1 There can't be any universal number for the -- 2 MR. EWING: It's not. If you look at levels for 3 like test equipment, it's like one volt per meter standard. 4 If you look for general industrial environment, it's 5 typically taken at levels of maybe 10 or 20 volts per meter. 6 MR. LEWIS: When somebody this morning commented 7 about the electromagnetic environment of the plant being 8 well below some of the standards that were being t 9 promulgated, 20 volts per meter in that range which is a 10 wavelength of just about a meter actually, is a fairly large 11 power for a walkie talkie, and you have to be right near the 12 antenna. 13 So, for the ambient electromagnetic field within a () 14 plant, I would expect the levels to be much lower. On the 15 other hand, I can imagine an ANSI standard which has to do 16 with people going under power lines and that kind of thing, 17 being much higher, t 18 Is there a problem of using a standard that is 19 meant for one set of circumstances under a different set of f 20 circumstances, and blindly using an ANSI standard? I don't 21 know that standard. l i 22 MR. EWING: I don't think we are trying to blindly 23 use the standard. 1 24 MR. LEWIS: I just wonder how much is being driven 25 by that 20 volts per meter. O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612'K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
I 190 ^ 1 MR. KERR: If you read the next part of this which .7 t-2 says protection system is reported to operate normally -- I 3 don't know what that means. I don't know if it means that i 4 normally one expects them to be in that field or if it's in 5 that field it will operate normally. But around sources of 6 radio frequency of 20 megahertz to 500 megahertz in a field 7 intensity of ten volts per meter. 8 MR. LEWIS: It says at ten volts per meter. If 9 that's to be believed, the 20 is only slightly above that. 10 But I would guess that the ambient within a plant under the 11 worst conditions would be rather lower than 20 volts per 12 meter. I 13 MR. KERR: It also says this data was -- which we ( 14 will ignore. 1 15 MR. LEWIS: This doesn't say how much lower. 16 MR. KERR: No. I would assume that it meant to 17 say these data were -- that's irrelevant. .i 18 MR. LEWIS: Yes. The question of where the number 19 comes from, it would be nice to straighten it out. What 20 circumstances the ANSI standard was directed at -- because 21 it's obviously -- those standards are set up by a group, and 22 the group has a scenario in mind. It would be nice to know 23 what their scenario was. l 24 Then, it would be nice to know how much mischief 25 is being done by that very large field strain. We can O i IJUI RILEY & ASSOCIATES, LTD. { Court Reporters 1 1612 K Street, N.W., Suite 300-l Washington,-D.C. 20006 -(202) 293-3950 j 1
191 1 straighten that out. 2 MR. EWING: The 120 volt per meter level was 3 actually worst case. 4 MR. LEWIS: I don't know what is meant by worst 5 case, so we are just getting in deeper. 6 MR. EWING: It's actually typically harsh 7 industrial environment. If you look at-the typical hand 8 held walkie talkie at about a meter away it's probably got'a 9 field strip level of maybe five or ten volts per meter. 10 MR. LEWIS: Twenty volts per meter for example, is 11 well above the U.S. standard for irradiation of people. 12 Therefore, any walkie talkie that produces 20 volts per 13 meter near the' antenna should not be used by a person.
- (
) 14 MR. EWING: The ANSI standards which are C-95.1, 15 are actually about 60 volts per meter. 16 MR. LEWIS:. Were are within factorsHof two. I 17 would guess that in a plant it's more like tenths.of ~ 18 hundredths of a volt per meter. There's a wide margin here. 19 I think it's important, not to be so over conservative in 20 setting a standard that you drive major efforts that are 21 really unwarranted. 22 Anyway, we will go into that in more detail. j 23 MR. VAGINS: One of the tasks that were given to 24 our contractors was to evaluate all standards, whatever 25-standards exist, in light of NPP requirements and actual p ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 1 J .1
i 192 i 1 characteristics and background. Since most of these 2 standards were never intended for application to NPP, 3 nuclear power plants. i 4 That was one of the real purposes of this 5 research, was to take available standards of whatever was 6 out there and put them all together, examine them and say 7 what is applicable, what is too harsh and what is not strong 8 enough. These are some of the recommendations that we 9 expect to come out of our research. l 10 MR. LEWIS: Well, I guess what I am saying and I I 11 will just say it once more is, I picked one number of this 12 one template that you showed us, and I wonder how applicable l 13 it really is NPP's. You will look into that and we will ) 14 hear from you eventually. 15 MR. VAGINS: Yes, sir. 16 MR. LEWIS: Thank you. 1 j 17 MR. VAGINS: Again, a real quick summary. We are 18 on schedule,.in fact, a little ahead of where we said we 1 19 would be. We are functioning right down the line for 20 regulatory guides on EMI and RFI and general qualification 21 methods, including all stressors. As you see, the Reg Guide 22 is out for internal review right now. 23 We are, wherever we can, helping to evaluate 24 industry programs. We have done a lot of evaluation of I 25 existing codes and standards and reg guides on O ANN RILEY &' ASSOCIATES, LTD. Court Reporters 1612'K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 o n - - -,, -
i 1 193 1 1 qualification. We are in the process of developing l i' 2 technical bases to move into the regulatory guides. And, as -] 3 you see, starting in October we will be finalizing the reg r 4 guide. 5 This was stretched out to 1995, to recognize the 6 difficulty of getting paperwork out, not necessarily a drop -7 dead date out here. We are on schedule. We are 8 progressing, and we should have help for the industry in the 9 regulatory process in the relatively near future. i 10 [ Slides.] 11 MR. VAGINS: A final point. There were three ? 12 questions raised again, concerns and recommendations, i 13 concerns for the Subcommittee recommendations for the main. [ ( 14 Committee. We were asked to look at smoke, specifically. 15 We have started that. We have done project definition 16 scoping reviews. We want to determine digital I&C 17 components likely to be used in a' study, which means what j 18 components are most prone to. fire and damage and what are 19 the most critical to survive smoke damage. 20 We have tried to identify exposure scenarios. 21. What are realistic smoke scenarios. It's not as easy as it ] 22 sounds. We have reviewed databases. These are LER's and-23 industrial military sources of information on smoke damage l 24 to digital equipment. The Illinois affair, we had quite a 25 bit of smoke damage there. There's some data coming _out on 1 ANN RILEY & ASSOCIATES, LTD. f Court Reporters 1612 K Street,.N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
194 1 that. Again, we have to define the credible smoke threaF 2 scenarios. 3 Phase two would be -- here, we say motivate. We 4 are going to try to work with industry, EPRI particularly, 5 and other utilities where possible, to gain more experience 6 and see if-there is any experimental data out there. We 7 will carry out very limited confirmatory research if 8 necessary, and experimental research. i 9 [ Slides.] f i 10 MR. VAGINS: There was a question on the risk { 11 significance vulnerability of I&C's and prioritization. In I 12 other words, what should we be studying first and what l 13 should be put our most emphasis on. This is going on ( )- 14 simultaneous with the rest of our work. Brookhaven and SAIC t 15 are working together, having input to this, evaluate all 16 plausible environmental stressors including smoke.affecting l 17 digital I&C. In other words, look at any stressor on that 18 list, and anything else that we possibly had overlooked. I
- I 19 We identify specific risk issues; i.e.,
look at ll 20 PRA's, look at reliability data from actual' plants, et j 21' cetera. Then, use importance measures to rank and 22 prioritize stressors. This will give us some idea which are 23 the most important and which, again, our maximum attention 24 from our limited resources. -i 25. Finally, the military -- we have found that non-i j .( . ANN R.ILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 .i Washington, D.C. 20006 '(202) 293-3950 f -l
i 195 1 classified DOD experience with EMI/RFI is concerned with .'k 2 testing of hardened platforms, systems, helicopters, fix l I 3 wing aircraft, et cetara. We have also made a very specific. 4 -- obvious finding -- that military' equipment is tested when-I 5 it's standing still. Military equipment is tested and are 6 serviced when it's not carrying out its mission, as 7 differentiated from a nuclear power plant, which is 8 continually being tested during its mission which has 9 components being tested and serviced during its mission. 10 The possibility of triggering an EMI/RFI incident 11 in is.anc or aircraft because of a test procedure is nil. 12 The possibility of triggering an EMI incident in a nuclear 13 power plant during operation is very high. So, there is a ) 14 little bit of a difference there. 15 DOD experience -- that's why I say they have l 16 little relevance to NPP's. Either the military environment ] 17 nor the equipment matches the nuclear power plants and the [ t 18 industry needs. 'I 1 19 Information on DOD -- 20 MR. LEWIS: I will simply record a disagreement -l 21 with this sequence'of conclusions. I think it comes from a. j 1 22 very, very narrow reading of -- i 23 MR. VAGINS: Right. I was going to modify this. .i 24 Remember, this is non-classified experience, and'this is' 25 only what is available in the literature. We are going to i ANN RILEY & ASSOCIATES, LTD. [ Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950
i 196 (\\ 1 get to that more later. Available literature on DOD ~ 2 experience seems to be -- the parts thr#_'s available with 3 outdated digital I&C equipment, this doesn't mean that we 4 are not using the same kind of digital equipment in nuclear 5 power plants. 6 We are talking about eight bit or less processors. 7 The available information is 12 and 15 years old. This is 8 available in literature which makes sense. They are not 9 about to put a lot of their classified literature and new 10 stuff out in unclassified literature. 11 Again, in summation, the detailed testing 12 information of state of the art digital equipment is not 13 available in the public domain. We are starting to work ) 14 with DOD through Scientech, and we will be getting more 15 information on their classified information. That's it. 16 MR. LEWIS: Thank you. Are there questions? 17 [:No response.] 18 MR. LEWIS: Okay, we are ahead of schedule. I 19 think that what we should do is the following, and I am 20 prepared to be overruled. I think we should give ourselves 21 a 15 minute break just to relax. Then, if we can, convene 22 the Committee. We don't need the visitors although they are 1 23 welcome, just to decide where we go from here. 24 We clearly have to report to the Full Committee, 25 presumably at the next meeting. I will not be at the next O I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950 a
-197 .-( 1 meeting, so one of you gentlemen will have to do that jcb. 2 Tom is going to do it, very good. That's right, we agreed 3 on that before. 1 4 Do you want to just keep going, or would you like -l S to have a 15 minute break before we talk about where we go j 6 from here? 7 MR. KERR: I, personally, have a 3:00 o' clock cab. 8 MR. LEWIS: Should we just go on for a few f 9 minutes? 10 MR. KERR: Yes. 11 MR. LEWIS: Do we need a recorder for this? I [ 12 don't think we do. We can cut the record. 13 [Whereupon, at 2:45 p.m., the transcribed portion 14 of the meeting concluded.] i-15 16 17 18 19 20 21-22 23 24 25 'I-ANN RILEY & ASSOCIATES, LTD. Court Reporters 1612 K Street, N.W., Suite 300 Washington, D.C. 20006 (202) 293-3950}}