ML20206D579
| ML20206D579 | |
| Person / Time | |
|---|---|
| Issue date: | 04/21/1999 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | |
| References | |
| ACRS-T-3072, NUDOCS 9905040156 | |
| Download: ML20206D579 (226) | |
Text
.::...: ;< ::..;. . .
~jy' - - .
.e - ' . ,
.. ? . ;. e, -;.y q ; n w .q YY;..q,_q.x, ,e : ..+ - .
- y.. . ;- :: ;, ;.: .. ::
- 3
- s.;.g::y.;p.t.3.m.7,&4g w.
ma . g. ; : yy:w., my >: m ;
,p ,
- G.g
. s
,w
- , =_
g'. . . .
. : . . c .:N::u';.;;g.g,.. . f._ g . .-.g:, y_
.n y Q. .:w;6mQ;.y. m:;,,;j;3.g :.$9 gf.k.] .,
n ,, . .-
4 . . f. ::
ym.
. l :. .. ,z.p.ll. ..:::: : . -l.,'%:.:Q.!D%
< f v . tq 4:r...:a,CWp\.t 9 .. ~
W-M}y_M 4 -'
~
s' '
N "
l _
%0 .' .
- ' ll:.l.?.Y.h llh.l_f.ff&.l%w::: t: W: ' l.N
~
'. .l. %lsf'l f: [d [. _f,
'~
l ~_ *P.' :lhlllylb:f:.y.
- f: n.n d % w.n u ..
r# "~, .
..: V )l W.Jy .h h .k W v-:..y :.:w. ; .w,%.g..:&'y 'V.g:_
. '_.WJ :,; ,
=
4: @ ,Q f Q.! 4
.&'<'i}QO.% '. .' .V:. 4: y 9 g e.'
. g . g .: .v; Vis '.
. . a. ..g. :s .m:dg.- a:M. :9.. . .s. iQ y.
, i. - '. I .- ... j ** -. - : '.
l ' f
.cf 1.hj XThh;f.iM.QMih.;. .;;gpM .,. ...)h.Q .'.[::;.y:n. w :..h...:r;'y/M.fi[h,
~
. k;.h'[h h ' k[ k. [.; O I ',' i.$[~} , 4.;
s [vtfs L V id #
,t
' / y..t_ . .G- .{.*y y "
g:.. .
.n 3
--L .. , . .
Q '.
. y. . . , .;; :, 4 , .e .; r~ s< -
.. <..gm %. ,;. p@-
% . v ~. y, y;:,.n:t -
- n. .-
u , m._?: ..
';n. .. - o .
gh, :.. :,)y. :.;:.:w.. [ , :j Y,[;$. 5:: v e,;v , *i
.- s,' %[r 4 # '
sc .
.e c.
e-
.: mi .e.. w.. c. , ,
j.?.], :,:yMQy.QQ:y... h,ig.,.( y :;" f.1.jf,.H
@i.(i:% .
;: _, -s
;..l ::9 c: ; h .. h . r:f, x.ap,Q:&y)f.';p
. .. t
. . . , V_.w: J.
h:
;i:r. i.M;: e :h :y .. :9 ,w :.
" J 4# . 6,n.a. u :.wh @ T 1,'. ,4 A. t ; ; .Q A;q k , ;j g "' - . . .- * .
. ..i.n";;W.lc 1.; v.1;.;: ::.R.v:< i v~ ~
';.;:. ' : ;. .~? ?.; Q%n.v.d;j,?: c. ;'9,Qyme p .W Q:h,?::.f&g %
~ . :- : - -
i
- ,. <;?.q
- .y .'.: :~.::: : ':
.A ; 7.g}. ,g ??:;.jg$.
.-j:;;p:. ;:%y . .g.Qg;T.(;V&,Ogh.9:: , r.
3.- n: 0;.,h
. ?"
1 h. ,>...y :.n A, . ,:.. .4
' ~
.1;~..b.. i. y a .: :.p .-:!;.5.g;g.8.c.
. , . u.: . ' .; . u i%. 4 pq .h 2.: :.:q:p ':n,: :djW 9 .
e
;4 .. -
;.;ys.v:.UN;h,430.J@[.%y:
.M.N 'T5M.- :f:;;;?
3 ' M. $ h(,hzhh%.Y$ili Ob: MC ? ' ) i s . .
; . : .- J
.,:y ;p.>,,
q .,. t : : &;, .. w:y:t .%.~.g./J.o.;y,d . c.
-?
;. f.
c u
. . x.. . - ..:. .s p%my;.. .;. l.r. gy w .
v ; .s. m p. 5 i ;p :y .
'4' . 9. yh,. :. ,. ' ' ' .
..q, l~ .?l.h..'.)p%.k$$Nh&
.m .. ,. ;v.m ,::. . ':
,,x - .
hl$ r
" ' '~
p.,. . .y- , .. , s .../. . :: .y 7.,',Mf
- u.f WQ.g
.. o :.
sa., ,
.f. . .-. ; , .. .
1.]q:
.*, 1. g A'?,
' ~
- [-:-
^ -
h' ' l m: .':.4.:.... ' :.%,
. :v..? .l::'. : if. c. 4ll':r:Y;.m%.
u.: ... - L ':-
? Q? g- ~.,y,,,,.., ..
o; ~ ,. . a,'....Q. 1 .l..::]!'.'$ :.t. ;, :Q .)
- k .* e et '_ ' '* ,
,.y:'..f(.:'.' , jI.yN: F;:.);J.l:[;i [ll.; ,: .. h. }n. Q:.a... .;,. . fM. h' . /.if
. ,e ., .o -
- o. .
P, .
. , .. :.- . . .+;;, y, g - .:4.. ,, . , . . ,..c. 7, ;ng. .ws g ..
r v
~ : r. , Q:j '
, ; y:-." ..k: j.;.qi:7 :::. . ..:,.K..Op;/.y,;.%:;,c . z. . *
*f **i.}:.;f:',
.s, :l. 'f'k .? qc.r'iQ:. ,j:gQ.g:l 32.g.f.:\'y ', ~ '* :
.3~ ,. y.y .?.
,.k ' :,::$,A *
>& l'l,": '_ i
'f.,,.
. q[.N . ._l:: u.l*
,' ) ;.,.:Mg.h;,f
- %. )f h,$ 3' *':.... .' '. ' ! '
-L' , ' .. ~
N,T :, g:;h, 1,[^ i t:.'[. ,.' , . ' #. .[ . ; ... . ,
.:;;-}Lf".[h
/
,. .. . '7 ~ $ ,,* ..
i:,... l ,'.- 5: . . .'.'h' . : ' .hi ;'dh :n ' '}j .Y. . 7 t': ' vl # .,
.~;_
e l: a:..'f.. . h.Y'1l.h.::. 5;, k:. h;,q. l c. '_,,,,., J m g m_;~;.: ., .
^
J . i[:ff:d# - %. . .>d.p .Mfy 4 k .e,, I ' k.f.'h;d., yi N;.
,}
w p. .E
-.f '
lf
- a :p . . .v w . ms - ,
q@l M - p: -
- p;.a.%,g&~ C ...my, ;t.,g.z.3p.<.
t -
~
... i-' .;
6ji - t y - g t
- .M..';,.g 7.x:.i $.p .$:. b ' . , ,.'- _
. ; Y S . Y : . .
^
.~
( 4 @l ,-+ d W.bg . y .
- f. ..mi.;G
- b ' . b i~l-
~
. ,i . Y *. i g. " .' ',..:
. ' 2:,D,K./v.N %,f2.
.: i
. .- b,hh, 'h [. ' .
. . .6 ...
'W ' ., gm .
;a - L fll,}ql:::(p
- g.4':*..l:j,c'/y;&il.', ej ,y ,
k;:
' ', , . j$. ';,t 1 . . . . N. ' -
- i.Q ;jf.;'l..;'j;_[.:,)kf,,f Q !.*,f:' ,: %,s. %.( , _ .;; .
M. q 4 ; ;:4. > : :.. y.., .
. ry..
- ,;. 7p.7::.p:m:;6y4.
.7. y e.r .,: p3.y3,p.y$.,c.,6c
. ., . .,,,,::4.p. g
- .. . V. ,: .. f , ; .1. . . <(. ':. u.
.; -. 3,d.: p: s . .. . :s
_4..up.g ':t.,p,, s:.44,3. 5 $ ,; < ,-
*e...:. .
.v:a m:r :.,n: in,: J.. Q.. .y ,.r.
.n f- .r.s '.. ~;.w- .M d, . -
7 %.
?
.. +4*!: .;
- /- ; ". m. ..
,'L- . ;. . . . i . k.;.
v.
- 7 T.
ph.,,.c.
*pr' ' .'- ... -.. .. . 5n. :&p 1.a;s g n~.r,
.' - :. ': .l' ~
e
- n. .. .
r.,s .- . n 7iz,.,:, . . <m.....u..f),p 9 gjg w s,.,.?
, 4..w:. .:
,4 ,
- ., . ,e:a.7 . v .- w. , .rn.. p.
. vc
,r ...':- . + .
-. . .. - . .. ~
pp.y .g p fL,t T . l - :; - n:.-.%... .. .b. .;,...m W,; : ti
* ; 4,; p..%.,.g ' ;; y
,p.
ma ..g: f;;TI Y ~ ?L. OJ . 5.
. . e,; ..t 4 .x. .s ., , ?. ..
~
^ !
MR.QQ: a : w .m m w% w;JR emA Q. . L, fL
.,.,+ .
..:y : ..n.t:y.::y.8 s.::y&ng . , .. x. . ..,;:,n; 4
. v .:
- . . .n . : . ; - . .o .x, . . :.
.,x
.c, s. . .yn.v ; 4,n n. 3,> . .
. m. t...cy..e
. y .7 :[ . . ., . .. . . . . .
],; ,, .
F~ ' N i AcRsr-sot 2. v . g OFFICIAL TRANSCRIPT OF PROCEEDINGS
' %f-UNITED. STATES OF AMERICA NUCLEAR REGULATORY COMMISSION
/
Title:
. RELIABILITY AND PROBABLISTIC RISK ASSESSMENT AND L.
.r;r REGULATORY POLICIES AND PRACTICES
.XL4 'ALKb)
I
' ?. i
.N RET"JRN ORIGINAL TO BJWHITE M/S T-2E26
'-(' 'j-.. 3 415-7130 T!WJKF ' Case'No.: - I i { l l
' Work Order No.:
ASB-300-752 i [ ( bFY 0"21 PDR LOCATION: Rockville, MD DATE: Wednesday, April 21,1999 PAGES: 1 - 180 x$ U U.. ,D2 ANN RILEY & ASSOCIATES, LTD. 1025 Connecticut Avenue,NW, Suite 1014
/9 k/ ' .
Washington, D.C. 20036 g;+. (202) 842-0034 fs a "n!Ce Cop %y- Retain
- JCthe Life c;f.the Ocgittet. i
t V DISCLAIMER UNITED STATES NUCLEAR REGULATORY COMMISSION'S ADVISORY COMMITTEE ON REACTOR SAFEGUARDS APRIL 21, 1999 The contents of this transcript of the proceeding of the United States Nuclear Regulatory Commission Advisory (( ) Committee on Rear, tor Safeguards, taken on April 21, 1999, as reported herein, is a record of the discussions recorded at the meeting held on the above date. This transcript had not been reviewed, corrected and edited and it may contain inaccuracies. A ( 1
'w.)
1 1 UNITED STATES OF AMERICA (h 2 NUCLEAR REGULATORY COMMISSION 'w) 3 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 4 *** 5 RELIABILITY AND PROBABILISTIC RISK ASSESSMENT 6 AND REGULATORY POLICIES AND PRACTICES 7 *** 8 9 U.S. Nuclear Regulatory Commission l I 10 11545 Rockville Pike 11 Room T-2B3 4 12 Rockville, Maryland 13 Wednesday, April 21, 199's 14 The subcommittees met, pursuant to notice, at () 15 8:30 a.m. 16 MEMBERS PRESENT: 17 GEORGE E. APOSTOLAKIS, Chairman, Subcommittee on 18 Reliability and PRA 19 THOMAS S. KRESS, Chairman, Subcommittee on 20 Reliability and Probabilistic Risk Assessment 21 JOHN J. BARTON, Member, ACRS 22 MARIO H. FONTANA, Member, ACRS 23 DON W. MILLER, Member, ACRS 24 ROBERT L. SEALE, Member, ACRS 25 GRAHAM B. WALLIS, Member, ACRS l () (,,/ ANN RILEY & ASSOCIATES, LTD. Court Reporters l 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
2 l 1 PARTICIPANTS: ! 2 E. ROSSI, RES
}
3 J. ROSENTHAL, RES I 4 B. YOUNGBLOOD, Scientech 5 P. KADAMBI, RES 6 M. MARKLEY, Cognizant ACRS Staff Engineer I 9 10 11 1 12 I 13 14 16
- 17. J 18 19 20 21 j 22 23 24 I i
25 i d (x 3 ANN RILEY & ASSOCIATES, LTD. Court Reporters (,,/ 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 3 1 PROCEEDINGS i ) 2 ! f)
\_ [8:30 a.m.)
3 DR. KRESS: The meeting will please come to order. 4 This is a meeting of the ACRS Subcommittees on Reliability ! 5 and Probabilistic Risk Assessment and on Regulatory Policies l 6 and Practices. It's a joint subcommittee meeting. I am Tom l ) 7 Kress, Chairman of the Regulatory Policies and Practices I 8 Subcommittee. Dr. Apostolakis is Chairman of the l 9 Subcommittee on Reliability and PRA. 10 ACRS Members in attendance are: John Barton, 11 Mario Bonaca, Don Miller, Mario Fontana, Graham Wallis and 12 Bob Seale, which pretty much is most of the committee except 13 for Dana and Bill Shack. 14 The purpose of this meeting is to review the A)
! 15 Staff's reconciliation of public comments on 16 performance-based initiatives, SECY-98-132; and the plan for 17 pursuing performance-based initiatives, candidate 18 activities, and related matters; and NUREG/CR-5392, entitled 19 " Elements of an Approach to Performance-Based Regulatory 20 Oversight."
21 The subcommittees will gather information, analyze 22 relevant issues and facts, and formulate proposed positions 23 and actions, as appropriate, for deliberation by the full 24 Committee. That is what we always do. I don't know why we 25 always say that. I guess it's for the public record. l ! ['N l (,,/ ANN RILEY & ASSOCIATES, LTD. Court Reporters l i 1025 Connecticut Avenue, NW, Suite 1014 22 84 b34 l l l
E 1 4
' l- DR. SEALE: That is what we always aspire to.
2 I DR. KRESS: Yes. Michael'T. Markley is the
- 3. Cognizant ACRS Staff Engineer for this meeting.
4 Rules.for participation in today's meeting have 5 been announced as part of;the notice of this meeting, 6 previously published in the Federal Register on' April 6, 7: 1999. 8 A transcript of the meeting is being kept and will 9 be made available as stated in the Federal Register notice, 10 so it is requested that speakers first identify themselves 11 and then speak with sufficient clarity and volume so they 12 can be readily heard. That means come to a microphone, 13 naturally. 14 We have received no written comments or requests () 15 for time to make oral statements from members of the public. 16 The only comments I have, in addition to what-I have said 17 before, I think the Staff is planning on going to the 18' Commission with a Commission paper on this issue, and they 19 want our -- I guess they want our opinion on the plans laid 20 forth in this Commission paper and so we will have to bring 21 this before the full Commission I guess in the May meeting 22 and write a letter, so a letter is expected. 23 MR. MARKLEY: It will be June. 24 DR. KRESS: June? 25 MR. MARKLEY: Yes. ANN RILEY & ASSOCIATES, LTD.
\ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
L , I 5 1 DR. APOSTOLAKIS: But I thought we were -- I~ 2 MR. MARKLEY: The Commission paper is not going to ; 3L be available prior to the May meeting. DR. APOSTOLAKIS:
'4. No, but I --
5 DR. KRESS: Go ahead. 6 DR..APOSTOLAKIS: I thought they had a deadline of 7 May 31st to send it up to the Commission. 8 MR. MARKLEY: That's correct. 9 DR. APOSTOLAKIS: So we will write a letter 10 afterwards.
)
11 MR. MARKLEY: In parallel with the Commission J 12 consideration of that paper. 13 DR. SEALE: Generally we don't write anything 14 until we have the appropriate document, so we know what it 15 is we are recommending. 16 MR. MARKLEY: Right. 17 DR. SEALE: Formally. 18 DR. KRESS: We do have a draft document. We have 19 something. 20 With that, I will call on my Co-Chairman to see if 21 he wants to make any comments before we start. 22 DR. APOSTOLAKIS: No, I think we should start. 23 DR. KRESS: Okay. We should start? So I'll turn 24 the meeting over to -- to whom? 25 MR. ROSSI: Let me start off. I am Ernie Rossi ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
6 1 and I am Director of the Division of Systems Analysis and 2 Regulatory Effectiveness in the Office of Research. (} As was 3 indicated, we are here today to talk about plans for 4 performance-based approaches to regulation. 5 First of all, let me thank you profusely for 6 arranging for this meeting on such a short time scale. We 7 received the Staff Requi.ements Memorandum on SECY-98-132. 8 It was issued on February lith and in that we were asked by 9 the Commission to provide a plan to the Commission by the 10 end of May, so we have a very short time in which to develop 11 a Commission paper with a plan on it. 12 We did have a public meeting last week. It was 13 not widely attended. I think we had the right people there, 14 but there are a lot of other things going on, as you may 15 (u )I know, within the Agency now that competes for time with the l 16 industry, but we did have that meeting and we wanted to have 17 it before we came here. l 18 With respect to our schedule, as I indicated, the l 19 Commission asked for the plan by the end of May so we are 20 going to depend largely on whether comes out of this meeting 21 today in developing the Commission paper and then, as Mike 22 indicated, we will meet again with you in June, but that i i 23 will be after the Commission paper has gone up. 24 Now the plan that we are developing is to be a 25 plan for the NRC as a whole, so that will include i I
/'% ANN RILEY & ASSOCIATES, LTD. I kms) Court Reporters !
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 i
7 1 involvement of all the major of fices and, as I think you 2 will find from our presentation, there are a number of f} v 3 activities in the offices that are underway now that we 4 believe to be the application of a performance-based 5 approach to the regulatory process. l 6 We are looking forward to the interchange of 7 information today and are very interested in your views on 1 8 this subject. I would like to point out that in your 9 package on the last page there is a set of questions that we 10 used in our meeting last week with the stakeholders and we 11 developed those questions as the things we thought we wanted 12 answers to, to help us in developing the Commission paper 13 this next month, so we are very interested in your views on l 14 anything you hear today but you might want to look at the I es () l 15 questions and we would very much appreciate it if you would 16 look at the questions in particular and any comments you 17 have, either on whether they are the right questions or if 18 you could provide answers and your view, that would be of 19 help. I 20 At this point 1 am going to turn things over to 21 Prasad Kadambi, who is going to give you a presentation on l 22 sort of the history of this issue and the work that we have 23 done to date, and this will then be followed by a 24 presentation by Robert Youngblood of Scientech. Prasad? 25 MR. KADAMBI: Thank you, Ernie. Good morning. My
,Q ANN RILEY & ASSOCIATES, LTD. \s ,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l' i 8 1 name is Prasad Kadambi. .I am with the Office of Research in b)
/" 2 the branch, it's called the Regulatory Effectiveness i 3 Assessment and Human Factors Branch --
4 DR. APOSTOLAKIS: Looks like "REHAB" -- 5 MR. KADAMBI: Well, maybe that is part of its 6 function. 7 DR. APOSTOLAKIS: That was good. 8 MR. KADAMBI: I'd like to lay out for you the 9 outline for today's presentation. I will be back to Ernie 10 for the management overview. We will go through some of the 11 historical background, talk about the Staff activities. 12 In the middle we will have a focused discussion on 13 the NUREG CR-5392, which has been announced as part of the 14 agenda and Bob Youngblood, who is the principal investigator () 15 on that, will speak on that subject, and after that I hope 16 we will get very much focused on the SECY paper and the 17 response that we have to develop on a very short time scale, 18 talk about the SRM and what we need to do to address that, 19 the stakeholder meeting input, and then the elements of the 20 plan that we have come up with up to now, subject to any 21 feedback we receive from you. 22 With that, Ernie would you like to just address 23 the points on the arrangement of -- 24 MR. ROSSI: Yes, I will say a few words. As you 25 know,_the Office of Research was reorganized, I guess it was ( ANN RILEY & ASSOCIATES, LTD.
\ ,e) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
9 1 the last week in March,.and the Division of Systems Analysis ( 2~ and Regulatory. Effectiveness has two' branches. .One of them l l 3 is the branch that Prasad is in. The other branch has the l 4 severe accident work, codes and experimental work, and the 5 thermal hydraulic work. 6 In the Regulatory Effectiveness Assessment and 7 Human Factors Branch, they have several teams. They have a 8 team for generic safety issues, one for human factors, one I 9 for: regulatory effectiveness, and one for operating 10 experience reviews,_so that is sort of.a quick overview of 11 the organization. 12 I did mention the fact that there were some . 13 ongoing activities in the other offices that are I j 14 performance-based approaches. Probably the largest-scale () 15 one and a very important one for the agency as a whole is 16 revising the reactor regulatory oversight process and 17 inspection program, so that will be going on as we do l 18 anything in addition to that, and what we intend to do is to 19 learn from it as much as we can but that program is well 20 underway and we don't anticipate or plan to do anything that 21 would have a major ef fect on it. 22 Obviously I'm sure that NRR will keep in touch 23 with everything else that is going on, so if something turns 24 up of value they will take it into account, but this is not ! 25 directed at that particular program, and then there are [ f)' ANN RILEY & ASSOCIATES, LTD. As' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
I 10 l 1 other things in the area of performance-based applications
'2 (f like Appendix J and the maintenance rule, and there are also '3 'some' activities underway in NMSS.
We don't have anybody 4 here to talk about them today, but NMSS did talk about their 5 activities in the meeting last week, so those will be 6 discussedLI guess in the Commission paper. Prasad? 7 .MR. KADAMBI: Thank you. 8 I guess I would like to begin the historical 9 background with the SRM that was issued for SECY-96-218. I 10 think this committee has heard about this SRM and the four 1 11 policy issues that were. addressed in it. i 12 The four policy issues were the role of I I 13 performance-based regulation and the PRA implementation 14 plan, plant-specific application of safety goals, () 15 risk-neutral versus increases in risk and risk-informed IST 16 and ISI. i 17 In the SRM the Commission asked us to consider 18 performance-based initiatives not explicitly derived from 19 PRA insights. They also asked how these would be phased 20 into the overall Regulatory Improvement and Oversight 21 Program. It was actually in order to address this in part 22 that the !)ECY-98-132 was prepared. 23 DP. KRESS: Did they spell out what those 24 performance-based initiatives were that don't come out of 25 PRA insights? Did they give you a list of them? O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025' Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036
-(202) 842-0034
I l t i 11 l 1 MR. KADAMBI- Well, the SECY-98-132 actually was, ' [)
%../
2 as I see it, an earlier version of the plans to do that. I l 3 think we are right now in part of that effort to develop a l l 4 process and a rationale to come up with the kind of list I l 5 l think you are asking about, j 6 DR. APOSTOLAKIS: The way I read the SRM is they ; i 7 are asking you to include in the PRA implementation plan 8 performance-based initiatives, and then they say the Staff 9 should include its plan to solicit input from the industry 10 on additional performance-based objectives which are not 11 amenable to probabilistic risk analysis, so this is slightly 12 different in my mind than what you have there. 13 In other words, they want to see a 14 performance-based approach to regulation that will be based j f"'g l ( j 15 on PRA when necessary or possible, but should not be limited 16 to the PRA insights. 17 Now what you are doing hern is limit it to non-PRA 18 applications or is it a total thing, and some of it is based 19 on PRA, some of it is not? - ^ '~ 20 MR. KADAMBI: I want to make sure that I am 21 cautious in answering your question because I think we may 22 be getting some SRMs mixed up over here. I don't know if I 23 am right, but I think the SRM that you read from is the SRM 24 298-132, whereas the SRM that was associated with 96-218, 25 which sort of led us into this work, used different () ANN RILEY & ASSOCIATES, LTD.
\_ / Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
12 1 language, f) V 2 DR. APOSTOLAKIS: I see. 3 MR. KADAMBI: And what it said is look into these 4 areas -- I am paraphrasing -- and either make it part of the 5 PRA implementation plan or develop a separate plan. 6 DR. APOSTOLAKIS: That's true, yes. 7 MR. KADAMBI: And, you know, that's when we got j 8 started on looking at what I will call in shorthand 9 "non-PRA" work and right now the plans that we are proposing 10 to develop in response to the most recent SRM will hopefully 11 answer some of the questions you just asked me. I am not 12 sure I know exactly what it will cover. 13 DR. APOSTOLAKIS: But given the earlier SRM, it 14 says the Commission has approved Alternative 1 with respect l O) ( 15 to the role of performance-based regulation, but 16 applications of performance-based approaches should not be 17 limited to risk-informed initiatives. 18 MR. KADAMBI: Right, and we are not planning to 39 limit it to risk-informed initiatives -- 20 DR. APOSTOLAKIS: Yes, but you are not going to 21 limit it to non-risk-informed initiates either. 22 MR. KADAMBI: No. 23 DR. APOSTOLAKIS: So today's subject is what? The 24 issue of performance-based regulation, period. 25 MR. KADAMBI: Correct. /~'\ ANN RILEY & ASSOCIATES, LTD. (_,) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
13 1 DR. APOSTOLAKIS: And in some places PRA may be
'[ 2 useful and in others it may not.
3- MR. KADAMBI: Correct. 4 DR. APOSTOLAKIS: We are not limiting it to !
- 5. non-PRA?
6- MR. KADAMBI: Correct. , l
.7 DR. APOSTOLAKIS: Okay, good.
8 MR. ROSSI: One thing though that in my opinion I 9 think the PRA part of this is probably much better developed 10 and on the road. We will consider.the whole thing as a 11 whole so we are kind of more concerned about the other part j 12 that is non-PRA based, so that is probably what we are 11 3 thinking about but we have to make sure that it is all 14 integrated -- O 15- DR. APOSTOLAKIS: ()_ Exactly. 16 MR. ROSSI: -- and so forth, and we.will do that 17 in one way or another. 18 DR. APOSTOLAKIS: Okay. 19 MR. ROSSI: And it also has to be totally 20 integrated across the agency with the other offices. 21~ DR. APOSTOLAKIS: Good. 22 MR. KADAMBI: The next step along the way was 23 .really.part of the strategic assessment and rebaselining 24 initiative, DSI-12. There was a paper issued and 25 - subsequently a COMSECY, which took a rather comprehensive O N ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p 14 l' approach including performance-based concepts into Staff l J 2 activities. 3 Now the-way these things I believe were integrated 4 into'the Staff's activities was.through the NRC Strategic 5 Plan, which said we.will implement risk-informed and, where 6 appropriate,. performance-based regulatory approaches for 7 power' reactors, so that as I see it gave the direction to
~8 the Staff saying.this is what you should be doing.
9 DR. APOSTOLAKIS: Has the Commission ever defined 10 performance? 11 MR. KADAMBI: Well, okay, if I can go on, maybe -- 12 I believe that we have the best definition up to this date 13 in the white paper, and that, if you don't mind, I'll put 14 answering that question off until we get to that. () 15 DR. APOSTOLAKIS: All right. l 16 MR. MARKLEY: Just.for the benefit of"the pubic 17 and the merc.bers here, that white paper is SECY-98-144. 1 18 Correct? ' 19 MR. KADAMBI: That's right. 20 DR. APOSTOLAKIS: This is the Commission paper 21 where they defined risk-informed -- yes. 22 MR. KADAMBI: In fact, the next two bullets speak 23 to what happened in the June, 1998 timeframe is when the 24 Staff issued SECY-98-132, " Plans to Increase Performance ) 25 Based Approaches in Regulatory Activities." The Staff also I ANN RILEY & ASSOCIATES, LTD. k'm,)/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 I (202) 842-0034 i l
l I 15 l issued 98-144, a white paper on risk-informed,
~
l' j 2' performance-based regulation, but in that SECY paper, the
~
-3 _w hite paper was,in a' draft form and was offered to the
)
4 Commission to deliberate and confirm the Commission's views 5 on'it. 6 'The SRM for SECY-98-144 was actually issued on 7 _ March 1, 1999, so we are. talking in a sense of things that 8 -happened bunched _together in time and then there was a long 9 period where not much happened, sort of a punctuated 10 equilibrium,.as I see it. 11 As Ernie. mentioned, in February, 1999, we got the 12 .SRM for SECY-98-132, and it directed the Staff to prepare 13 plans for performance-based initiative after obtaining
-14 stakeholder input, and that is part of the exercise that we 15 are currently involved in.
16 DR. APOSTOLAKIS: What is the message from this 17 history? What are these bullets telling us? I mean there , i 18 are a couple of SECY' documents, Commission SRMs, so then 19 that message is what? 20 MR. KADAMBI: You know, at least to me the message 21 is that we have got to use all the direction that has become 22 available from the Commission to put together a credible 23 plan that will help us meet the Strategic Plan objectives. 24 MR. ROSSI: I think the message from the history 25 -is primarily just background information, to tell people O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
16 1 what has happened in the past and bring everybody up to the () 2 same point as.to what has transpired, I don't know that 3 there is any particular message other than people ought to 4 -know how it started and-what has been written on.it, 5 'primarily.
-6 DR. KRESS: I guess the message I get is that the 7 Commission is very serious about performance-based 8 regulation and wants---
9 MR. ROSSI: There is also that message in there 10 also, yes. That is.why they asked for a plan by the end of 11 May. 12 DR. APOSTOLAKIS: Have.you seen an evolution in 13 the Commission's thinking about this issue over the last two 14 years or so, or -- because I read the SRMs and it seems to 15 me they keep saying the same thing. 16 MR. KADAMBI: I believe that with the issuance of 17 the white paper in final form, the evaluation that I 18 observed happening through the strategic assessment and 19 rebaselining process and the issuance of the DSI-12 papers, 20 I observed a certain evolution and I think it has come to 21 some kind of a fruition in the white paper. That is the way 22 I am viewing this process that we have been involved in and 23 trying to use the white paper essentially to the extent that 24 it will help us guide the work. 25- So focusing a little more on the SECY paper ANN RILEY & ASSOCIATES, LTD. OA Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 1
1 17
- l-
'itself, SECY-98-132, at.the time that the_SECY was prepared' 2 t we were,also working with the excellence plan that the Staff i i
3- had sent:up to the Commission and-in it there was one of the 1 4' . strategies that.was described was in the excellence plan -- I 5 a Strategy 5, which'soughtito look at regulations and 6' tregulatory. guidance and some other things to improve 7 effectiveness'and efficiency, which were described as the 8 components of excellence, so SECY-98-132 tried to take i 9 advantage of that effort and also address the Commission's i 10 direction on performance objectives which are not amenable ! 11 to probabilistic risk analysis. l 12 The SECY again did not focus only on reactors. It
- 13. took a comprehensive approach. The ACRS issued-a letter on j
.14 April 9th, 1998, and asked the question how and by whom
() 15 performance parameters are to be-determined and deemed 16 acceptable and how'to place acceptance limits on them. 17 DR. KRESS: Sound like good questions. I 18 MR. KADAMBI: Well, that is the reason why I have 19 got it up there, to let you know that we are still thinking
'20 about it. We may not have all the answers but we have heard 21 you.
22 The EDO responded on May 4th and the response 23 basically said the white paper is going to help us and there 24 is this effort on DSI-13, which is industry, interaction 25- with industry. O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r .. 18 1 DR. APOSTOLAKIS: Speaking of good questions 2' though, I look at the last viewgraph; questions for the 3 stakeholders, it doesn't.look like you asked them that 4 question. Did you? 5 MR. KADAMBI: We'did ask the stakeholders these 6 questions. 7: MR. ROSSI: No, he's talking about did we 8 specifically ask the stakeholders the question'of how and by 9' whom performance parameters are to be determined and deemed 10 acceptable, and how to place acceptance limits on them. No , f 11 we did not ask that as a specific question. +
]
12' What we I guess are looking for_from the 13_ stakeholders is specific views on regulations and other 14 regulatory activities beyond what is being covered in the () 15 risk-informed work that we might try to apply 16 performance-based techniques to, and so I think that 17 question is kind of an implied question in what we did ask. 18 I guess to date we haven't gotten very much 19 specific input from the industry on what things they would 20 like to see more performance-based. 21 I think, Prasad, in the agenda to the meeting, you 22 had some ideas of areas. 23 MR. KADAMBI: Yes. What I wanted to point out is 24 that I-think there is an expectation that we would_ meet with 25 you before we went out to the stakeholders, but because of
. ANN RILEY & ASSOCIATES, LTD.
s Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
19
- 1. the crush of events, because of the way things have-() 2 3
developed in sort of' spurts we have not been able to do that. We are coming to you after we-met with the
~
4 stakeholders. 5 In SECY-98-132 the Staff presented a suggestion 6 that a separate policy statement on performance-based 7 approaches may be beneficial but the Commission did not 8 address this question. 9 DR. KRESS: Did you want us to address that in'our 10 letter, that particular question? 11 MR. KADAMBI: Hell, I think as it will come out 12 later, our approach right now is to use the white paper as
.13 the policy statement, in effect.
14 DR. KRESS: I see. 15 MR. KADAMBI: We have always recognized that 16- resource requirements presents a really difficult challenge, 17 and it still represents a major consideration in our work in 18 this and at the same time we also recognize that there are 19 many research needs, many unanswered questions that we have 20 to tackle, so that came out in that SECY paper also. 21 DR. WALLIS: What do you mean by research? How do 22 you do research on a subject like this? 23 MR. KADAMBI: I guess the way I would look at it 24 is it's the kind of research that will help us provide a 25 technical basis that can be used by the program offices if ANN RILEY & ASSOCIATES, LTD.
\ Court Reporters i
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
7 20 1 they identify candidates that they would like to convert to n f ') 2 a performance-based approach.
\J 3 DR. WALLIS: Well, research to me, sir, means 4 having hypotheses and testing them and seeing what works, 5 and research in this kind of field seems to mean dreaming up 6 something and then arguing about it with other people.
7 MR. ROSSI: I think this area that I think you 8 described your view of what would be done. I would say it a 9 little bit differently. I would say that what we need to do 10 is we need to give careful thought and discussion on where 11 we can apply performance-based approaches, and the ACRS 12 letter up there had one area of how to place acceptance 13 limits on the things that we define as the parameters that 14 ought to be used to judge performance. (3 y ) 15 I think that is an area where it takes some 16 analysis and thought as to how you would go about developing 17 .the acceptance limits, so there would not be any experiments 18 or anything of that sort, but there would be careful thought 19 and discussion. 20 I think you described it in a slightly different 21 way but I would say careful thought and discussion and try 22 to develop a consensus on what parameters ought to be used 23 and how to place acceptance limits on them, and what 24 particular regulatory activities could this be applied to. 25 DR. WALLIS: So what you are describing is not a f') NJ
. ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
O l i 21
- 1. scientific _ process, it is a political one. It is not let's' 2 findLout by testing and by looking at what has been done l(
3' before what actually works on some basis which could be 4 called scientific. It is actually putting people together ~
- 5. .who have some stake in arguing or wrangling or discussing, i 6 whatever you want to say, until people get tired and agree 7 to something. That.is a political process.
8' MR.'ROSSI: Well, we would try to determine
- 9. parameters that could be used for judging maybe something 10 like quality assurance, and then how would you tie that 11 analytically to safety in some way. That is the kind of 12 thing that we would look for.
13 DR. MILLER: What is the process of determining 14- those parameters? Is there an organized process or we just ( ) 15 sit around the table determining what they are? 16 MR. ROSSI: Well, I think at this point in time we ! 17 are at the point where we are trying to determine what 18 activities might be amenable to the process in the first 19 place, and then once we do that we would have to -- I mean 20 we are trying to develop a plan for doing many of the things 21 that you are asking us questions about. 22 DR. BONACA: The NUREG CR-5392 seems to suggest in ' 23 the beginning an approach to do that. Is it going to be 24 presented? 25 MR. ROSSI: Yes. [~" ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
22 1 MR. KADAMBI: That's part of the presentation this e s. l ( 2 morning. i 3 DR. MILLER: This NUREG is part of your research, 4 right? 5 DR. KRESS: Correct. l l 6 MR. ROSSI: Yes. That would be an example of l 7 research, right. Yes, that is absolutely right. 8 DR. KRESS: That's what passes for research here, 9 Graham. Under your definition, would you have categorized l 10 Einstein's special. theory of relativity as research or not? l 11 DR. WALLIS: It's a hypothesis which is testable. I 12 The diamond tree is an interesting idea but I don't see it 13 as a testable hypothesis. 14 DR. MILLER: Why not? Why isn't it testable? {)\ 15 DR. WALLIS: It is testable if it is used in 16 various circumstances and shown to be somehow better than 17 something else. Then I guess you have some measure of 18 whether it is good or not. That is what I am looking for. 19 DR. FONTANA: Excuse me. I think that the test 20 would be, once you have worked this out, is to try it out on 21 a pilot plant with a test that maybe lasts 10, 15 years and 22 keep doing the other ones the old way, and then if a problem 23 keeps cropping up, you may be able to see it. 24 DR. BONACA: Actually I thought that NUREG had a 25 lot of good thoughts in it and I think the important thing f~')
\/
ANN RILEY & ASSOCIATES, LTD. Court Reporters ! 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 l
e 23 1 is to almost roll up your sleeves and just go to an exercise 2 ( }- and put in some of those boxes some of the activities that 3 you have done in the past and are thinking of not doing any 4 more, and then see how the thing works. 5 DR. KRESS: To pull off a pun, there were a few 6 gems in there, weren't there? 7 DR. BONACA: Well -- I don't know -- 8 DR. MILLER: Mr. Chairman, you can move ahead on 9 that one. 10 MR. KADAMBI: The last point is that we did not 11 incorporate this work into the PRA implementation plan, so 12 ask of now we are really dealing with something that is 13 different f rom the PRA I.aplementation Plan. 14 DR. MILLER: So that never will be put in the PRA A) l 15 Implementation Plan, or is it just for the -- 16 MR. KADAMBI: Never is a long time. I don't 17 really know. 18 DR. MILLER: There is no plan to put it in there 19 in the near future? 20 MR. KADAMBI: That is not part of what we are 21 proposing. 22 DR. SEALE: Is there an intent someplace else in 23 the bowels of Research to take another look at the PRA 24 Implementation Plan? After all, it's about three years old 25 or so and most of the goodies have been shaken pretty hard. [\ ANN RILEY & ASSOCIATES, LTD.
\_ / Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
7 24~ 1- In fact, some of the things are counter to the original
'2' statements in the-plan, and I was wondering if you were 3 going lto take.a look at.it.
4 MR. ROSSI: -Well, it does get periodically 5 reviewed. I believe it gets reviewed every quarter and 6- revised and sent up to the Commission, right? -- so you I 7 would have that. 8 Again, some of the questions you are asking I { 9 think are things we need to think about when we develop our
]
10 . plan. We simply don't know the answer to them. I mean 11 depending on how we go, we certainly may want to add things 12 .into the PRA Implementation Plan because the one thing we do 13 :want.to do is make sure that this thing is all looked at in 14 a coherent way.
'15 We don't want'two things going off in 16- semi-different directions that overlap so these are all ,
. i
.17 things we have to consider when we develop our plans.
18 DR. SEALE: George, we may want to ask these 19 people to bring us up to date on the latest status, the 20 details of the PRA Implementation Plan at some point. 21 MR. MARKLEY: I think Dr. Seale's question was a 22' little bit different. If you look.at the PRA Implementation 23 Plan today it is full of more completed items than it is 24 future items,.and that is part of it. They don't see the 25 roadway that it is going down, next step if you want to call ANN RILEY & ASSOCIATES, LTD. ( Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036
-(202) 842-0034
25 1 it. [^ j 2 MR. ROSSI: I don't think we have the people that U 3 are-able to talk in a lot of detail about the PRA -- 4 DR. SEALE: I appreciate that. 5 MR. ROSSI: -- but let me say one thing. There is 6 the effort on risk-informing Part 50 that we went to the 7 Commission and described how that might be done, and so 8 there are the beginnings of things to do, more major things 9 in the risk-informed area. 10 I am sure plans will be develop 1 for anythir.g 11 that is undertaken, and how it is factored in to the PRA l 12 Implementation Plan I can't really tell you. 13 DR. SEALE: Okay. 14 DR. MILLER: You are going to go through this one {) ( j 15 now, Prasad. There's been a statement these two are saying 1 I 16 the same thing, 98-144 and 132. Can you point out places ; l 17 where they may not be saying the same thing? ' 18 MR. KADAMBI: Actually, I did not mean to imply 19 that they are saying the same thing as such, but they are 20 generally -- you know, they are directed towards addressing 21 the same sorts of activities, I believe, and as I see it 22 they are taking a step at a time in a direction that today 23 we can do a better job of articulating some of the plans 24 that will get us where we want to go than we were able to at 25 that time. /\_')/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
26 1 .The white paper, as I mentioned, that went up to 2 the Commission had a draft of this statement on 3 risk-informed performance-based regulation. I can't tell 4 you exactly how it was changed by the Commission's 5 deliberations, but it was changed, I believe, so I can't 6 really address the question to what extent is it the same or 7 different from 132, which was issued in June of 1998. 8 DR. MILLER: And I assume that you will elaborate 9 on Bullet 5. I find it intriguing. I'll let you go ahead. 10 MR. KADAMBI: Okay, but the white paper does 11 articulate a number of principles and I believe these have 12 been very useful in our thinking -- 13 DR, MILLER: The four principles that are 14 articulated --
) 15 MR. KADAMBI: I'll get to that also.
16 DR. MILLER: Is it in the overheads -- 1 17 MR. KADAMBI: Actually, the very next overhead I 18 talk about the characteristics of -- 19 DR. MILLER: Those are the principles? Did you 1 20 say a' number? 21- MR KADANBI: Well, that is definitely as far as 22 performance-based is concerned those are the principles, but 23 I think in the white paper it addressed other things, you 24 know, the current regulatory framework 25 DR. MILLER: Those are not principles. O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r 27 1 MR. KADAMBI: Those are attributes -- you know,
'\
the Commission said these are four attributes. As I see it, [Y 2 3 they would become necessary but not necessarily sufficient 4 conditions for a performance-based approach. 5 I mean that is sort of getting ahead, but -- 6 DR. WALLIS: Could I ask you -- I'm sorry. Is 7 this something invented by NRC or is it the history of 8 society using performance-based approaches to regulate 9 -something else, so that we know what works and what doesn'* ' i I 10 work? l j 11 MR. KADAMBI: I am not sure it was invented by NRC 12 but as part of the work on NUREG CR-5392, there was a 13 litarature search done, and I think Bob may address some of 14 the other places that he has found where the concepts of O( , 15 performance and results oriented principles may inform the 16 activity. 17 DR. MILLER: I guess maybe it is performance-based 18 regulation used in other regulatory processes. 19 MR. KADAMBI: Yes. In fact, one of the statements 20 that was made at the stakeholders' meeting was it would be 21 wrong to think that somehow the performance-based approach 22 started with the SECY-96-218 or whatever, you know, that I 23 alluded to earlier, that there are many other things that 24 have been performance-based and in fact SECY-98-132 talks 25 about how we would consider ALARA as a very [} ss ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
m l 28 1 performance-based concept that has been around awhile and () 2 3 that has worked quite well. DR. WALLIS: There's isn't a case study, though, l l 4 like aircraft maintenance or something, where someone can
- 5 show that introducing performance-based methods improved --
i 6 MR. ROSSI: I believe the maintenance rule was 7 intended to be -- that the NRC? 8 DR. WALLIS: That's again NRC. It's always been 9 NRC. 10 MR. ROSSI: Okay. 11 DR. WALLIS: Is there anything from outside, as a 12 reality check? 13 MR. KADAMBI: Again, the only one that I am aware 14 of and I don't necessarily know everything that is going on () 15 is the work that we sponsored from the Office of Research in 16 5392. 17 DR. MILLER: So I think Dr. Wallis has a good 18 question. In other arenas, say aircraft and so forth, we I 19 don't have any evidence of performance-based? 20 MR. ROSSI: Bob, you have done work in this area 21 on a literature search. Maybe you are the right one to try 22 to address this question. 23 MR. YOUNGBLOOD: Yes. It wasn't going to be part 24 of my slides, but the EPA has regulatory processes that are 25 performance-based in the sense that for monitoring of ANN RILEY & ASSOCIATES, LTD. O)
\s- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
29 1 emissions you can sort of afford to go by how well people
T are doing in controlling them if they are actually being (d 2 3 measured, so you can afford in that case to have kind of a
{ i 4 feedback loop and take a performance-based approach to 5 monitoring that kind of thing. I 6 That example is in the back of the report. j 7 And one or two similar examples are in the back of 8 the report. 9 DR. MILLER: Those are in the appendices? l l 10 MR. YOUNGBLOOD: Yes, I I 11 DR. MILLER: I'm sorry, I didn't get the ! 12 appendices. 13 MR. YOUNGBLOOD: It is not a focus of what I am l l 14 talking about here because, although the report wasn't about l () 15 reactors and isn't, and I think the thoughts apply more 16 broadly than reactors, I think that for reactor oversight, 17 there are things about reactors that make other industries 18 hard to compare. 19 DR. MILLER: Isn't Part 20 somewhat 20 performance-based? 21 MR. KADAMBI: Yes. And ALARA is part of that. 22 DR. MILLER: Well, ALARA comes out of Part 20. 23 MR. KADAMBI: Right. 24 DR. SEALE: Well, the new leak rate, Appendix J, 25 is highly performance oriented. (T
\_,/
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
i 30 , l 1 MR. ROSSI: I think the question, though, that Dr. l 2 Wallis had, had to do with its use in other arenas in [~')' O 3 nuclear regulation. l 4 DR. KRESS: Chemical. 5 MR. KADAMBI: It may well be, and I would suspect l 6 that with the GPRA, Government Performance and Results Act, 7 there is a push for applying it in many areas that perhaps i 8 the past had not considered it. l l 9 If I can go on with this, I guess the bottom line i 10 on this slide is that I considered this white paper as being 11 very important to our work. I believe that it provides what 12 I call a touchstone for implementing the Commission's 13 direction. We plan to use it as an equivalent to a policy 14 statement. () 15 of course, we have been aware of the broad 16 outlines of the Commission's thinking on DSI-12 and that is 17 what, in a sense, informed the work on 5392 also, and, you 18 know, there are some clear indications that the Commission 19 wants us to view this kind of work as being comprehensive 20 and not directed only towards one regulatory arena. But I 21 believe what they are also saying is right now we are in a 22 better position to use risk-informed regulation, regulatory 23 concepts and we need to position ourselves better in order 24 to be ready for performance-based concepts. That is the 25 message I get out of reading the white paper.
/'N ANN RILEY & ASSOCIATES, LTD. \_,) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
31 1 And RES, the Office'of Research, I believe, you 2 know, we want to contribute to' position the agency in a 3 better state of readiness to employ these concepts to the 4 extent that would be useful in actual rulemaking or 5- developing regulatory guidance. 6 The revision to the reactor regulatory-oversight 7 process that Ernie referred to, I think is a prime example 8 of something that is going on right now from'which we can 9 learn the lessons and we in a better position to offer the 10 kinds of guidance that people may find useful. j 11 DR. MILLER: Back to bullet 5 there, Prasad, that 12 implies that we are ready to do risk-informed and we are not 13 ready to do performance-based, is that what that means? 14 MR. KADAMBI: Right now I think, as a formalized 15 approach, you know, although people say that 16 performance-based approaches have been used, I am not sure 17 that we can pull together the sort of a formalized process 18 that we can offer to the staff as guidance, and that is one 19 of the elements in c:ur plan. 20 DR. MILLER. In other words, we need something 21 equivalent to a 1.174 for performance-based, is that? 22 MR. KADAMBI: Well, I am not sure what we will 23 find will be the best way to do it, but, you know, 24 .certainly, I think 1.174 represented sort of a watershed 25 event in the developments on the risk-informed area. And I C ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
32 1 hope that we can learn lessons from that developmental 2 process also as we go forward with performance-based work. 3 DR. APOSTOLAKIS: Well, since you mentioned 1.174, 4 it seems to me that it would not take a great effort to 5 develop a similar approach to performance-based regulation, 6 because 1.174 really states principles and concerns and 7 expectations. It doesn't get into actually doing ; 8 risk-informed analysis of particularly regulatory issues. 9 And you already have some principles or attributes here, and 10 what you are trying to do, judging from the documents I have I l 11 read, is you are trying to go way beyond what 1.174 did and l 12 actually see whether you can define some performance 13 criteria somewhere. 14 But if you keep it at the level of 1.174 it seems 15 ( to me you already have most of what you will need to write j 16 something like that. The only principle that 1.174 uses 17 that you may not have, and you probably have to think about, 18 is the delta CDF, delta LERF kind of criterion, which may or 19 may not apply to your case. It may not be applicable at 20 all. 21' But other than that, in terms of principles, I 22 mean if you think about it, 1.174 says be careful, make sure 23 that you comply with the regulations. Make sure you don't 24 compromise defense-in-depth too much. And, you know -- 25 DR. KRESS: It is those last two that you get in /~N ANN RILEY & ASSOCIATES, LTD. ( ,). Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p. i 33
- 1. trouble with' going to performance-based regulation.
() '2; 3-Complying with the regulations, I mean that is the'whole idea, . we are trying to change it. I 1 4 DR. APOSTOLAKIS: Yes. ;
~5 DR. KRESS: And the other thing is the
.6 ' defense-in-depth principle. You liave got to do something 7 better'than what is in 1.174.
8 DR. APOSTOLAKIS: No, but if you keep it-at the 9 level of 1.174, I think we have the material. l 10 DR. KRESS: Well, you keep it a level of 11 determining principles and things.. 12 DR. APOSTOLAKIS: As a starting point. Yeah, and 13 what you should worry about. I mean a lot of the stuff in I 14 the NUREG that Scientech has prepared talks about it, you () 15 know,. that they took the NEI example and they raised some 16 questions and so on. So you can turn it around and write a 17 set of principles for performance-based regulation that 18 'would be I think at the same level as 1.174. And perhaps 19 you should think about doing something like this to have a 20 product on the way to more detailed work later.
- 21 DR. MILLER: And don't try to solve all the 22 problems at once. The high level principles, then go from 23 there.
24 DR. BONACA: I feel exactly the same. What I mean 25 is that I think you 4.a.1 a lot of information right now and ANN RILEY & ASSOCIATES, LTD. O. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
34 1 you have even some approaches that I believe are quite () 2 3 useful in a pragmatic way to begin to build something. mean without an example in front of us, you know, I am lost I 4 in -- such difficulties seem to be represented by what you 5 are presenting here. I believe that some progress can be 6 made pretty quickly. 7 For one, I believe that whenever there was no 8 concern that the failure to meet the performance indicator 9 would result in unacceptable consequences, past regulation 10 went to performance-based. For example, ALARA is a typical 11 example of that, where failure to meet certain goals are not 12 catastrophic, I mean simply you don't meet those goals. So 13 that is why the regulation was performance-based. 14 And so I am trying to say that, you know, I agree (O/ 15 that there are some many practical elements, okay, from past 16 regulation on what you want to do, to begin to build 17 something there, and the diamond, this gem, was a good 18 example of how you could go about that. And, you know, I 19 wonder if there is an effort going on right now to attempt 20 to do something of that type. 21 DR. APOSTOLAKIS: There isn't. 22 MR. KADAMBI: Well, what tempers my thoughts in 23 this area is, although I was watching it from the outside, 24 the Reg. Guide 1.174 was a huge commitment of resources. 25 DR. APOSTOLAKIS: Yes, but that was because it was Ci ANN RILEY & ASSOCIATES, LTD. k/ s Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p 35 l 1 done'for the first time. Now, we shouldn't -- I mean~now we
'2 are experienced. I mean things should flow relatively 3 easier. l 4 And let me give you an example again with 1.174 l 5 .that shows that when you actually get into a specific issue, l 6~ you do other. things that what is in 1.174. 1.174 talks in i
i ! 7 terms of the five principles that feed into the integrated l 8 decision-making, and one of the criteria, as I said earlier, l l 9- is. delta CDF and delta LERF. And then you go to another 1 10 Regulatory Guide that deals with graded quality assurance 4 i 11- and you see that delta CDF and delta LERF are not used. l In - I l 12 fact, another approach is proposed there that classifies t l 1 13 ' components according to the importance measures. l 14 So here is a good example of having a document of 15 principles and then when you go to an actual application, I l l 16 you realize that that is not sufficient and you do something i 17 else. You do more.
'18' DR. MILLER: Or you do less.
19- DR. APOSTOLAKIS: You do less in one area and you ! 20 do more in another area. So I don't see -- l 21 DR. MILLER: Identifying the principles. 22 DR. APOSTOLAKIS: I don't see what is different 23 with performance-based regulation. You can pool all this 24 . knowledge that you have with the attributes that you have, 25 four attributes and so on, put them in the form of
/i ANN RILEY & ASSOCIATES, LTD.
( s/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l l 36 1 principles, discuss the issues, just like 1.174 does, you (")) 2 know, incompleteness, you know, all that stuff that they
\_
3 i have there. I am sure you have other issues he re. And, ' 4 again, the NEI example that Scientech worked out is very 5 enlightening in that respect. And then that is a first 6 document that sets the principles for performance-based j 7 regulation. 8 And then you continue now looking at specific 9 cases and what specific issues arise. Because that way we 10 will we have progress. And I don't think it is going to 11 take as much effort as 1.174, which really was created 12 basically out of nothing. 13 DR. MILLER: So you could use the principle, the 14 attributes, and then we already have experience with the x i Q 15 maintenance rule and so forth, and test those attributes, go 16 along with Professor Wallis' approach and see if those 17 things we already have experience with are the attributes 18 that are sufficient. 19 MR. KADAMBI: Well, I think everything you say 20 makes a lot of sense and, certainly, the advice would be 21 very useful, but I have also got to make it fit into the SRM 22 that we have to respond to within the resource limitations 23 that we have. 24 MR. ROSSI: Well, I think all the ideas we are 25 hearing are things that we need to consider how to address ANN RILEY & ASSOCIATES, LTD. ("%) g,,f Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 I 37 1 in the SRM. I mean we might not want to commit in our plan ( 2 that we present to the Commission to doing what you are D) 3 suggesting, But, certainly, we.will want to think about 4 whether that is-the right thing to do.and make a conscious 7
.5 _ decision at some point in time on how you write it down and 6 how you have Reg. Guides and so forth.
7 And it might be appropriate at this time.if you 8' 'put up the next slide, because the next slide seems to be 9 .the.one that really talks about performance-based regulation 10 in some substance. 11- DR. MILLER: Well, the attributes for me, it is on 12 the next slide, those captured a lot of thinking, I thought. 13 MR. ROSSI: Yeah, those are the ones that we need 14 to focus on. And the think that we need to do is to -- is () 15 this the one? Yeah, this is the one. What we would really 16 like, or what I would like -- I shouldn't say we, because 17 others may not agree, is to have some good area that we 18 could apply these principles to that involved PRA to a 19 lesser degree than some of the other things that are going 20 on in the agency and work it through as a pilot somehow, and
- 21. apply all of these principles to that area.
22 And I think in the meetings we have had with 23 stakeholders, we have tried to seek sugges~tions of where we 24 might do this, and I am not sure we have gotten any 25 substantive suggestions as yet. One of the reasons we may O ) 1025 Connecticut Avenue, NW, Suite 1014 ANN RILEY & ASSOCIATES, LTD. Court Reporters Washington, D.C. 20036 (202) 842-0034 l
38 1 have not gotten any furtherLsuggestions is that the ('21 2 stakeholders, particularly the industry, may feel a little V 3 reluctant to get involved in another major area at this 4 point in time. That could be the thing. 5 But this I believe is the description of exactly 6' what we want to do, and what we.would like to do is to find 7 appropriate regulatory -- appropriate rules or regulatory 8 guidance where we could do exactly what is up there on this 9 slide. 10 ^DR. FONTANA: Let me.ask a question to try and 11 clarify the direction in which your thinking is going here. l 12 You have got a performance-based regulation of some kind, it 13 leaves a lot of flexibility on the part of the licensee to l 14 develop his procedures and such-and-such. l () 15 16 DR. APOSTOLAKIS: have flexibility. It doesn't say a lot. It says 17 DR. FONTANA: Well, whatever. Whatever it is, it 18 is supposed to. Now, the question is, particularly related 19 to that last bullet that you have there, the last empty 20 bullet, the question is, the failure to meet a performance 21 criterion should not in itself be a major safety concern. 22 Okay. So that means that someone has to determine what the 23 consequences of not meeting a particular procedure or 24 whatever, whatever, whatever. 25 Now, the question is, who does that analysis that ANN RILEY & ASSOCIATES, LTD. ()/ g,, Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 ) l I l
39 1 indicates what the consequence of not meeting that 2 particular requirement is? Is that going to be the ( } 3 licensees? In which case, if all the licensees are doing it 4 differently, then the NRC is going to have to be able to 5 track a lot of different ways of doing it. Or will the NRC ; 6 specify, if you want to call it that, a methodology by which 7 one could do these analyses, and, therefore, save themselves I 8 a lot of work because the analyses are pretty similar? 9 Which way are you thinking? 10 MR. KADAMBI: Well, I am not sure that we have J 11 come to the point where we can offer thoughts that address 12 that question. 13 Ernie, did you want to say something? 14 MR. ROSSI: I was going to say that I guess the i l(3j l' 15 best example that we have is the implementation of the 16 maintenance rule, and I didn't come prepared to talk about ! 17 all the details. But I think, my understanding of what was 18 done is that that is a performance-based regulation and it 19 says you have to have criteria to judge whether a component 20 is being maintained in the proper way. And when you get to 21 that criteria, it ought to be based on something that you do 22 exactly what the fourth item up there is. 23 And with respect to who does it, what I believe 24 happened on the maintenance rule was we started to prepare a 25 Reg. Guide in the NRC and then the industry prepared their ('] ANN RILEY & ASSOCIATES, LTD. (_/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
40 l l' own document,.and then at some point we endorsed the I
\ l document prepared by the industry, so that it was done (d 2 3 pretty standardly across the industry.
4 DR. FONTANA: The~ industry being NEI. 5 MR. ROSSI: NEI, right. 'And then NEI got the
.6 ' industry's buy-in-on their guidance document. I believe 7 that -- and, again, I am speaking off the top of my head, 8 and there may be others that know more in the room.
.9- DR. WALLIS: Well, let's take this --
I 10 MR. ROSSI: But that is' kind of a model for how -- 11 that I think addresses your question. So it is an example 1 12 of what was done in one case, j 13 DR. FONTANA: Thank you. 14- DR. WALLIS: Let me suggest something. You said
) 15 it would be getting to a level of great detail. And one 16 could say the only thing that matters is CDF. The only 17 performance indicator that means anything is CDF. So I will
-18 have a CDF meter, it will run all-the time, and I will have 19 fines if it goes too high, and I will have rewards if it 20 goes too low, and that is all I need.
21- DR. MILLER: You have. great confidence you can 22 measure CDF that well? 23: DR. WALLIS: I have more confidence in some sort 24 of vision of what I am aiming at, at this sort of level, 25 ~ than getting lost in all the details and saying we don't / ANN RILEY & ASSOCIATES, LTD. Court Reporters
~1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
7 1 41 1 know how to procedure because we haven't had all the l) V 2 meetings. 3 DR. KRESS: In essence, Graham, if you look at the 4 Scientech report, one of their concepts, which I like, by 5 the way, was they define margins, in terms of the last 6 bullet,'in terms of what is the conditional CDP. And that, 7 in essence, if you use that as that criteria, that is, in 8 essence, doing what you said. It is just that you measure i i 9 CDF by looking at surrogates. 10 DR. WALLIS: If you have a vision of something l like this, it is how you might have the ideal performance l 11 l 12 indicator, -- l 13 DR. KRESS: I think that is one of the insights to l 14 come out of that. [-
\_
15 DR. WALLIS: -- then you can work out the detail. 16 DR. BONACA: And I think that is the process by 17 which that comes down in the diamond, which makes all the 1 18 sense. What troubles me is that if you really look for 19 perfection and completeness, you never get there. Okay. 20 Right now, you get to look at a situation right now, right 21 now the industry has been using for 10 years INPO 22 performance indicators, which is really what NEI is talking 23 about. And they are proposing it, they are monitoring 24 those, performance-based. Some of them in my book are 25 insufficient and I think the diamond evaluation presents
/% ANN RILEY & ASSOCIATES, LTD.
k ,)
% Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 42 l 1 1 that in a very articulate way. f~) 2 Now, the NRC, it seems to me, is monitoring the %J ' 3 industry below the beltline of the diamond, well below down \ l 4 there. Okay. So something is happening out there, which is 5 industry is monitoring up here and is not good enough. NRC 6 is monitoring down here, except in some cases like ALARA, 7 where they go up to results. And there is a pretty good l 8 workable way in which you can come in between, and I think ' 9 that the Scientech report shows that to be an effective 10 implementable way. 11 Now, I agree that we will never get completeness I 12 or proof, this is not research that will come out with 13 perfection. Okay. However, I think there is such an 14 opportunity for progress there, that I am kind of puzzled by (<~s)15 che shyness of -- you know, I don't see the movement in that ; 16 ll direction. 17 DR. KRESS: I agree with you. I agree. 31 MR. ROSSI: Well, I think the plant performance 19 assessment and oversight process is working towards exactly 20 what he has described there, because they are looking at the 21 things that can lead to core damage, like initiating events 22 and system reliabilities and trying to build a framework for 23 monitoring things. So they are doing -- that is another 24 good example of doing this. 25 They really have risk-informed thresholds and (~N ANN RILEY & ASSOCIATES, LTD. (_) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
43 1 risk-informed things to look at, so that is another example { Lf'd 2 that is pretty far developed at this point in time and is I V 3 getting a lot of input from the stakeholders in industry so 4 that it is bringing them closer together. 5 DR. KRESS: Mario, I agree with practically 6 everything you said, and in fact if you look at these four 7 attributes, basically you could say each one of them with 8 the possible exception of the third one applies to our 9 current regulations. It's just a matter of what level you l l 10 are at in determining your performance indicators. It is a 1 11 matter of where are you going to put that level. I think l 12 the diamond tree process does allow you to organize your ' 13 thoughts. It's a good way to look at your thinking. 14 DR. BONACA: Absolutely. l 15 DR. WALLIS: (")\ q Those bullets apply to anything. 16 Those bullets apply to academic education, when you measure 17 the student performance. 18 DR. KRESS: That's the problem. They are not good 19 principles to design to. j 20 DR. BONACA: That's one of the reasons why I said 21 that before. I feel the sense of urgency because the 22 industry is still looking at the top of the diamond there 23 with those limited performance indicators. They are saying 24 I am performing well. The NRC is looking down and they say 25 you are not performing well, and there is lack of t) (_/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
44 1 consistency or a common set of measurements that we_can use () (f 2 _ mathematically to help agree-on what is good performance and
~
3 . bad performance. 4 DR, APOSTOLAKIS: Let me propose this, and tell me 5 why_you can't do it. As a prelude, Professor Wallis asked 6 earlier what are other industries doing. It just occurred 7 to me that the fire protection community has proposed in 8 some countries like New Zealand, I believe, and maybe 9 Sweden, I'm not sure, performance-based regulations, you 10 know, for' buildings. The performance measure they use'is 11 really equivalent to our risk measures. 12 They are using individual risk as a performance 13 . measure. Now you try to calculate individual risk when you 14 have a fire in your high-rise building. There's a lot of ( 15' uncertainty, a lot of judgments, and yet that is what they 16 use. 17 I believe that in our agency using, in our 18 business using the top level goals is out of the question, 19 .because the agency has already stated that there are other things it worries about. We have a good example in the 21 oversight effort where we have the cornerstones, okay, so
.22 the agency is not only interested in core damage frequency, 23_ for example, it also doesn't want to see very frequent 24 initiating events, it doesn't want to see high 25 unavailabilities of mitigating systems and so on.
ANN RILEY & ASSOCIATES, LTD.
\ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034.
?
y 45 1 MR. ROSSI: But those things are tied to core
'( 2 damage. I mean they have a relationship and so they are
.f derived from.that concept.
4 DR. APOSTOLAKIS: And I totally agree, yes, but 5 what. I am saying -- I am -just arguing why core damage 6 frequency, for example, could not be the only performance 7 measure. .Why can't I write then 2.2174 -- whatever -- for 8 . performance-based approaches which says, the first three 9 bullets, has the same principles -- you see, what's missing 10 .here is the. expert panel, the integrated decision-making, 11 .which was not mentioned anywhere in your documents. That is 12 what I'like about 1.174, that all these things feed into an 13 integrated decision-making process. 14- Do you remember that figure, with the principles r ( ,) 15 feeding into the decision-making process? Where it says 16 delta CDF, delta LERF I replace that box by five other boxes j 17 that say the performance measures will be the-frequency of 18; initiating events -- and I will give you a delta F -- so if . 19 you are above this, your performance is not good. The 20 mitigative system unavailability and I will give you a delta l R21 Q, so you can't exceed that or you take action. Emergency 22 preparedness, and I'll give you sore criteria for that -- so 23 I replace it by these four or five, the cornerstones, and 24 then I have the equivalent of 1.174. That states very 25 clearly what I want to do -- and why not? Why can't I do O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036
-(202) 842-0034
46 l' 1 that? And that satisfies Professor Wallis's concern. () 2 It is high level, reasonably high level. It 3 satisfies defense-in-depth requirements, because of course 4 we have already set the -- that is a statement of 5 _ defense-in-depth at that level. 6 DR. BONACA: But the larger issue that Tom was I 7 talking about is not addressed here. 8 DR. KRESS: Neither that nor whether or not you 9 have a sufficient number of indicators. l 10 DR. APOSTOLAKIS: The. margin issue will be in the 11 deltas. 12 DR. BONACA: How do you deal with, assume that you 13 have a failure of -- I don't know -- high pressure or decay ! 14 heat removal system? That is one where Criterion Number 4 ) gs y ,) t 15 says it is unacceptable to monitor at that level because, 16 first of all, it is difficult to measure the frequency so 17 low, so it would be meaningless, but the other thing is once 18 you have a failure -- 19 DR. APOSTOLAKIS: A failure of what? 20 DR. BONACA: I don't know. The example they make 21 I believe is RHR. 22 DR. APOSTOLAKIS: No, but the cornerstones are at 23 the lower level. The cornerstones are at the mitigating 24 system level. The example of NEI is at the higher level. 25 DR. KRESS: Not NEI -- not NEI, the Scientech
) ANN RILEY & ASSOCIATES, LTD.
~/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l 47 1 report. l (f 2 DR. APOSTOLAKIS: The Scientech report says you 3 can't talk only about the temperature limit, because you 4 have to worry about losing component cooling water system. l 5 You have to worry about losing AC power. But each one of l 6 these has systems in it and you will monitor the 7 unavailabilities of the diesels, you will mcaitor the i 8 unavailabilities of the component cooling vater system, so l 9 the criteria are at the lower level thar the NEI. 10 DR. KRESS: You have already decided what level of 11 the diamond tree you are going to -- 12 DR. APOSTOLAKIS: I didn't. 13 DR. KRESS: Without a principle. l 14 DR. APOSTOLAKIS: The agency did. l [ 15 DR. KRESS: s
% I know, but that was -- they didn't 16 have the principle to guide them. They just chose things.
17 DR. APOSTOLAKIS: I don't know that you can prove 18 in a mathematical way that you have to go with initiating 19 events and so on. This is a value judgment. 20 DR. KRESS: Absolutely, and that is one of the 21 characteristics of performance-based regulation. It doesn't 22 show up there. 23 DR. APOSTOLAKIS: But it has to be a value 24 judgment. 25 DR. BONACA: You may have a good use there for an /~'% ANN RILEY & ASSOCIATES, LTD. (m) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
7
~
48
^1' expert panel.
L[ 2' DR. KRESS: Well, there needs to be a process by l 3 -which to guide that value judgment. 4 DR. APOSTOLAKIS: And the oversight program showed
- 5. .us_the' process.- It's not that'we didn't. talk -- they said
.6 _-we have reactor safety. We have -- what was the other 7' one? - radiation protection -- and I don't remember now.
8 There'were three or four of them. Do you remember'the 9 hierarchy they developed?
'10 DR. KRESS: I think Mario and I are saying that 11 the diamond tree process does give you' a structured way to {
12 make those value judgments. It tells you what to'look at. 13' Now it.doesn't tell you how.to look at. It doesn't tell-you i 14 how to look at them. It. tells you what to_look at. ] 15 DR. APOSTOLAKIS: And I think that is what the oversight' group has done. 16 They didn't call it a diamond, l 17 -but they developed a-tree -- 18 DR. KRESS: Sure.
)
19 DR. BONACA: Sure. 20- DR. APOSTOLAKIS: -- and it is a value thing, so 21' there'is no proof, and the ACRS did not object to it. 22 MR.'ROSSI: -It seems to me that in the 23 implementation or in the writing and the implementation of 24 'the maintenance rule pretty much what you described has been 25 done, because what they do in the maintenance rule is they ) ANN RILEY & ASSOCIATES, LTD. ' V ' Court Reporters i 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 ; (202) 842-0034 '
P 49 1 first identify the most risk-significant components in the
}.
2 . plant and'then they' apply a process where they monitor how 3 well'those components perform.and then when they are not 4 performing well enough they go back and look to see whether ; 5 maintenance-is_the reason, so that is being'done there, and
.6 I think that that certainly is a good approach, and the tie 7- to core' damage frequency is that they have to go through in 8 the maintenance rule and determine the risk significance of 9 -the components.
I 10 .That is the tie, and that ties it to core damage 11 frequency. ' 12 DR. APOSTOLAKIS: Right, but it's at such a high 13 level though, i 14 DR. BONACA: Yes, well the thing I like about the j () 15 approach that we are presenting in the document is that is 16 actually the approach to determine whether the performance 17 indicator, which is performance-based, is acceptable or not. 18 If it isn't acceptable, you go a step below until you find
.19 out that will be acceptable, because it will have all these 20 four attributes met.
21- DR. APOSTOLAKIS: I didn't see that, by the way. 22 DR. BONACA: I don't think that you have many 23 cases where you have to go so deep, but you may have some, 24 and I think it's just an exercise and again, if you are not 25 afraid of completeness, okay, then you can go to the ANN RILEY & ASSOCIATES, LTD. O' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 n-
50 1 exercise. ( .2 DR. KRESS': The weakness there was you determined 1 3 its acceptability by some criteria. One, can you measure it a 4 at that level -- i 5 DR. BONACA: That's right. 6 DR. KRESS: -- but two is what its conditional 7 core damage frequency is, and that requires a PRA. 8 DR. BONACA: That requires a PRA, yes. 9 DR '. KRESS: You'll read some places where you have i 10 to deal with it a different way, and they talked about it. 11 They recognize that. l 12 DR. BONACA: But there is information at the 13 plant-specific level to extract that information and make
- 14. the judgement, and again a panel, expert panel, would be 15 _ critical to do that tailoring.
16 DR. APOSTOLAKIS: That is why you need the 17- equivalent of 1.174, to set those principles, that these 18 four attributes will be declared as satisfied by a panel. 19 DR. BONACA: .Yes, j 20 DR. APOSTOLAKIS: So you need a principle like 21 that, and I think we should separate values from technical 22 arguments. I mean there is no technical basis for saying
.23 yes, I want the frequency of initiating events to be low.
24- This is a value judgment. The agency wants that low because 25 - of public outcry, potential public outcry and so on. f) \ j. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
e 1 l 51 1 Now once you-have declared that as your objective, l'& \ '2 'then defining the proper performance indicators to make sure 3' .all that happens'is a technical issue. Okay? 4 So first of all, this work was-done'I guess in 5 parallel to the oversight program so that's why me don't see ' 6 the.: impact fromLthat, right? 7 MR. KADAMBI: There were many. things happening at 8 .the same' time and there.were many people that were common to 9 both of them but what we see over here are the products of
- 10. discrete efforts --
11 DR. APOSTOLAKIS: Yes. l 12 MR. KADAMBI: -- and we have to pull them ' 13 together. 14 DR. APOSTOLAKIS: I don't blame anybody for that. 15 I mean that is the way things happen, especially if you want 16 to do a lot of things in a short period of time, but it 17 .seems to me that having a short document that very clearly 18 ' states these principles emphasizes the integrated nature of 19 the decision-making process, identifies the issues pretty 20 much like what Scientech did, what you should worry about, 21 because if you go to 1.174 it doesn't tell me what to do 22 about model uncertainty. 23 All it tells me is that I have to worry about it. 24 It doesn't tell me what to do about incompleteness, but it 25 tells me, look, we may ask you that, but it is a statement ANN RILEY & ASSOCIATES, LTD. O' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l l, 52 1 in black and white that you have to worry about it. l i f) V 2 Now in some instances it is not important. In I 3 others it may be, and I think a short document like that 4 will be useful, and the second point is we cannot afford to have different values in one project, in the oversight 1 5 6 project, you know, where they declare the cornerstones as i 7 being the driving force, and a different set somewhere else, 8 so either as an agency we decide that the cornerstones are 9 indeed the way to go, or if you guys don't like it for 10 example, then you have a debate with the oversight team and 11 say we don't like this or we want more there and less here, 12 so we will have an integrated approach at some point. 13 But I think the statement of principles and what 14 needs to be done and what you should worry about would be a O ( ,/ 15 useful document to have, and it will also show progress in 16 this area, because a lot of the stuff that I read in the 17 documents you sent us really tries to go way down to the 18 detail and solve the problem, which I think is not wise, 19 similar to what 1.174 -- 1.174 did not try to solve the 20 risk-informed regulatory problem. 21 DR. WALLIS: I would like to say what is happening 22 here. I mean you'll notice the discussion has come up from 23 the ACRS, so what's happened is that the ACRS is thinking 24 about the problem and coming up with ideas which look good. 25 What we hear from you is you haven't got much input from T ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
53 I 1 industry, you've got to do research. [} sv 2 You looked at this problem for a year or 3 something. I would expect you to have had all the ideas 4 that we have had in the first few weeks or days and to come 5 up with some sort of a much more mature thing. 6 We have looked at the problem and this is what i 7 needs to be done. These are our ideas and the reason we are I 8 doing it is because we don't hear it from you.
]
9 MR. FLACK: If I could make a comment to George's 10 earlier comments. This is John Flack from the Office of 11 Research. I am also in the same branch working on the same 12 problem. I 13 Getting to the cornerstones, we did do that. In i 14 fact, something very similar -- they called it a football 7- 1 , ( ,) 15 and not a diamond tree, and they did go through the thought
)
16 process to develop at what level you would start to collect 17 information for the PI. 18 When we are talking about 1.174 we have to 19 remember that 1.174 had a certain purpose, and that was to 20 make a change in the licensing basis of a plant, and in 21 there one of the principles was also performance monitoring, 22 so_it does capture this element, and when we are talking 23 about another guide, I am trying to think of where this 24 application would be. What did we not capture in 1.174, for 25 example. /~D ANN RILEY & ASSOCIATES, LTD. 5 -)
% Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l
)
/
54 1 The intent of what Prasad is presenting is we are 1 ( 2 . intending to do is stand back and look at our regulations in 3 a global sense, not a specific plant or a change to a ] l 4 specific plant, which was the intent to 1.174 -- to caange ' 5 -something on a' specific plant, but to look more globally 6 across the regulations to see if there were areas where we 7 could be less prescriptive, and that was where this was 8 coming from. 9 Now the concept of developing another guidance for 10 performance-based, I am still trying to understand where 11- that would go with respect to changes to the licensing basis 12 that would not be captured today by 1.174. I i 13 DR. APOSTOLAKIS: The 1.174 does not address these 14 things, so that is what I am saying, that you would have -- ; () 15 I' don't know if it has to be a regulatory guide,'but a 16 document. 17 MR. FLACKi Yes, a document of some sort, right. 18 DR. APOSTOLAKIS: But if you want to go this way, 19 here is the way to do it at a fairly high level, the same 20 level that 1.174 is on. You will have to have these 21 attributes. You will have to have an integrated 22 decision-making process, and you have to worry about certain 23 things. 24 For example again, I really like Chapter 2 of the 25 Scientech report, where they took -- I think it is Chapter O ANN RlLEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
7: 55 l 1 2 -- the NEI example and analyzed it, talked about the cut l i ['
\~
2 sets and so on. Something like that would be very useful to 3 have. It will not change the direction you are going. All l 4 I am saying is it will be an intermediate document that will ; 5 solidify, so to speak, all the thinking that has gone into I I 6 this without trying to go all the way down and solve the 7 whole problem, which is the way I think you are going now. 8 You are really trying to find performance goals at a fairly 9 low level. 10 DR. KRESS: I think -- to say it another way -- I 11 don't think George is saying take anything explicitly out of 12 1.174, but use it as a guidance as to how to structure the 13 approach to this thing. 14 DR. APOSTOLAKIS: Yes.
/%
; ) 15 DR. KRESS: 1.174 has very little to offer in the I
%/
\
16 sense of things you can take right out of it. It is a 17 process -- 1 1 18 DR. APOSTOLAKIS: Exactly. ! 19 DR. KRESS: -- and a way to develop -- 20 DR. APOSTOLAKIS: And you have already done SO 21 percent of the thinking. This is not new to you because you 22 have already thought about it. 23 So pull all that together, create a document, and 24 then the other thing, John, that I am not sure about, is 25 this idea that you are trying to do it in a generic way and (T ANN RILEY & ASSOCIATES, LTD.
\s/ Court Reporters 1025 Connecticut / +>nue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p. h; h 56
~1~ -identify performance indicators that would apply to the
- / 2 whole industry.
3 The question is why didn't 1.174 try to do that?
- 4. Why not try to'a risk-informed, develop a risk-informed 5- l approach that_would be industry-wide? It didn't do that.
6- -It said no, the licensee should do that. It is too 7 plant-specific, and then we saw that in the graded -- and 8 too issue-specific. 9 In the GQA -- if you read the GQA Regulatory i 10 . Guide, you find things in there that are not in 1.174. 11 There is an implication that if you do these things, delta 12 CDF and delta LERF will remain acceptable, but we never 13 really quantify it, and 1.174 doesn't say anything about 14 that, so the question really is is it feasible to do what ( )' 15 you are trying to do? 16 MR. KADAMBI: Well, let me break that question 17 into two parts. You began with the question,.why can you 18- not do this? And I can't answer-the question why we cannot 19 do this. It seems like something we could do, but we would 20 have to be in a much better position that we are today in 21 terms of what is going on in the cornerstones area and 22 'everything. 23 So, you know, I think what I hear from you is we 24 'need to pursue what Ernie said we will be doing, which is 25 making this a much more integrated effort and learn from i-L ANN RILEY & ASSOCIATES, LTD. k- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
57 1 that. es (N ) 2 DR. APOSTOLAKIS: But let me offer you another 3 thought. I think there is an implicit assumption here -- I 4 may be wrong, but I think there is an implicit assumption 5 that these four attributes should lead to performance 6 criteria that will be equivalent in some sense to what we 7 are doing now. 8 MR. ROSSI: Well, to give an equivalent level of 9 safety I think is correct. But what this is intended to do 10 -- I mean basically what performance-based approaches do is i 11 they get you out of the mode of looking at the procedures l l 12 for doing the maintenance, and they get you at looking at l 13 how well is the equipment working. 14 DR. APOSTOLAKIS: Right. I understand. () 15 MR. ROSSI: And so you want to do that in other 16 areas. But the idea is you get at least the same level of 17 safety, maybe even an improved level of safety, but you get 18 a much more focused effort and more flexibility on the part 19 of the licensee. 20 DR. APOSTOLAKIS: Right. And I would call that a 21 bottom-up approach. Right now we have a certain level of 22 safety by doing certain things. Now, we look at an area, 23 and we say, well, gee, we really don't need to do this group 24 of things here because we can set the performance criterion 25 a little higher and let the licensee worry about it. l)
\/
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 l (202) 842-0034 e
I
)
58 1 In a top-down approach, like the oversight program \
/s j did, you ask yourself first, what are my objectives? We 2
l ! U) t i 3 don't ask the objectives here, what the objectives are and 4 work down. We say, if my objective is to limit the 1 5 frequency of initiating events, then what would be the i j 6 performance criteria that would do this for me? Okay. So 7 the approach is philosophically different. And that is the 8 kind of thing that, again, a principles oriented document 9 should address. What exactly are we trying to do? 1 10 MR. MARKLEY: I think that is part of what John l I 11 was talking about here. It sounds to me like they don't l l 12 have a regulatory decision in mind. I mean if you are l 13 looking at Reg. Guide 1.174, it is a change to the current 14 licensing basis. Here it doesn't appear to be a particular () 15 decision that they have in mind for it. So it is nice to I 16 have this information, but what are you going to do with it? 17 MR. ROSSI: Well, our intend would be, again, to l 18 look at can we change rules so that you focus less on 19 procedures and more on performance and give a license. So 20 it is actually rule changes and change to the guidance that 21 we are looking at here. So that is the decision. l 22 MR. MARKLEY: 1.174 doesn't address changing i 23 rules, per se, it is difference means of meeting the rules. l 24 MR. ROSSI: Right, I know. 25 DR. APOSTOLAKIS: Right. But a document like i I ANN RILEY & ASSOCIATES, LTD. (_)) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 l (202) 842-0034 l l
F 59 ! l' that, in this. context, would address the question I just ! ()
/ 2 raised. What:are we trying to do? Are we trying to 3 maintain equivalence with the existing system?- Or are we 4 changing our approach and we are going.now top-down and we 5 are saying the objectives are -- the cornerstones or maybe 6 something else, I don't know. But, finally, we have to 7 settle on that. We.can't have a set of cornerstones for the 8 oversight program and another set for somebody else.
9- MR. ROSSI: Well, I think in principle that we 10 would focus on the cornerstones wherever they can be focused
.11 on. Now, there may be some areas like fitness for duty rule
'12 and that kind of thing, where you can.have a I 13 ' performance-based approach, and you can't tie it directly to l 14- the: cornerstones. And I think that is the areas that we are
() 15 looking at.
]
16 A lot of this stuff I believe, when you tie it to 17 the cornerstones, does come out of the effort to risk-inform i 18 the regulations. I mean I think that will come out of that 19 part of it. And so that is why we are trying to expand it 20 in a coherent way to things like fitness for duty, maybe 21 more into how you do the quality assurance and quality 22 control, and there were some other areas where you can't as 23 easily tie it to the cornerstones. 24 DR. APOSTOLAKIS: And I appreciate that. I mean 25 it is not that I think the cornerstones are the answer to O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
60 1 everything. But these~four attributes, I guess my comments /N 2 'really refer to the last three:words of the fourth bullet, U
.3 "immediate safety' concern." Maybe we can change those, I 4' mean instead of calling it immediate safety concern, use 5 something like objectives ~or something.
6 But right now, this doesn't'tell me what would be 7 of safety concern. Scientech assumed that it was core 8 damage, and I don't know that is the case. 9 DR. WALLIS: Isn't-it very simple? This is 1 10 performance-based, it is not independent of risk-informed. 11 They go together.
- 12. MR. ROSSI: They go together, yes.
13 DR. WALLIS: Risk-informed, to me has to mean a 14 measure of risk. () 15' MR. ROSSI: Yes, they go together, no question. 16 DR. WALLIS: And I understand that is something 17 like CDF. So the only question is, what can you measure 18 that tells you something about how they are doing on CDF? 19 That is the only question. Why is it so complicated? 20 DR. APOSTOLAKIS: If CDF is your objective. 21 DR. WALLIS: Well, that is what risk-informed 22 mean, isn't? If it means something else, then tell me. 23 DR. APOSTOLAKIS: No , but the agency is on record 24 as.saying that it is not only the core damage that they are 25 interested in. They don't want to see -- O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
~
p: 61 ] J 1- DR'. WALLIS: But instead of facing the main l 2 question, 'we have sort of_ spread it out into.all the
.3 details.
4- ' DR . APOSTOLAKIS: No, I agree with you. 5' DR. WALLIS: Let's go to the heart of.the matter. 6 DR. APOSTOLAKIS: The main question is, what are 7 the objectives of that? 8 DR. WALLIS: But it seems clear to me -- maybe I 9 am wrong, but isn't it to implement this CDF as a measure of l 10 risk and to figure out -- l 11 DR. APOSTOLAKIS: Well, maybe a level lower than 1 1 12 CDF, but it is certainly up there. 13 DR. BONACA: But, again, if you look at the
,14 example in the context of the bullet number 4, you know, the
() 15 example is when you_ lose RHR in mid-loop operation, okay, 16 and only rely on operator action to recover. Okay. Now, 17 what they are saying is that if you rely on an indicator to 18 measure your performance, and you fail just that time, that 19 is not good enough, for a couple of reasons, but the main 20 reason is you can't rely purely on these recovery actions as 21 a means of, you know, staying away from core damage, and 22 that is the point of margin that we are talking about. 23 So that is a case where, Graham, they show that 24 just looking at performance, okay, is not good enough, 25- because it doesn't give you the margin there. , /' ANN RILEY & ASSOCIATES, LTD.
.- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
62 l
)
1 1 DR. APOSTOLAKIS: But, Mario, what you just said I
/~'g o
( ) v-2 think is consistent with what Graham and I have been saying. 3 They used, as a measure of how close to an undesirable 4 situation, CDF. That is what Scientech did. 5 DR. BONACA: Yes. 6 DR. APOSTOLAKIS: And yet that is not mentioned 7 here. 8 MR. ROSENTHAL: This is Jack Rosenthal, I am 9 Branch Chief of the Reactor -- 1 10 SPEAKER: REA. ) 11 DR. APOSTOLAKIS: We know who you are. 12 (Laughter.] 13 MR. ROSENTHAL: I am still learning how to say my 14 name.
,8N
( 15 DR. APOSTOLAKIS:
'm, ) We know who you are.
16 DR. MILLER: That is your title today. 17 DR. SEALE: We know who you are, Jack. 18 MR. ROSENTHAL: Clearly -- clearly, we want to 19 learn from the oversight process. We don't want to 20 duplicate the process. I think that if you look at the 21 underlying history of the documents, in the area that is 22 risk-informed, the agency is moving forward, taking steps, 23 it has plans to risk-inform Part 50, t cetera. We will 24 learn from that exercise. 25 My concern is how do we take on these other () (s ,/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p L 63 l; 1 programmatic activities that are not.necessarily amenable to () 2 3 delta core damage and that the bigger contribution to the agency would not be to duplicate an already identified
;4 effort, but to take on come of these other roles such as 5 fitness for duty, training,. quality assurance, commercial 6 ' dedication, procurement processes. There is a whole, in my 7 view, perhaps burdensome infrastructure that could be 8 replaced by more performance-based goals. And that is --
l 9' DR. MILLER: CDF can't be a measure of that. I l 10' DR. APOSTOLAKIS: No, but it can not -- 11 MR. ROSENTHAL: I am sorry. what was your -- 12 DR. APOSTOLAKIS .: CDF can not be a measure of
'13 that. I 14 MR. ROSENTHAL: Right. And that is why we want to .
()' 15 come up with other attributes. And one of the things is, is l
.16 this the right four attributes? What other attributes 17 should we be looking for? Do you' agree with that?
18' I think of the discussion that you have been 19 having on do we look at the core damage frequency, what we 20 do at the equipment and reliability level is very important 21 to us. My own view is that you should do things which are 22 measurable, although not necessarily reported to the NRC, 23 nor would you necessarily require that it be reported ever, 24 but that it is directly measurable. And you could do more 25' than count, maybe you want to count and divide by something. O t i ANN RILEY & ASSOCIATES, LTD. v Court Reporters i 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 l (202) 842-0034 l,
64 l 1 But you might not want'to get into a fancy numerical scheme. '( 2 But that.is my own view, and I would like to hear 3 your views ~on -- well, you called it, Mario, the beltline, i 4 About how far down do you think we should go? And get away 5 from the initiating frequency example we are all too ) 4 6 familiar.with, and let's talk about_some hypothetical 7 training or some other programmatic activity. At what level 1 8 should we get going? And that is where the guidance would 9 help us. 10 DR. APOSTOLAKIS: And I understand that. I guess 11 what I am saying is these four attributes are not sufficient 12 to define performance criteria -- or goals, actually, not 13 criteria -- goals. 14 And one question, one issue that they do not ( 15 address is what objectives you have. What is the immediate
'6 . safety concern? Now, in the PRA case, I think it is easier 17 if you have a PRA. You can take the four cornerstones. If 18 you don't have a PRA, then you can still have objectives and 19 see how the other thing.
20 MR. ROSENTHAL: Yes. 21 DR. APOSTOLAKIS: What did you say? 22 DR. FONTANA: Excuse me. Go ahead. 23 DR. APOSTOLAKIS: Did you say " guess"? 24 MR. C.OSENTHAL: No, "yes." 25 DR. APOSTOLAKIS: Oh, "yes." Okay. Now, so the 1 j O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 , (202) 842-0034 l J
65 1 objectives I think is important. () 2 3 Second, and I think Jack just put his finger on it, calculable. Do you go with the fire protection 4 community's approach, you know, the individual risk? I mean 5 there is so much judgment going into it. Is that really a 6 valid indicator? Unavailability is very simple, you divide 7 two numbers. Nobody questions that. So where do you draw 8 the line? That is a real issue here. Because when we say 9 "or calculable," actually, there is much more to it, 10 calculable and perhaps believable, or with some confidence. 11 DR. FONTANA: Well, it doesn't -- 12 DR. APOSTOLAKIS: So, I still think a document 13 like 1.174 should put all this together, and it will be very 14 useful. r~ (T) 15 DR. FONTANA: Well, I think those attributes, I 16 think are very good attributes. I think they are subsidiary 17 to a higher level objective, like George says. 18 The thing is there is much -- a slip between the 19 cup and the lip, in that these are all very good, but when 20 you start to apply them in a real care, then you see the 21 problems that are involved in trying to make them work. 22 Now, the question is, what do you have in mind in 23 using these for test cases? 24 MR. KADAMBI: All I can offer at this point is it 25 is part of our plan to sort of look into this sort of \ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 66 1 question. Where might the pilot projects come from? (} . 2 But, Ernie, you were going to say something. 3 MR. ROSSI: Well, yeah, what I was going to say is 14 that'we have had a couple of meetings now, and.this could 5 even-be considered as a third meeting, where we have the 6 question, and it is one of the questions that is in your 7 handout there - what specific rules might we use.to apply 8 this technique to? Because what we have is we have a fairly 9 -- a very robust effort underway to apply PRA to the 10 regulatory process and risk-inform the regulations. And at 11 lot of Laat already includes performance-based approaches-12 like in1the oversight process. 13 So what we would like to do is find examples that 14 the industry would support to apply this process to outside () 15 of the.other things that are going on, but are coherent with 16 .it. And as yet,1we haven't heard any specifics at this 17 point in time. 18 DR. FONTANA: But the test case I think would have 19 to include some of the things that Jack raised. 20 MR. ROSSI: Right. 21 DR. FONTANA: Things that are not measurable. By 22 measurable, I mean it may not come out with a number, but it 23 is something that could indicate a qualitative assessment of 24 like -- what will one more screw-up do to me?
- 25. DR. KRESS: Yeah, but, Mario, let me comment on
,Q ANN RILEY & ASSOCIATES, LTD.
(/ Court Reporters
.1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
b 1 1 67
'1 that. .Because iffyou can apply a PRA, then.I think things
[~).
\~/
2 will[ fall out-and we.can figure out where.to go. And, ) { J3 technically, _you-can get there.
.4 DR. FONTANA: That's fine. l 5~ DN. KRESS: Those things you can't apply a PRA as 6 analogous to_our problem with the process versus product in 71 the INC. With a PRA you'can determine the product, the 8 change in risk, the change in something. .You don't -- and'I 9' think Jack is saying, well, we don't want to go to a new 10' rule network where we correlate these things like QA and i 11 management to a change in the soft score even. That is 12 going a little bit too far. And I think what you have to do 13 in those is you have to focus on the process again, at least
-14 I'would think, when you get down to that level.
() ~ 15' MR, ROSENTHAL: If I can offer, we can go back and
)
)
-16. look at.Part 20 and say, has that been a -- what is the
{ 17- story, has that been a success? And has it been 18 implementable, measurable, et cetera? We can look at 19 Appendix J., The story there will be a mixed story, but we 20 can look at our experience with that. 21 DR. MILLER: When you say look at, does that mean 22 you are going see if these attributes are applicable? 23 'MR. ROSENTHAL: How many people -- right. Can you 24 measure it? I mean, you know, is it working? Do we have 25 the intended containment integrity that we wanted when we
/ ANN RILEY & ASSOCIATES, LTD.
( Court Reporters
~1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
68 1 moved from a prescriptive to a more performance-based rule l
- 2 change? Are we still maintaining containment integrity? Is j 1
3 it an implementable rule?- How much of the industry has ) 4 adopted the rule? So we can learn from the past. 5 We can learn from the maintenance rule. Okay. l 1 1 6 And we can learn from.the performances -- the new oversight 7 process, and we intend to do that. What we would like -- 8 and that is in our plan, to go back and learn from those 9- _past experiences. I 10 We would like to also identify, and we have it 11 listed as pilot programs, some conceptual areas where we 12 could -- it would be pilot exercises where we would identify l 13 some other rule maybe, some training, although there doesn't 14 seem to be enthusiasm. I would just go to fitness for duty D _q) 15 as an example, where maybe you say that the amount of drug 16 testing at a plant should be proportional to the problems 1 17 that they find at the plant. You know, could come up with 18 some scheme. I am only using this as an example to push us 19 away from the reactor one. I 20 And then having nominated some new rule to be i 21 changed from performance to -- from prescriptive to 22 performance-based, then use that as a test case, a pilot 23 case. And we would love to hear suggestions for rules or 24 Reg. Guides that would be candidates for this pilot. 25 DR. MILLER: It seems like you ran through a ANN RILEY & ASSOCIATES, LTD.
- O Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 ,
Washington, D.C. 20036 ' (202) 842-0034
c 69 1 pretty broad gambit right there. 2 MR. BARTON: There are some good ones right there. [dT 3 MR. ROSENTHAL: Those are the ones I know about. 4 So what about the ones I don't know about. 5 DR. SEALE: Jack, you made a reference a while ago 6 to the idea that there were some performance indicators that 7 might not be reported to the NRC. 8 MR. ROSENTHAL: No, what I said was that -- I'm 9 sorry. That -- well, one, we would like to hear your views 10 on the first bullet. Is that an appropriate attribute that 11 would be measurable? And then I just offered up that on a 12 personal basis, in my own view, I don't even think they have 13 to be things that are reported, but rather things that at 14 least the licensee could measure. (~% 15 DR. SEALE: Very good. That is the point I was
) j 16 driving at.
17 DR. FONTANA: Well, they ought to be measurable 18 where you can measure them, but there are a lot of other 19 things that you can't. j i 20 DR. SEALE: Could I make my point? 21 DR. FONTANA: Oh, I am sorry. I thought -- 22 DR. SEALE: The aspirations of the industry I 23 think are that as we go to a more risk-informed and l 24 performance-based regulatory process, that there is also 25 going to be a disentanglement of the NRC inspection process (~ ANN RILEY & ASSOCIATES, LTD. (_ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 70 1: and a lot'of things with the_ day-to-day operation of the
'2 plant. At the same time there.is a recognition within the )
3 industry, and I think this is a fair statement, that there 4 i are things that they need to keep track of that are not on 1 5 :the level of the NRC's radar screen. 6 It strikes me that whatever process they come up 7 with there is somehow a rich area to be mined for the kind 8 of information that you are talking about. What is the
- 9. integral effect, or the cumulative effect of these'non-radar 10 screen detectable, measurable, trackable and verifiable and 11 comparable across the industry between plants and so forth?
12 What is.the sum of those experiences that somehow might take 13 a form that would be useful as an overall performance 14 indicator for you? () 15-16 Now, obviously, you can't answer that question till you know what these things are. But we also know, for 17 example, that there are people in INPO who have said we are 18 interested in other endpoints for assessing satisfactory 19 performance besides core damage frequency. If it is that 20 late, forget it. I mean we are already dead in the water. 21 Certainly, that plant is dead in the water. We need to find 22 things that are sensitive at a level that is low enough to 23 tell us that there are things that need to be done before we 24 .get to the core damage frequency. 25 One of the things that bothers me is if you get N ANN RILEY & ASSOCIATES, LTD. ms / Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
71 too high on'this list, .your feeling the problem through
~
1 2 awfully' thick mittens. You are not. feeling the problem at a
- 3. low enough-level.
4 Now, I agree at the same time you can't be all 5 nerve endings down there. That is the problem we have had 6 in the past. 7 DR. MILLER: The key question is which level you 8 want to be. 9- DR. SEALE: Well, there is somewhere in between. 10 DR. WALLIS: Well, the key thing, what do you 11 first? You can't say we won't core damage frequency because 12 we are worried about drug testing. If you know how to do 13 CDF, you do it. You don't want until you can -- 14 DR. SEALE: We need to ask ourselves what are () 15 :these other things. 16 DRs WALLIS: But you do, you bear it in mind. But 17 you do the high priority thing you can see how to do first, 18 and get on with it. 19 DR. APOSTOLAKIS: I think we should clarify 20 something. Core damage frequency itself, if it reaches a 21 certain level, is not too late, it is a frequency of an 22 event. It is the core damage itself you don't want. Let's 23 not confuse the two. 24 DR. SEALE: Yeah, but there are a lot of things
- 25. that have to with unavailability.
O ANN RILEY & ASSOCIATES, LTD. (msl Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 i Washington, D.C. 20036 I (202) 842-0034 i I l 1 j
I 72 1 DR. APOSTOLAKIS: Sure. ( 2 DR. SEALE: And the adequacy of the maintenance 3 and all that sort of thing,. which are a lot easier and more 4 sensitively detected-at a lower level of screening, if you 5 will.. 6 DR. APOSTOLAKIS: I agree. I' agree that there are j 7 issues that, you know, you can not see the core damage i 1 8 frequency. l l 9 DR. SEALE: And what are those things that the )
.10 industry people are looking at? They keep coming to us. I l
11 think you need to'go look at them, too. j 12 DR. BONACA: Let me just say about that, because i 13 that ~is really a point, right now, if I built a hypothetical 14 diamond there and I try to fill all these boxes, I would say i l (O) 15 that_almost every box is being tracked somewhere inside a 16 plant, literally. There is information being gathered. I 17 mean there are hundreds of indicators out there. What is 18 terrible for the plants is that they don't know oftentimes 19 what it means. They know specifically that if I measure ' 20 this, I get a feedback on that particular piece of 21 equipment. 22 But the confusion they are having, they have no 23 guidance from anywhere on what counts and what doesn't 24 count. 25 DR. APOSTOLAKIS: No, as to what the objectives I ANN RILEY & ASSOCIATES, LTD. f'Ng_) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r e 73 1 are. 2 DR. BONACA: . What the objectives are. 3 DR. SEALE: Yes. 4 DR. BONACA: So what happens is that then the 5 corrective action program may be extremely important, but if
)
6 the corrective action program is overwhelmed by the search 7- for comments missing.on some documents,.then you just don't 8 have focus on what counts. And I think the objective here 9 on our part should be the one of developing some guidance on 10 how do you go back from the very important, and how far you i 11 monitor. And at times you may have to get down to the I i 12 corrective. action program. 13 DR. SEALE: What are the diagnostic requirements 14 of the corrective action program? () 15 16 DR. BONACA: But the point I want to make, again, the resources are being expended today. The question is, 17 where do you draw this line that_is not just a straight 18 beltline out there, but is somewhere, you know, just jagged 19 to reflect insights from PRA? Really. You know, how do you 20 draw the line? 21 DR. KRESS: I am not being a very good junkyard 22' dog, I am letting the meeting get away from us. I regret 23 it, but I do have to declare a recess at this time because 24 we have some other obligations we have to meet. You know, I 25 hate to break in right in the middle of the discussion, but O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
h 74
- 1 LI.am goingitofdeclare a recess for'15 minutes, until 10:30,
~/N 2 at this time, at'which time we will come back and I will try b 3 to ride' herd on this bunch'a little'better.
4 (Recess.] 5 DR. KRESS: We need to get started again. George 6 may be delayed aflittle while, but he will join us as soon 7- as possible. In the meantime, I-think we can continue with
.8 your presentation. Let's see, .I guess we are on what, the
- 9. second slide? Okay.
10 MR. KADAMBI: We have been through a few. Thank 11 you, Mr Chairman. I would like to resume the staff's 12 presentation. 13 Where we left off was on the discussion of these 14 attributes which are taken directly out of the white paper,
-() 15 and, again, I want to reiterate that the white' paper-has a 16 lot ofLstuff in it. It tries to pull together many of the 17 ideas that will help us with risk-informed, E18 performance-based approaches to regulation.
19 Having gone through the white paper, I tried to 20 capture on one slide what seemed to me to be rather 21 important messages coming out of it. The first one is that 22- the deterministic approaches should be changed but should be 23- done incrementally. This isn't something the Commission is 24 looking for to do wholesale. 25 DR. KRESS: Let-me ask you about that, do you O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
75 1 think that is possible? You know, it seemed like if you
/~T ~2 ;followedithe advice in the Scientech report that that is U
3 going to be:very difficult to-do that way. -You always have 4 to take the regulations as a whole.
,5 MR. KADAMBI: It.is not for me to argue with the 6 ' Commission.
7 If the-' Commission tells you do that, I guess you 8 have to do it is what you are saying, yeah. 9 DR. WALLIS: It reminds me of the approach to 10 Kosovo, you set out your objectives ahead of time. 11 MR. KADAMBI: I want less to argue with the 12 Commission. _) 13' DR. WALLIS: Then you say that, yes, but you 14 should say we -- our hope is that in ten years we will have 1 . ,d~5 ( () 15- 90 percent' performance-based or something,-'some sort of a j 16 goal out there. If.you are saying you are tentative and 17 incremental, then that is a pretty poor way to start a major 18 program. i 19 MR. KADAMBI: I think really at this stage doing { 20 it incrementally makes more sense to me than to undertake ! 21 something, because although there was a suggestion made that 22 maybe we are trying to solve all problems at the same time, 23 really, what we are trying to develop, I believe is some 24 sense of confidence that there is in fact a success path 25 that we can chart, you know, before we really bite off too ANN RILEY & ASSOCIATES, LTD. Q(_f Court Reporters
~025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
m 1 76 1 much more-than we can chew. i
'DR. WALLIS:
e _2 I think'we-all agree with that. If 3 just put incremental upfront instead of the long range 4' _ objective'upfront, it gives the wrong message. 5 DR. KRESS: Well, he is just extracting the ; i 6 message he got out of the white paper I think. j i 7_ MR. KADAMBI: Yes. I am just trying to reflect -- l 1 8 DR. WALLIS: I guess I have the benefit of 9 independent thought. 10 DR. SEALE: I think there is another value to 11 Graham's suggestion, too, and that is it would be nice to 12 .take the' Commissioners' temperature on this issue. Do they j 13 . expect 90 percent in 10 years, or 5 percent in five years, 14' or what? (3 15 DR. KRESS: ( j I think you have to see if this thing { 16' works first, before you make a wholesale change in the' l
' 17 ' regulations. I kind of agree with this, you try it where 18 you can. You know, it is too risky to change the whole l
~19 regulatory structure all at once.
20 DR. SEALE: I agree with that, Tom, but on the 21- other hand, I think we also will have to agree that not very 22 long ago we reached a point when some of us were asked 23 whether or not we thought the NRC staff was prepared to move 24 to risk-informed regulation, and I think the answer they got l 25 was they are as prepared as they are ever going to be until l ( ANN RILEY & ASSOCIATES, LTD. T, Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
i 77 1 you tell them that they will. ( 2 DR. KRESS: I see. 3 DR. SEALE: So the will, a clear-cut statement of 4 the will of the Commission is an important attribute in 5 motivating that process as well, and it probably would be a 6 good idea to remind the Commissioners of that occasionally. 7 And, also, to pass on to the collective Commission, as it 8 evolves with different members and so forth, a clear 9 statement of what the aspirations of the existing Commission 10 is that launched the Commission -- the staff on this 11 undertaking. End of speech. 12 DR. FONTANA: When you look at the top to bottom, 13 the objectives to the bottom all the way down through the 14 operations, down to the final preparation of procedures, () 15 ultimately, they are all prescriptive, aren't they? Those 16 procedures that are usually written by the licensee are 17 prescriptive. They almost have to be, right down to the 18 very bottom line. And the question we are asking here is, 19 where along this spectrum the NRC does its regulation? Am I 20 helping any? More confusing? 21 MR. KADAMBI: Well, if I can -- 22 DR. FONTANA: In other words, the prescription is 23 always there at some level if you go deep enough, you know. 24 MR. KADAMBI: I think there are many prescriptive l 25 elements associated with procedures and things like that. ANN RILEY & ASSOCIATES, LTD. ()T (, Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 ; 84 I b34 l 1 J
.78 1 But I think today what we think of as procedures and the
(T r, 1 2 processes that happen at a nuclear power plant, at least now
%d 3 we are thinking about it in something more than a 4 one-dimensional or three-dimensional form. We are actually 5 thinking about hierarchies where, you know, one level may be 6 more important than another. So, to me, that is something 7 that is new and we need to see, you know, how we can apply 8 that more effectively.
9 So-you can be prescriptive at one level and it may 10 be quite appropriate, but another level it may not be as 11 appropriate. And having a way to select the right level of 12 prescription for a given level is part of the question. 13 DR. FONTANA: Part of the problem is that the NRC, 14 the regulator, does not run the plant. The licensee runs 7N 15 the plant. And the question is, at what level does the (~s) I 16 regulator regulate and monitor things so that any mistakes 17 that the licensee may make are not -- are at least one step 18 removed to an unacceptable event. 19 MR. KADAMBI: Maybe that leads to the second 20 bullet. I believe what that is telling us is that the 21 traditional approach, you know, has been successful, that it 22 has insured no undue risk to public health and safety in the 23 use of nuclear materials. 24 DR. KRESS: And you know that how? 25 MR. KADAMBI: Well, I am taking the message from ("N 1 ANN RILEY & ASSOCIATES, LTD. ( )' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 l
79 l JL' the white paper. Again, I -- 2 DR. . WALLISi What is your performance measure of 3 no undue risk? I mean this-whole thing is about performance 4 indicators, so'how do you measure this no undue risk? 5 MR. JKADAMBI : Well, I can read you the specific 1 6 words out of-the white paper. 7- DR. WALLIS: No, that is by decree, that is by J 8- assertion, that is not by measure. 9 MR. ROSSI: Well, let me just say a couple of j 10 things about the word undue risk. There is another term ! i 11 that is frequently used, which is adequate protection. And 12 I think we all understand that that term is not as precisely 13 defined as perhaps we would like, and we may never be able 14 to define it precisely, but it is my understanding that the () 15 16 Office of Research is going to try to work on that to get a better definition of it. So I am not sure that we can do 17 much in terms of trying to discuss it today.
-18 DR. WALLIS: But one of your specifications for 19 any new system or framework presumably is that whatever the 20 measure of no undue risk to public health is, it is 21 maintained.
22 MR. ROSSI: Yes. 23 DR. WALLIS: I don't see that here. I guess maybe 24 it-is somewhere here. 25 MR. ROSSI: I would say that that would be -- (3-(_) ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
80 1 DR. WALLIS: That would seem to be the upfront -- (^N 2 MR. ROSSI: Probably a working definition of what 3 we might want to do is we might want to make things less 4 prescriptive and more performance-based without lowering the 5 level of safety. Now, I don't know whether we can measure 6 that or not. We probably could. 7 DR. WALLIS: I think you need a measure of safety 8 if you are going to do that. 9 MR. ROSSI: We could measure a delta I think. 10 DR. KRESS: I think the answer to the question we 11 asked is it is an after the fact determination by the use of 12 the IPEs and PRAs, which are incomplete. You know, we don't { I 13 have a full answer to that question, but the indications are 1 14 that it is true. And, you know, I think you can proceed on j [/)
\_
15 the basis that that is a true statement, even though we 16 really don't have a full -- what you need is a complete PRA 17 for every plant, and compare it with, say, the safety goals 18 for each plant and then you could say, yeah, that is true. 19 And I think the indications are that it is true, but I don't 20 think we really know that. 21 DR. WALLIS: Maybe the problem is in the word 22 risk. I mean it is ensure no undue risk. That means the 23 public was never at risk. What you probably may mean is 24 that history has shown that there has been no undue harm 25 and, actually, in terms of real measure, not probabilistic (~] ( ,/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
c 81' 1 type risk, but in terms of real measure. Is that what you
,L ') 2 mean?
.3- MR. KADAMBI: 'I am not going to attempt to explain 4 what:the Commissi~on meant when using those words.
15 DR. WALLIS: It is a bit like the nuclear war 6 thing. .I'mean we haven't had a nuclear war so everything 7- has beenufine. But at times we have been at pretty 8 untolerable risk maybe. So it is really a question of what 9 your measure is. 10 DR. KRESS: I think they actually meant risk.
.11 DR. WALLIS: They meant risk rather.than --
12 DR. KRESS: -And I actually think they meant risk 13 in terms of the safety goals. 14- DR. WALLIS: So you didn't mean actual -- that () 15 people have not been harmed. You meant -- j 16 DR. KRESS: Well, I am interpreting, too. But I 17 think they meant risk, and risk in terms of the safety 18 goals.. 19 DR. FONTANA: Well, you know, going back to the 20 history of the early days, I am not sure they did. I think 21 .they didn't want that -- 22 DR. KRESS: There might have been a time when they 23 might have, but I think at the current level of regulations 24 and oversight, you and probably say this with some assurance 25- , as a true statement, based on what you see in the IPEs. ANN RILEY & ASSOCIATES, LTD. O, Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 82 1 DR. WALLIS: You mean that Three Mile Island was i 2 [O l not an event that posed risk to public safety? ' 3 DR. KRESS: Oh, no , I didn't mean that. 4 DR. WALLIS: Well, that is tradition. I mean 5 tradition goes back to '79. l 6 DR. KRESS: I said currently, at the present time, 7 based on the current regulations and the current oversight. 8 And in my mind, the only real measure of that we have is the 9 IPEs. 10 MR. ROSSI: Well, you have the accident subsequent 11 precursor program I think, too. 12 DR. KRESS: Well, that is another way to look at 13 it, too. 14 MR. ROSSI: And the trending, yeah. () 15 DR. KRESS: Yes. 16 MR. ROSSI: Because they tell you how close we 17 came en individual events and how many of those there were. 18 DR. KRESS: So I should not have left that out, 19 because I think that is a very important part of it. 20 DR. FONTANA: Well, that is going back kind of 21 after the fact, after the plants were designed pretty much, 22 and modified as such. 23 Originally, I am almost sure that they basically 24 used the defense-in-depth approach more from the point of
.25 view of the multiple barriers, then it kind of built up as
. ('N ANN RILEY & ASSOCIATES, LTD.
\s,/ Cou.t Reporters 1025 Connecticut Avenue, NW, Suite 1014 Wa ehir.o ;on , D.C. 20036
. 202) 842-0034
83 1 1 it went along. The question, I think you are really getting 2- to the third bullet, the question isn't so much how
}
3' . performance-based approaches affect defense-in-depth, I ' 4 think much more of a question is, how much of a margin do 5 you have left? That is hard to measure and hard to 6 determine. -I think you can almost always show you have got 7 some defense-in-depth'somewhere. 8 MR. KADAMBI: Well, if I can stay.with the second 9- bullet, really, the implication of the first sentence is, 10 why do anything at all? Aad that is I think addressed in 11 the second bullet. That is the efficiency, consistency and 12 coherence of the regulatory framework can be improved using 13 risk-informed and/or performance-based approaches. I 11 4 DR. WALLIS: That is a statement of faith? That () 15 is a statement of faith. 16 DR. KRESS: Well, I think you will find that 17 statement in a couple of ACRS letters also. 18 DR. WALLIS: That is a statement of faith -- 19 DR. KRESS: Of course. l 20 DR. WALLIS: -- until somewhere has a calculation i 21 or an experience that shows that, yes, indeed, we 22 implemented something and the efficiency increased. It is a 23 statement of faith. 24- DR. KRESS: It is a statement of faith, I agree 25 with you. And I wouldn't even disagree that the word , O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
U 1 84 l I 1 performance-based adds.to the coherence, it may even detract
~ ; 2 from it But I certainly agree with the statement in terms 3 of risk-based, risk-informed.
4 DR. WALLIS: You see, the thing is this seems to l 5 be backwards, cart before the horse. Someone has decreed we t 6 should-do something and then it said it is going to have 7 these benefits. The way.that I have been taught to problem 8 solve is you first identify'the problem. The agency is 9 inefficient. Let's have a measure of inefficiency. Let's 10 figure out to fix it. 11 You know, this --.your exercise seems to be in the
.12 other direction. Let's decree we will do something, and 1 1
13 then decree it is going to fix efficiency, and let's keep l 14 working, and then eventually, after 20 years, we might have (( ) 15 a proof that efficiency has gone up. 16 DR. FONTANA: Well, you know, deficiency being ; 17 defined as the desired output as compared to the amount of 18 effort that it takes to do that. The question is, is the 19 existing safety of reactors adequate? And I think we would 20 say it would. 21 So what you are getting at in terms of efficiency 22 is reducing the cost on both the part of regulator and the 23 part of the licensee, I think. 24 MR. ROSENTHAL: There is two contemporaneous 25 examples that have been discussed with the ACRS in other i ("] N,,e ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r l 85 1 formats. One is risk-based ISI and the other is
,m l t i 2 risk-informed -- risk-informed ISI, risk-informed IST. In V
l 3 both cases we are saying, let's take ISI, we are saying we 4 know that we can do a more efficient job and increase safety 5 also by inspecting the right piping, dropping the 6 -inspections on the stuff that is silly, and you will have 7 greater efficiency for the regulated industry. 8 DR. KRESS: There is certainly evidence of that. 9 MR. ROSENTHAL: That is a concrete example that I 10 think has been fleshed out. The other contemporaneous one, 11 I would argue, is risk-informed IST, where we can see where 12 we would have potential safety improvements and improved 13 efficiency. Now, those are both risk-informed. I don't 14 have a good performance-based example for you. j t (rw) 15 DR. KRESS: Well, I think there is certainly 16 evidence out there to support that bullet. It is not wholly 17 faith-based. And not only that, I think we are being told ! 18 that the Commission has decided that these are good 19 objectives and that this is a way to achieve them, you know. 20 So, we kind of act on that some way anyway. 21 DR. FONTANA: Well, the thing about efficiency, 22 and I take this to mean lower cost, if safety is the same -- 23 of course, there could be situations where applying your 1 24 effort on more important things could actually increase j i 25 . safety, but let's assume that you are trying to keep it the I I l I i l ("] (,,/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D,C. 20036 (202) 842-0034 l
r 86 1 same and reduce the cost, the -- I had completed the thought 2 .that I was getting to, I.will get back to you. .I found a 3 flaw in my train of' thought. 4 MR. .KADAMBI: It is not yet clear how 5 ' performance-based approaches may affect defense-in-depth. 6 That'I think is an important question.we will have to 7 address and keep at the back of our minds as we -- or even 8 in. the front of our minds as we cp> through some of these 9 things. But I feel I can state, based on the work that we 10 .have done so far, and what I have seen, that indicates that 11 defense-in-depth can in factLbe strengthened if it is done 12 right. 13 ~ DR. KRESS: I think you need a pretty firm 14 definition of what you mean by defense-in-depth in order to () 15 address it properly when you go about changing the 16 regulations, and I am not sure we have one. There is one in 17 .the 1.174 which comes close to being a good definition, but 18 I think it is still incomplete. 19 The definition of defense-in-depth has to some way
'20 involve the uncertainties in your measures of performance, !
21 and a defense-in-depth need should somehow be related to how
.22 uncertain your performance measures are. So somehow you 23 have to connect the two. I think when you come down to 24- ; acceptance criteria on your performance indicators, and when 25 you come down to this level at which you don't want to have ANN RILEY & ASSOCIATES, LTD.
O Court Reporters
'1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
E 87 l' margin to unacceptable consequences, I think ( defense-in-depth enters into the uncertainty you have in 3- both of those determinations. And somehow you have to make 4' that' connection.
'5. MR. KADAMBI: The other thought ~that I had was
! 6 that.these, the attributes that are.for performance-based 71 regulatory approaches that are given in the white paper 8 really provide the basic elements of a screening test of 9 'some sort, which, you know, we are tasked to develop as part 10 of our plans. lSo I think, you'know, the white. paper is 11 again serving a useful' purpose, I believe. I would like to 12 know if people disagree with that. 13 DR. KRESS: Well, I think the sense of this group 14 is that, as.I have heard it, these are good descriptive () 15' attributes that can almost be applied to anything. I. don't 16 .see how you can go from there to a real screen. Something 17 different, I think it has sort of a screen. Certainly, they 18 are_necessary attributes, but not sufficient. We will need 19 something more tolgo with it. 20 MR. KADAMBI: Okay. 21 DR. FONTANA: What I was trying to get at before, 22 is there really an additional cost here? Because while you
-23. are' implementing these new approaches, you still have to 24 maintain the old system. So, does the regulator and the 25 licensee really have to do both in parallel at the same L ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025-Connecticut Avenue, NW, Suite 1014 l Washington, D.C. 20036 l (202) 842-0034 j
88
- 1. time, and, therefore, during the transitional period, the
() 2 ' thing is costlier than it would have been before? How long 3 is this going to last? 4 MR. KADAMBI: I would say those are the kinds of 5 ~1essons we would learn from the maintenance rules. 6 DR. FONTANA: What have you learned so far? 7 MR. KADAMBI: I haven't looked enough. Ernie may ! 8 know more about it. 9 MR. ROSSI: Well,.we were told last week by NRR in 10 the public meeting we had that there was a fairly sizable 11 cost upfront in the maintenance rule. And that could be the 12- case in anything we do in this area. However, I think that 13 what we would do-is proceed with specific examples and 14 pilots and see what the costs are. () 15 I think if the costs turn out to be more to do it 16 'than we gain back,_and it is only a question of being less 17 prescriptive and not a question of safety, then we will' 18 proceed differently than if there was a big reduction in 19 cost. But that is something we have to take into account, 20 no question about it. 21 DR. WALLIS: Could I go back to what Ernie said I 22 think a long time ago, that there had been very little input i 23 from industry on this? And it seems this is being done by j 24 decree from the Commission, but the people who really stand 25 to benefit, surely, are the utilities and the industry. Why l 3 ANN RILEY & ASSOCIATES, LTD. s_/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 l 89 : l l 1 aren't they knocking on the door and saying get on with it? , l [%_]' 2 MR. ROSSI: Well, I think they are knocking on the l 3 door and saying get on with risk-informing Part 50, and get 4 on with some other things. And the situation may be that 5 this is at this point in time just lower on their priority. l 6 That is what I suspect, because there are lots of things 7 going on. 8 And, again, I have said this before, but I will 9 say it again, there are things being done on 10 performance-based approaches, because there is the reactor 11 oversight and performance assessment process that is being 12 revamped, as well as the inspection program. So those are 13 major areas where performance-based approaches are being 14 used, and risk-informed approaches are being used, and they (- ( ,j 15 have a significant effect on the industry. 16 DR. SEALE: And the industry has expressed 17 significant -- 18 MR. ROSSI: Interest in those. 19 DR. SEALE: Interest and displeasure with the past 20 situation where they have been micromanaged they feel. 21 MR. ROSSI: Right. 22 DR. WALLIS: Yes, but as soon as you suggest 23 another scheme, there seems to be a great deal of reluctance 24 to go ahead and let you implement it. 25 MR. ROSSI: Well, this isn't intended to be [~h ANN RILEY & ASSOCIATES, LTD.
'\m / Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
rr 1 i
)
\
90 1 another scheme, this is intended to a furtherance of what is (. 2 t
%.))
already going on, in my opinion. I mean it has to be made j 3 coherent and carried out in a coherent way with the I 4 risk-informed approaches. 5 DR. KRESS: And I think NEI has waded into this, { 6 so they are interested in it. 7 DR. SEALE: It is like cod liver oil, it tastes 8 bad when you take it, but the long-term effect is j 9 appropriate. l 10 DR. KRESS: What is cod liver oil? 11 DR. SEALE: That is back in the olden days. 12 DR. KRESS: I know what it is. 1 13 MR. KADAMBI: Okay. Let's see, the next bullet 14 addresses the third attribute in the list of attributes in () 15 the white paper for performance-based approaches. The 16 specific words the Commission has used are, " Licensees have 17 flexibility to determine how to meet the established 18 performance criteria in ways that wi]l encourage and reward 19 improved outcomes." You know, I guess I am just not sure 20 exactly how to use the word, or what meaning to attach to 21 the word " reward." This is going to be something that we 22 will have to figure out. 23 DR. KRESS: Less oversight is the reward. 24 MR. KADAMBI: The next bullet addresses attribute 25 4 and, in my mind, looking at that, it seems to be -- the
/~T ANN RILEY & ASSOCIATES, LTD. \s,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
?? l 91 1 'best definition that I have seen for getting at this concept N~ b- 2 of margin, margin to' unacceptable consequences, which can, l i U 3 you'know, be margin in physical parameters, temperature l 4 pressure, et cetera, but, you know, in my mind it could also JS be time. :You know, it could be if there is the right margin 1 6 built in there. It could be time for operator action. l 7 DR. KRESS: I think you need a coherent principle 8 on that; margin. I think that is going to be the key element 9 in performance-based regulation. It is going to be the 10- hardest thing to determine, how much margin you have to 11 what. So you have to define what an unacceptable l 12 consequence is first. That is George's saying, in your l 13 principles, you ought to put what your objectives are. And l
'14 your objectives are to do this, this and this. It might
() 15 very well'be CDF, but it also could be'other things. 16 And then, once you establish those, you have to l 17 determine what are acceptable levels of those that you never
- i 1
18 want to exceed. And then the margin is going to be hard to 19 determine'between there and where you put your performance 20- indicator. And you will need some sort of guiding principle 21 there, and that is the place where you really need it. And 22 that margin, as I said before, has to be related to your 23- -uncertainty in determining the relationship between this 24 performance indicator and this objective, what you are 25 trying to achieve, and it has to be related to that
/"'\ ' ANN RILEY & ASSOCIATES, LTD.
(s,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 i
r 92 1 somewhere. And that is, to me, the trickiest, hardest
). 2 technical issue you'need to face right now. And you need a 3- coherent principle ~on that. That is where I would really 4 put some thought into it.
5 I like, for example, the Scientech example of i 6, conditional CDF. .I mean that is a reasonable one. It might 7- be conditional.something else, if you have other objectives, q 8 But you have to say what is the uncertainty in that measure I 9 of conditional CDF, and your margin has to be related to 10 that uncertainty. 11 DR. WALLIS: Well, the biggest uncertainty is 12 probably the human performance. That is the one that is 13 toughest to estimate, even without uncertainty. 14 'DR. KRESS: Oh, I agree. But you have to figure () 15 out how to deal with it some more. And that, in a sense, is 16 what I would be calling part of your defense-in-depth. 17 MR. KADAMBI: Yes, I understand what you are 18 saying, and I also agree, it is one of the more interesting 19 issues coming out of this. 20 DR. WALLIS: I think most defense-in-depth is 21 actually put in because humans can screw up. If you think 22 about why you have air bags and safety belts and so on, 23- .although a car is a very steerable, controllable device, if 24 used properly. It is because of human error that you put 25 these defense-in-depth items in there. ANN RILEY & ASSOCIATES, LTD. O' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
93 1 MR. KADAMBI: Okay. The last one is the white e s, ( 2 paper clearly says that the licensees must continue to V; 3 comply with regulatory requirements. But, of course, the l 4 requirements may change as a result of performance-based 5 approaches. 6 DR. KRESS: That almost doesn't need to be said. l l 7 I mean you are always going to have that as a principle of 8 regulation. 9 DR. WALLIS: That is a big assumption. 10 PR. KRESS: Well, you know, that is why you 11 inspect and audit and do things. 12 MR. KADAMBI: Let's see, I think at this_ point I 13 can get down to earth a little bit and talk about, you know, 14 what we have been doing in the Office of Research. I
/~h i
() 15 mentioned that SECY paper 98-218, you know, I guess the SRM ' 16 on that came.out January or so of '97, and we got started I 17 thinking about the subject. And in June of 1997, the Office ' 18 of Research initiated the research project that resulted in l 19 the NUREG CR-5392. 20 It was actually done with a very small amount of 21 money which would have evaporated if we had not come up with 22 some idea to use it. 23 The intent of the research project was really 24 rather simple, you know, to do a literature search and use 25 what comes up in some kind of case studies. And we focused (~N ANN RILEY & ASSOCIATES, LTD.
'sm , Court . Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 1
94 1 on the Commission's direction on performance-based (~xi t 2 objectives not amenable to PRA. Bob Youngblood was the
'V 3 principle investigator on that project. And the work was 4 actually completed in April of 1998, and this I will point 5 out is well before some of the work that is now going on on 6 the revisions to the reactor regulatory oversight process. 1 I
7 But the final report on it, the printed version, 8 didn't get issues till January 1999, mostly because of -- 9 there is a new publication process that tries to use newer 10 technology and we sort of -- this was the first report that 11 went through this new process and it got caught at many 12 points along the way. l 13 DR. KRESS: Do you mean just printing it and l 14 putting the document together? (O) 15 DR. MILLER: It was in Office '97 or Office '98, 16 or something crazy like that. l 17 MR. KADAMBI: Well, we had to put it on CD-ROM 18 and, you know, lots of things that were associated with j 19 that. ;
)
20 DR. MILLER: New technology slowed down the work. 21 MR. KADAMBI: Since we had it in electronic form, 22 we also put it up on the Internet on a conference site, and 23 this way we are hoping to get involvement of the technical 24 community in furthering this discucsion, and hopefully get 25 stakeholders involved who don't normally come to stakeholder [') ANN RILEY & ASSOCIATES, LTD.
's s' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 s
m 7 95 If meetings and things like that. We have got about four
) ?2 comments.up'till now, not very many. But, you know, it is 3 interesting.
- 4 DR. WALLIS: 'Well, let me hope that waiting for-
-5 other peonle's ideas doesn't prevent you from having your 76 own.
7 DR. SEALE: There is:another thing here.
,8 DR. WALLIS: No, seriously. I think if you lead
- 9. the'way'and say,.look, these are the things we are thinking 10 .about, and ask for reactions from stakeholders, that is a 11 good way to proceed, rather than waiting. I am sure you are 12- not just waiting.
13 MR. KADAMBI: We are not waiting. I mean this is 14- something that is out there and I monitor the site, you
-15 know, once a day and see if somebody has weighed in with_-- 1 16' .DR. WALLIS: George is gone, but George got all 1.7 sort of involved in thinking about ways to think about this.
18 I would think that you must have done this sort of thing, 19 you could almost give an exposition of the way you have-
.20 thought and analyzed the' situation. That is what I would 21 like to hear, too, really, rather than waiting for some 22 stakeholder to -- because they always think about the 23 difficulties and the hazards.
24 MR. KADAMBI: Yeah, maybe after I get the plan 25 done, you know, then we can -- T' ' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 1
96 1 DR. SEALE: Yes. The other thing that would be [} v 2 interesting is if there are any comments specifically 3 related to DSI-13. Now, I know when you did them, you went 4 out and got public comment on the' direction setting issues. 5 But now here is an application and it is in the context of a 6 specific situation where it would be interesting to see if 7 there is any significant difference in the response to the I 8 idea in DSI-13 in detail, as opposed to in principle. 1 9 MR. KADAMBI: Yes, I think that is something that ' 10 we could do and it is really included in our plans because 11 the Commission very specifically mentioned DSI-13 i l 12 stakeholders. l 13 DR. SEALE: That's right. And I think they need 1 14 to get feedback on that particular issue. 1 C) (_j 15 MR. KADAMBI: Yes. We will keep that in mind and 16 we did keep that in mind when a stakeholder meeting was held 17 on September 1st, 1998 in Chicago on the role of industry 18 and, you know, performance-based approaches was one of the 19 agenda items, but we didn't hear almost anything at all on 20 that. But this is basically the background on the work that 21 went into the NUREG CR-5392, and at this point, I guess I 22 would like to ask Bob Youngblood to go into that if there 23 are no -- 24 DR. KRESS: Yes. Let me intercede here just a 25 minute. I hate to do real damage to our agenda. George ['N ANN RILEY & ASSOCIATES, LTD. (-) ' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
m 97 1- Apostolakis, in particular, wanted to be here during this () 2
- - 3
> presentation of Dr. Youngblood's. He , unfortunately, is
' detained-in a meeting with one of the Commissioners right 4 now.
5 DR. SEALE: They are just back. 6 DR. KRESS: Wonderful. That takes care of my 7 problem. I was going to suggest a real -- 8 lMR. BARTON: But we all got fired, so we can't 9 comment on your -- 10- 'DR. KRESS: Is that good or bad? 11 DR. APOSTOLAKIS: Everything is fine., 12 DR. KRESS: George, we have just now arrived at 13 the point where we are going to hear Dr. Youngblood's 14 presentation on the Scientech study.
/
(%) 15 DR. APOSTOLAKIS: Okay. 16 DR. KRESS: So we will proceed with it then. 17 MR. KADAMBI: Thank you, Mr. Chairman. I guess I 18 will request Bob to take over now. 19 DR. APOSTOLAKIS: Do you plan to go over each one 20 of these viewgraphs, Bob? It is going to take forever. 21 MR. YOUNGBLOOD: That is up to you. I have 22 brought.more than I strictly need to show, and I will try to 23 go as fast as I can. 24 Could I ask what time we are shooting for a break? 25 DR. KRESS: Well, 1 think two members of the
; ANN RILEY & ASSOCIATES, LTD.
A Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 i
98 1 ' subcommittee, or.three at least, have to get out of here by j 2 1:00 to 1:30. So I don't know. I can certainly abbreviate 3 the lunch period, cut into it, make it very short. 4 DR. APOSTOLAKIS: Twenty minutes perhaps. 5 DR. KRESS: Yes,_ even 20 minutes. 6- DR. APOSTOLAKIS: Is that okay with you? 7 MR..YOUNGBLOODA You are suggesting I shoot for 20 8 minutes? 9 DR. KRESS: Oh, no, no, no. ' 10 DR. APOSTOLAKIS: Can you? 11 MR. YOUNGBLOOD: I can shoot. 12 DR. KRESS: I am suggesting that we shoot for 13 about 1:00 to 1:15 as an ending time. 14 DR.' MILLER: With a 20 minute lunch. () 15 16 DR. KRESS: With a 20 minute lunch break. And so if you can do the subtraction. 17 MR. YOUNGBLOOD: Okay. Yes. It is not clear to 18 me.that all time is mine'necessarily. 19 MR. KADAMBI: No, I do need some time to focus on 20 the SRM for the SECY_ paper and our ideas that we now have 21- for the plan that we have_to submit, develop and submit.
' 2:2 DR. WALLIS: 'What I would like to go away from 23 this meeting with is the hope, or the impression that 24 someone has some ideas which are workable and is likely to 25 contribute something.towards a solution. If you could show ANN RILEY & ASSOCIATES, LTD.
C-) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
99 1- us why what you have done is workable and contributes to a () 2 solution of the problems, then that would be great. I don't 3 want.to get involved in the details so that message gets 4 lost. 5 DR.-KRESS: How many members will disappear around 6 1:00? 7 DR. APOSTOLAKIS: Four, they are losing quorum. 8 DR. KRESS: We are. losing. Well, we only have to 9 .have.two people for quorum. 10 DR. APOSTOLAKIS: But it is not fun anymore. 11 DR. KRESS: Well, we lose a lot of -- 11 2 DR. APOSTOLAKIS: Why don't we let Bob start and 13' we tell him which viewgraphs he can skip. 14 MR. YOUNGBLOOD: Okay. I am Bob Youngblood, one () 15 of the authors of this report. I would like to mention a 16 couple of other names, in particular, because I will be 17 . talking about things that they in particular brought to this 18 report. Bob Schmidt did the piece of modeling that has been 19 mentioned several times in connection with the heat removal 20: at shutdown, and Niall Hunt is the guy that introduced the 21 rest of us to the diamond tree idea. 22 And in case I forget to say that later, I would 23 like to point out that when he did.that, he was working on 24- plant availability issues as a plant guy. He was at 25 Baltimore-Gas & Electric at the time, and they saw this as a 4"' ANN RILEY & ASSOCIATES, LTD. k,,,) . Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
e 100
'l tool to_ promote plant availability. And as you may have
() 2' seen in the report, you'can go down into great detail of 3 operational stuff. People either can or do track a lot of 4 things, and it was that~ background that he brought into this 5- issue. 6 If I were issuing this report today, I think I 7 would choose a word different from " oversight." Oversight
.8 to some people means inspection and enforcement and, as
- 9. several of you have pointed out, that_is not what this has 10 been about. This is'about changing requirements.
11 DR. APOSTOLAKIS: Is that the same as changing the 12 licensing basis? 13 MR. YOUNGBLOOD: I think it would have -- in my 14- mind, yes, I don't know, you could chop logic on that, but, 15 yes. 16 DR. APOSTOLAKIS: So I would have to apply 1.174? 17 MR. YOUNGBLOOD: Well, no, you would go -- you 18 would change requirements, and then it would entail a change 19- in licensing basis. But you wouldn't be leaving the rules 20 alone and changing the licensing basis the way 1.174 does. 21; You might'just be wiping out whole categories of i 22 requirements. 1 23' DR. KRESS: The licensing basis would probably 24 still remain.the FSAR, and I don't think that would likely ; 25 . change. It might. I ANN RILEY & ASSOCIATES, LTD. ) s Court Reporters i 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 _(202) 842-0034
I
)
101 1 MR. YOUNGBLOOD: I think you could imagine tech i ( ) 2 spec changing as a result of this.. 3 DR. KRESS: Tech spec changes, definitely, yes. j 4 MR. YOUNGBLOOD: So this report is really about 5 two topics. One, the much discussed attribute, the fourth l 6 attribute where your monitoring criterion shouldn't become i i 7 -- shouldn't be set so high that, by the time you trip it, { 8 you have got a real problem. And I will talk a little bit 9 about the example that we'did to shed light on that. 10 The other area was how to do better at considering 11 areas in which PRA does not appear to do a good job, and 4 12 that is sort of a clumsy formulation of it. But the general l 13 idea is that -- the way I would explain it to myself is I 14 that, when I read reports discussing significant events, and l () 15 as a PRA guy, ask myself whether work that I had done or 16 even seen really captured what was going on in that event, 17 then maybe the' answer is no or a partial no or something. 18 So there is a lot going on in event reports or even some 19 information notices and that kind of thing that you don't 20 really see well done in PRA, and that was issue two, and 21 that is the issue that drove in part -- one of the two 22 reasons I think that the diamond tree came out strongly. 23- DR. APOSTOLAKIS: It seems to me you have to be 24 careful with the language here, Bob. 25 MR. YOUNGBLOOD: Yeah, I probably should. [~h ANN RILEY & ASSOCIATES, LTD. N/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
102 1 DR. APOSTOLAKIS: In your report, page 3, there is () 2 a sentence, "It'is well known that PRAs do not typically 3 model everything that is important to safety." Do you 4 really believe that?
- 5. MR. YOUNGBLOOD: Oh, yeah. But that is actually 6- --
that sentence is actually meant to address a different . '7 issue. 'There is a lot of components that -- that sentence, 8 when written, was meant to' refer to a lot of components that 9 are not -- that don't have basic events in the fault tree, 10 is what that sentence meant. 11 DR. APOSTOLAKIS: But why? I mean if they are 12 important to safety, why aren't they there? I don't 13 understand it. Because there was a meeting, in fact, the 14 workshop that you had on April whatever -- 14, where Mr. () 15 Lochbaum I'think said that PRA is fiction. 16 MR. YOUNGBLOOD: Well, a lot of things were'said 17 that day. 18 DR. APOSTOLAKIS: Are you supporting that 19 statement? 20 MR. YOUNGBLOOD: No. No, I think it is a
-21 different thing. Well, just compare how many things are 22 there on a Q list and how many things are there in a PRA?
23 The number is different. That is all I really meant there. 24 DR. MILLER: It seems like Lochbaum is certainly 25 rather dramatic in his statement, but it seems like -- I ANN RILEY & ASSOCIATES, LTD. O'- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
~- -
103 1 agree with George that his statement and this one are kind s
\ 2 of consistent. Lochbaum is saying it is fiction because we
[\_) l 3 believe that everything in a PRA is safety-related, and you 4 are saying the same thing. 5 DR. APOSTOLAKIS: And then Mr. Riccio, public 6 citizen, stated that PRAs do not reflect reality. And here 7 we have a NUREG that sort of confirms that. 8 MR. YOUNGBLOOD: Well, the sentence, I guess I 9 will take hits on the language of the sentence. But what I 10 meant by it I think is correct. There are instrumentation 11 systems that plant people care about that you don't see 12 modeled. There may be a basic event in a PRA that says 13 probably people will respond to this and do the right thing, 14 and that event may tacitly take credit for kinds of O g j 15 instrumentation that you don't then have a little tree 16 reflecting the possibility that all these instruments will 17 fail. 18 You don't call out every piping run in a typical 19 PRA. You can go back and do that if you want to, but the 20 thing just explodes. 21 DR. APOSTOLAKIS: No, but it seems to me that when 22 you do walkdowns and you do space-dependent, common cause 23 failure, potential common cause failure analyses, you worry l 24 about these things. 25 MR. YOUNGBLOOD: You worry about them, and then, [ ; ANN RILEY & ASSOCIATES, LTD.
\_/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 i l
l l
104 1 having worried about them, maybe you decide that you don't () 2
.3 need to put them in, either because they don't contribute significantly.to-the train you are modeling, and, in 4 -addition, they don't take down multiple trains when they go.
5 You don't see cables in most PRAs, but people care about 6- them. 7 .DR. APOSTOLAKIS: Fire. 8- MR. YOUNGBLOOD: Well, sure. 9 DR. 7"OSTOLAK1S: When you do fire PRA, you do. 10 MR. YOUNGBLOOD: Yes. Not cable by cable, not 11 tray by tray necessarily. 12 DR. APOSTOLAKIS: But I am not convinced that the 13 cable by cable, each little cable is important to safety. 14 MR. YOUNGBLOOD: No. No. () 15 16 DR. APOSTOLAKIS: See, that is the thing,.there is a systematic approach for identifying what is important to 17 safety. 18 MR. YOUNGBLOOD: Yes. 19 DR. MILLER: You raised an example of 20 instrumentation. What instrumentation that is not modeled 21 is important to safety? Just as an example. 22' MR. YOUNGBLOOD: Well, just as an example that 23 appears in the report, there is level instrumentation that 24 played a role in the event analyzed that -- well, of course, 25 you could say shutdown PRA is pretty general to begin with, - j ANN RILEY & ASSOCIATES, LTD. {s'7) _- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p 105 1 in some cases. I don't<know whether that instrumentation
.I~'
g) -- 2 would have appeared, and that strikes me as something, if
- 3. you care about level instrumentation in that operational 4 mode, you would highlight it as important. ~
I am not sure
- 5. that things-that.are like that are always in PRAs.
6 DR. KRESS: It may appear implicitly in how much 7 credit you give for operator action or something. 8 DR. APOSTOLAKIS: Right. 9 MR. YOUNGBLOOD: Yes. To go back to important to 10 safety for a moment, I don't -- this is probably the wrong day to get off onto that, but it seems to me that PRAs 12- actually don't systematically identify what is important to 13 safety, they are about identifying vulnerabilities.
.14 DR. APOSTOLAKIS: That is a pretty strong 15 statement. Typically, they don't do that. I mean then what 16 do they do? And what is safety? I mean, you know, 17 remember, people are conditioned by the regulations of 40 18 years to worry about a lot of things, and that is why we are 19 moving to risk-informed regulation, to make sure we worry 20 about what is important.
21- DR. MILLER: If this statement has some validity, 22 that would mean that -- 23 DR. APOSTOLAKIS: It is way too strong. 24 DR. MILLER: If I go to your level example, if my 25 level instrument fails, indeed is a safety-related ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 I (202) 842-0034
=- 106
-1 instrument', it causes my delta CDF to change by so much, but I
[ 2 it is not modeled,.I won't see it. 3 MR. FLACK: Yeah,- I think that -- this is John ! I 4 Flack. I think we agree. j i
;5 DR. MILLER: Am I going too far with this !
6 statement? MR. FLACK: Yeah,.I mean the staff generally 8 agrees with that, is,why we are moving along the 9 risk-informed regulatory process. When I see it is not-l 10 complete, I think of security, those kinds of-things that 11 are not explicitly modeled, because, at this point, that is i
-l 12 a limitation of the PRA. )
13- DR. APOSTOLAKIS: That is a difference statement. 14 MR. FLACK: This is different, and that is the way I O) y 15 I think we should take that statement at this point.
]
16 MR. YOUNGBLOOD: I, in fact, didn't necessarily l l 17 mean to be -- I think that PRA does what it does, but there i 18 is a lot of. things you care about that it doesn't explicitly 19 have, is all I meant. 20 DR. APOSTOLAKIS: But those things you care about 21_ are.not necessarily important to safety. This is way too 22 strong. It is well known -- not to me -- that PRAs do not 23- typically model everything that is important to safety. 24 Well, I take exception to that. Anyway. , 25 DR. KRESS: Everything is a pretty encompassing O, ANN RILEY & ASSOCIATES, LTD.
\s / Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
F 1 ) i I 107 1; word, George. EHe just: leaves.out one. thing-and that 2' - statement-is'truei. 3 DR. APOSTOLAKIS: No, I don't believe so. I don't
'4 believe so. I think you' worry about a lot of things-that
,5 are not necessarily important to safety, because the 6 regulations tell you.you should worry about them. But to 7 say that:the PRAs typically do not-model them, I would like 8 .to see specific. examples tofbe. convinced.
9: DR. MILLER: Is this meant to apply to non-plant 10- related issues, I mean issues like security and so forth? 11 DR.. APOSTOLAKIS: Well, if it is incomplete, it is 12 i~ncomplete. Yeah, but then there is a statement upfront 13 that this thing has not been modeled. j l 14 DR. MILLER: So, in other words, this statement () 15 may be valid, but all those things that aren't modeled are 16 stated pretty clear they.are not modeled. 17 DR. APOSTOLAKIS: Yes. You know what you have not
-18 modeled. As John said, you know, security issues and so on, 19 you have not modeled.
20 MR. YOUNGBLOOD: Well, when I put it in -- 21 DR. APOSTOLAKIS: Yeah, let's go on. 22 MR. YOUNGBLOOD: When I put it in, I was thinking 23 of a level of detail issue, and I am not sure it is on 24 today's agenda. It may be worth another discussion.
'25 DR. APOSTOLAKIS: My point is that if Mr. Lochbaum ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025. Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p 108 1 has said this thing before, that the reason why you cannot
'~ /)
V 2 have risk-informed regulation is PRAs, you know, he hasn't 3 put it as strongly as in the last workshop, that they are 4 fiction, but he said, you know, they are not realistic. 5 MR. BARTON: Well, he says fiction because of its 6 incompleteness. That was his basis for saying it was 7 fiction. And I would agree that a lot of PRAs are not, you 8 know, quote, complete. Everything is not modeled. 9 DR. APOSTOLAKIS: And that is why it is 10 risk-informed regulation. 11 MR. FLACK: That's right. That's exactly right. 12 DR. APOSTOLAKIS: But there is a big difference 13 that and saying that, typically, they do not model 14 everything that is important to safety. (O ) ( 15 DR. WALLIS: George, I think we have to move 16 along. 17 DR. APOSTOLAKIS: That is what I think, too. 18 MR. YOUNGBLOOD: Okay. Still on this slide, we 19 took as input to this report that -- essentially, that 20 risk-informing would be done. In other words, this report 21 was supposed to be about how to go to performance-based and 22 not about how to do risk-informed. So we didn't argue, 23 really, whether it should be risk-informed. We didn't 24 really argue whether performance-based was a good thing. 25 And we assumed that there would be what I called here policy
/\ ANN RILEY & ASSOCIATES, LTD. '( - Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
I l l 109 c 1 finputs, namely, somebody would decide, is CDF the thing.we l f 2 worry about? You know,.what do we mean by defense-in-depth ! 3 and how do we want to see it reflected? That sort of thing. 4 That is cne big category of input-to the process that we. 5 talked.about. T6 And another big category.of input is, how sure do-7, you want to.be that you-are getting satisfaction of those? 8' So those'two kinds of inputs, how much safety and what do i 1 9 you.mean'by it, and how sure do you want to be that you are I l 10 getting it. Those are sort.of dials on the box. { 11 DR. KRESS: Those are typically policy issues. 12 MR. YOUNGBLOOD: Yeah. 13 Finally, I would say at some point that the 14 process is intended to apply more broadly.than just to r) ( 15 16 reactors although today I think we will naturally gravitate toward reactors. 17 What I am going to try to do in this presentation 18 is talk a little bit about the two issues mentioned on the 19- preceding slide, go through steps to develop an alternative 20 set of regulations in a.way that tries to address those l 21' issues, and then throw out some ideas. 22 This one, Number 4, has been discussed several 23 times-.here today implicitly. .The word " performance" has 24 meant different things to-different people. I have begun to 25 try to use the word " outcome oriented" to describe something ANN RILEY & ASSOCIATES, LTD. O- Court Repcrters
.1025 Connecticut' Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r. L 110 1 isfmore performance-based if it is more pegged towards 2 . outcomes. The issue of whether your performance criterion 3 has.been set at too high a level I think is natural to 4 -discuss ~on this slide and on the left I am showing here ASP, 5 That stands for the Accident' Sequence Precursor Program. 6: It seems to me that many events that come out of 7 that program has having seemed significant are events that 8 sort of pop up at the' system level. A component is not 9 typically an accident sequence precursor but complete loss 10 of a system perhaps is and so earlier when you were talking 11 about tracking things at the system level, if ASP events are
-12 typically system failures and if system failures seem like 13 events with not enough margin left in them, then maybe train 14 would be a 1etter idea, and I must say I don't plan to get
~
15 {) into this level of detail in the presentation but I came l 16 away from this agreeing or believing as other people have 17 believed before in performance indicator programs that train 18 level indicators would make a lot of sense for many of these 19 things. 20 DR. APOSTOLAKIS: If I look at the second column 21 here, examples of performance measures, they are all really 22 PRA measures, aren't they? It is all frequencies,-and I 23 thought you" (mphasis was on non-PRA performance measures. 24' Can you have a non-PRA performance indicator that i 25 uses frequencies?
'% ' ANN RILEY & ASSOCIATES, LTD.
Court Reporters i 1025 Connecticut Avenue, NW, Suite 1014 '
. Washington, D.C. 20036 (202) 842-0034
i 111 1 MR. YOUNGBLOOD: Maybe I don't tend to think of 2- non-PRA examples. I guess I was trying to think of numbers. (} 3 DR. APOSTOLAKIS: So this is an overall approach. 4 It does not' emphasize not amenable to PRA issues that we 5 need to develop performance measures for? 6 DR. KRESS: I think he gets to that. 7 DR. APOSTOLAKIS: But here we don't. 8 MR. YOUNGBLOOD: Yes, you certainly don't see it 9 on this slide, and -- 10- DR. APOSTOLAKIS: I mean the NEI example again in 11 their appendix is not PRA-based. They don't use any 12 frequencies. They just say maintain the temperature below 13 158 degrees. 14 MR. YOUNGBLOOD: Well, there may be an element of () 15 how often did you trip that -- 16 DR. APOSTOLAKIS: But they don't say that. 17 MR. YOUNGBLOOD: Yes. 18 DR. APOSTOLAKIS: They say that every single time 19 you do it, the temperature has to be, you know, below this 20 level because then you have sufficient margin, so this 21 particular slide does not address that. This is really 22 PRA-driven. 23 DR. KRESS: Agreed -- except below the line there, 24 there are things -- 25 DR. APOSTOLAKIS: Frequency of loss of function? O
\/
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l Washington, D.C. 20036 (202) 842-0034
n 112 1 DR. KRESS: Well, that is the only one -- 2 DR. APOSTOLAKIS: -- rates -- 3 DR. KRESS: Maintenance effectiveness, maintenance 4 rule -- nothing under institutional factors. Those clearly 5 are not PRA. j
)
6 DR. WALLIS: Would you describe what this is J
- 7. supposed to show? The question is what level is appropriate
(
-8 so I am looking for which levels in this picture are j 9 appropriate.
10 MR. YOUNGBLOOD: Well, the process -- I don't 11 believe in a generic answer to that question. 12 DR.-WALLIS: How does the figure answer the , i 13 question? That is what I am trying to get at.
)
14 MR. YOUNGBLOOD: The figure really poses the () 15 question. It is where on here -- we will return to this 16 figure later -- 17 DR. WALLIS: Horizontally or vertically or what? 18 MR. YOUNGBLOOD: A higher level -- do you want to 19 monitor, for example, at the system level or at the function 20 level or at the train level or down here? 21 DR. WALLIS: Oh , it's vertical segregation you are 1 22 talking about here. l I 23 MR. YOUNGBLOOD: Yes. 24 DR. MILLER: Before we are done, we are going to 25 have a criteria or at least proposed criteria on where to
/"'s ANN RILEY & ASSOCIATES, LTD.
(,,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 l
1 113 ! 1 choose the appropriate level? (~') 2 DR. KRESS: The process, I think. kJ 3 MR. YOUNGBLOOD: Yes, the process, and it would 4 depend on if you had a CDF objective. Well, you'll see -- I 5 hope you will see. 6 DR. WALLIS: I don't see how you could possibly 7 put human actions down below. Humans are the most likely to 8 make errors. The functions are more likely to be performed l I 9- or the devices than by the people. 10 MR. YOUNGBLOOD: There are actually human actions 11 that belong at that level, because they are like components. 12 DR. WALLIS: Absolutely. 1 1 13 MR. YOUNGBLOOD: This thing is meant to be sort of 14 implementation of programs, so we don't -- if this work goes r'%)
%J 15 on and goes on in this vein and goes on using, trying to use 16 a hierarchy like this, we will get better at labelling those 17 tiers, but that tier called Human Action here is really 18 about things like performing maintenance.
19 DR. WALLIS: Putting a barrier across the middle, 20 it kind of implies that anything below that line is less 21 important? 22 MR. YOUNGBLOOD: Well, actually there are figures. 23 This figure is derived from figures that appear later, where 24 that barrier has some other significance as it would be 25 better for the plant if we didn't try to monitor below the ANN RILEY & ASSOCIATES, LTD. ()% (,- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
)
l i 1 114 ! 1 level of that barrier, but I just forgot to eliminate it
/~T 2 from this.
3 DR. WALLIS: That's Mario's Wasteland, is that -- l l 4 MR. YOUNGBLOOD: Yes, exactly. ) 5 DR. WALLIS: So human actions don't exist except 6 below the waist? 7 MR. YOUNGBLOOD: Belt, yes. 8 DR. KRESS: Let's keep it clean. 9 MR. YOUNGBLOOD: So in the example, NEI had l 10 proposed an example where basically an outcome would be 11 keeping temperature below some number, and it seemed natural 12 to model that to see what it would mean if you did try to 13 monitor that way. l 14 DR. KRESS: Actually, I thought that was very l
) 15 illustrative.
16 DR. APOSTOLAKIS: But again, I took the NEI 17 example and the Scientech analysis and again if you look at 18 the previous figure, the Scientech analysis says that the 19 .NEI approach or at least the example they gave deals with 20- -more or less normal operation, that they want to keep the 21- -temperature below 158 degrees and they did not consider the 22 possibility of having a rare occurrence like loss of 23 component cooling water, system or station blackout and so 24 on. 25 But in a performance-based system wouldn't you
- p. ANN RILEY & ASSOCIATES, LTD.
\ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
115 1 also have some performance criteria on these systems? So 2 maybe', you know, they should have' continued the example to 3 . consider these contributors and say now if I have station 4 blackout, I have to lose the diesels, and I have lost tne 5 grid ofLcourse. Now I will have information on the grid. I 6 can't do much about it, but for the diesels I can. 7 MR. YOUNGBLOOD: Yes. 8 DR. APOSTOLAKIS: So I will.have performance 9 criteria, so I will have my precursor. i 10 MR. YOUNGBLOOD: That's right.
'11 DR. APOSTOLAKIS: What they are doing is perhaps 12 incomplete in the sense that they did not finish the 13 example, but it is not inconsistent with what you did.
14 MR. YOUNGBLOOD: I think if we had applied our 15 process to that model of shutdown we would have come out [Vl 16 with the kind of thing that you are talking about. What I 17 carried away from the example is that pegging the criterion
-18 in temperature space is not a good idea, but putting 19 performance requirements at perhaps the train level on RHR l 20 and on the key supports is the kind of thing that I would 21 expect to come out of our process.
22 That is I think below the level that example 23 wanted to see them go. 24 DR. APOSTOLAKIS: Again it depends on how you 25 phrase it, because they can come back and say, well, if we O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
I 116 1 really wanted to implement this, you know, we would realize
,2 that it is impossible to keep the temperature below 158 3 degreesLunder.all' conditions so we will have to consider now 4 the conditions where.this might now happen, and that would 5 have led them'to the analysis you have done.
6 MR. YOUNGBLOOD: Fine. 7 IMt. KRESS: But the question is what does NRC now 8 go. inland look at-to assure themselves that the safety case
.9 is maintained?
10 DR. APOSTOLAKIS: Yes. 11 DR. KRESS: If they are only going to look at this 12- temperature, I don't think they are doing their job. 13 DR. APOSTOLAKIS: I am not sure they said only. 14 That is the thing anyway. () 15~ 16 MR. YOUNGBLOOD: Setting aside what NEI really i h
. meant, I thought that they were trying to reach for the 17 purest outcome based possible thing, where the outcome 18 really is keeping the temperature low.
19 DR. APOSTOLAKIS: They also were making the point, 20 I think, that was the_ main point of that appendix, that a
.21 performance criterion can be defined in the deterministic 22 space'. It doesn't have to be frequency driven. I think 23 that was the main message there, that by monitoring the 24 temperature you have a performance criterion that is, you 25 know, monitoring a physical variable rather than frequencies
/"Sg ANN RILEY & ASSOCIATES, LTD. (,s/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 i Washington, D.C. 20036 (202) 842-0034 J
117 1 1 and unavailabilities. 2 Anyway, I don't disagree with-what you have done. ({ 3 I'am just saying that I don't think that example was 4 intended to be a complete demonstration of how you would do 5 it, and what you have done is very. reasonable. 6 But again, though, Bob, everything that you are 7 doing here seems to be PRA-driven, isn't it? You are 8 talking about conditional probability of core damage given 9 heat-up. You give this example here. If I don't want to 10 use a PRA, what would I do? I would-just count D, P, F, G 11 and say, you know, I have so many boxes therefore I am in 12 good shape? 13 MR. YOUNGBLOOD: Well, this example is PRA-driven, 14 but I think if I can -- well, in reactor space where we are () 15 talking about safety functions, it is really hard for me not 16 to think in terms of PRA, but if we were talking about some 17- other kind of radiological facility where the descriptions 18 of scenarios that we were worried about would not be 19 embedded in what you would call a typical PRA, then -- 20 DR. APOSTOLAKIS: How would you analyze fitness 21 for duty using that? 22 DR. KRESS: Well, that is the whole point, I 23 think. 24 DR. APOSTOLAKIS: What is the whole point. 25 DR. KRESS: Well, you have to think in terms of ANN RILEY & ASSOCIATES, LTD. O' Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
k i 1 l t 118 I l 1 PRA in terms of your high. level objectives because they are J i
'2 the things that PRAs do, but the point is you get to a. point 3 where the PRA is not useful to you anymore. Then what do 4 -you do? And this is what they are leading up to.
5 DR. BONACA: One thing there is that NEI gives you 6 a scenario with some deterministic criteria. We are all 7 puzzled by'that. .Is it adequate?
'I 8 DR. KRESS: Where do the criteria come from? What )
9 does'it have to do with safety? 10 DR. BONACA: But since the criteria you making is 11 core damage, the only tool you have really to analyze that 12 is PRA, and that is really what they have done. 13 DR. APOSTOLAKIS: You are changing the scope,. 14 Mario. The Commission's -- l () 15 DR. BONACA: No, wait, wait, wait, wait. What it l 16 is doing is performing an analysis using PRA to determine 17 whether or not the cut-off point to monitor is adequate. 18 DR. KRESS: You just can't arbitrarily pull those 19 things out of the air. You have to have some reason for 20 having that. 21 DR. BONACA: And you can't use -- they have to use 22 PRA criteria to see if core damage, if your margin to core 23 damage here is an adequate measure, and really, I'll be 24 frank with you, I don't understand, I do not like the NEI 25 approach. I did not understand as well the weakness of it O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
119 1 until I saw a PRA based -- I'% 2 DR. KRESS: It's really pointing out the weakness l N--l l 3 and what you will find out now is your thinking will lead l l 4 you down to things that are important to core damage but you 5 don't know how to deal with them with a PRA, and then they 6 are going to ask the question well, what do we do about 7 these? I think that is going to be the important question 8 and they have some ideas on it but it's not complete. 9 They haven't given you a recipe yet. 10 DR. BONACA: I would add that PRA will be 11 fundamental in my book in drawing the line through the 12 diamond. 13 DR. KRESS: Yes, you use the PRA wherever you can 14 in this process. You'll find out you can't use it for all (.n) us 15 things. 16 DR. SEALE: It doesn't answer all the questions. 17 DR. KRESS: Yes. 18 DR. APOSTOLAKIS: I don't disagree with what you 9 said, but first of all, there is an assumption here that 20 what is going to happen is an initiating event, so if I 21 want, if I am monitoring -- again, my cornerstone is to make 22 sure that the mitigating systems have an unavailability that 23 is acceptable, so a mitigating system now is Box D, okay? 24 It is Box D. Now the margin I have is all the 25 other boxes . In other words if that system is unavailable /'N ANN RILEY & ASSOCIATES, LTD. k,,) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
120 l 1- in order to go to core damage I have to have an initiating f~\ 2 event occurring. I'have to have E, F, G occurring and all
\_ / - 1 3 that stuff, so it is not just a one-way street, right?
l 4 I can take an accident sequence, monitor a few 5 things, and then take one thing out and everything else is a l 6 conditional probability of core damage. j i 7 DR. KRESS: I don't get your point. l 8 DR. APOSTOLAKIS: That is it not just an 9 initiating event. I mean it is not just such a nice l 10 sequence all the time. 11 MR. YOUNGBLOOD: Oh -- no, it's not. 12' DR. APOSTOLAKIS: Okay, fine. I have no problem I 13 with'it. 14 MR. YOUNGBLOOD: Actually this slide is a good () 15 place to re-introduce the idea of the precursor event. I
-16 think what you were just describing basically is accident 17 sequence precursor analysis, but it came to seem to me to be 18 natural to think about, to always be thinking about the 19 precursor idea, that in some sense the conditions that we
'20 are monitoring are not precursor in the sense of the 21 program, which had a threshold and if you were worse than 22 that, then you were a precursor, but all of this is really 23 about monitoring precursor conditions in some sense, and as 24 you say, it doesn't always go like -- sorry?
25 DR. KRESS: That is an excellent way to look at p)- (,, ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
K 121 1 it.
/~'\ 2- MR. YOUNGBLOOD: And as you say, it doesn't always U
3- go'this way and'that is.why'we would be maybe monitoring 4 . rain level indicators out here and so forth. 5 DR. WALLIS: The whole subject is 6' performance-based, so does this figure help me to understand
'7 what you do in a_ performance-based world?
8 MR. YOUNGBLOOD: _This figure doesn't help you 9 much. -It is really keyed to -- you know, this axis up here 10 corresponds loosely to the vertical axis in the last figure i l 11 that we were talking about, and it is really keyed to that 12 ~ example where the NEI criterion corresponded essentially to l 1
-13 this and the point of this slide, which is to justify --
14 which is really about the numbers on the following slide is
-{(,/)
15 that sometimes the thing that took you to this point also 16 affected the mitigating systems. 17 DR. WALLIS: But really everything here has 18 something to do with performance. 19 MR. YOUNGBLOOD: Yes. 20 DR. WALLIS: If F goes down, that is not 21 negligible performance indicator. 22 MR. YOUNGBLOOD: Right. 23 DR. WALLIS: -- selection is still there -- 24 MR. YOUNGBLOOD: Yes. Yes, it is, and in fact 25 that is a comment I meant to remind myself to make.
/N ANN-RILEY & ASSOCIATES, LTD.
( ,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
122 1 DR. WALLIS: And humans could intervene at any
, 2 time presumably to screw up one of these by taking some
( 3- inappropriate actions. 4 MR. YOUNGBLOOD: Right. That's right, and the 5' . process that I will get to. eventually would lead you to be 6 interested inlall of those things and not just the things to 7 the left'. 8 While I'm still on this slide, different 9 scenarios, different cut sets, might have boxes E and F on 10 the left. 11 DR. WALLIS: Right. 12 MR. YOUNGBLOOD: That's a potential confusion 13 introduced by the slide, but ultimately if you had a 14 criterion where some boxes say -- if Box G is always on the () 15 16 right no matter in all the scenarios and where you've drawn the line, then it sort of follows to me from a figure like 17 this that there will be things that you won't even think 18 about doing in a performance-based way because there are 19 some things that.are just always going to be to the right of 20 any reasonable line that you draw, and so a point I meant to 21 make here is that.I can't imagine for -- where you have a 22 severe consequence that you're trying to prevent, I cannot 23 imagine a purely performance-based approach. 24 DR. WALLIS: Well, the margin is -- the margin 25 must have~some units of measurement presumably. 1 l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 I
123
- 1" MR. YOUNGBLOOD: Well, down here it was it .
2 probability. 3 DR. WALLIS: Oh, it's' probability? 4 bHl. YOUNGBLOOD: That's right. Down here it's 5 ' probability,.andLup there it was. sort of loosely probability 6 but correlated :with narrative things happening in the 7 scenario. 8 .DR. WALLIS: So it's not really margin. There's
'9 always'a finite' probability of core damage. It's how much 10 you've lost in CDF or something?
11 DR. BONACA: I think it's probability and 12 uncertainty, however. 13 MR. YOUNGBLOOD: Yes. 14 DR. BONACA: Well, because that was a /( - 15 consideration there, right? I~mean, you had no other boxes 16 that have to do with human recovery. 17 MR. YOUNGBLOOD: Yes, that's true. 18 Let's see if I can get quickly through this. This 19 is showing -- the column labeled here initiating event 20 frequency is really the frequency of getting the kind of 21- heatup that corresponds to that criterion, and that's based 22 on a little piece of logic that had boxes A, B, C in it 23' basically. So it's a core damage model with some attention 24 paid to actually quantifying an intermediate step in the 25 sequence. Otherwise it's a normal model, but we made it put ANN RILEY & ASSOCIATES, LTD. O. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l l 124 1 out both this frequency as a synthesized quantity and core () 32 3 Ldamage and then probability of core damage given that as synthesized quantities. 4- Actually all of_these numbers are -- all of these 5 conditional probabilities are high by the standards of the
'6 ' precursor program,.I-think. And in that sense it's, you 7 ~know, all of them sort of trip that flag. But within this 8 group of all.large numbers, some of them are still a lot 9 bigger than others, and that's just intended to make the 10 point that the character of the event, what actually 11 mattered is significant and not-just whether you got to that 12 temperature.
13 So to' summarize those points, a calculation was 14 done to evaluate, just show what it might mean to be using () 15 -heatup as an outcome-oriented performance. We did the 16 things in these bullets. We saw a big variation in l 17 conditional probability of core damage, and I think it would 18 be -- this is formally I think the way you address that 19 fourth attribute. 20 I mean, maybe it's a big -- you wouldn't go to 21 that level of detail all the time, but I think that's a 22 thought process, you know, what does your criterion mean and 23 are.there important areas of that criterion where you don't 24 really have the margin that you would like. 25 Okay. The other kind of question that --
- f. ANN RILEY & ASSOCIATES, LTD.
s Court Reporters 1025' Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
125 l 1 DR. KRESS: The margin that'you would'like, you O 2. . view that as a policy issue --- V 3 MR. YOUNGBLOOD: Yes.
'4 DR' KRESS:
That.the Commission or somebody would i
-5 just.have to set based on their comfort level.
6- MR. YOUNGBLOOD: I'm not sure that they-would have
]
7 to articulate it as margin, but how sure do-you want to be 8 that if we follow this scheme we won't drift into a 9 higher -- 10 DR. KRESS: When you say how sure, you're 1 11 incorporating the concept of uncertainties.in ' i 12L the.' determination'-- 13 MR. YOUNGBLOOD: Well, both are you getting enough 141 data and are'you getting it in time, and I really -- how 1 (); 115 sure do you want to be has all kinds of things mixed in. 16 Let me -- I should have just given you a simple yes,
'17- actually.
18 DR. KRESS: Okay. 19 MR. YOUNGBLOOD: Okay. Other kinds of issues. I 20 believe I mentioned earlier that Niall had-been working on 21 the diamond tree in connection with plant-unavailability 22 issues. . Basically the top half of the thing is a goal tree, 23 and the goal tree is pretty well known, and then the bottom 24' half is less hardware-oriented things that affect the-goal 25 -tree. So the key word there is " hierarchy." If you think ~f ANN RILEY & ASSOCIATES, LTD. ( Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
126 1 clearly about the levels of that hierarchy, actually just a (} ^2 list of'those' levels has come to seem significant to me. 3 DR. APOSTOLAKIS: I don't understand the title. 4 What does the diamond tree have to do with PRA? 5 MR. YOUNGBLOOD: Uh -- 6 DR. APOSTOLAKIS: Drop-the title and it's okay. 7 MR. YOUNGBLOOD: Okay. ; J B DR. APOSTOLAKIS: They're entirely different 9 things. The diamond tree tells me, you know, what 10 influences what. l 11 MR. YOUNGBLOOD: Well, the title, half-of it, just 12 the goal-tree part, some people would'say that the top half 13 of it is like restating your PRA model and success space. 14 DR. APOSTOLAKIS: But the PRA doesn't have any () 15 goals. PRA does an assessment of what you have. Here 16 you're talking about values and goals and objectives. I 17' mean, there are two different things. 18 MR. YOUNGBLOOD: Well, I guess even without 19 talking about it as a goal, though, you choose to model 20 certain states in the PRA. They're your top events of 21 your -- 22 DR. APOSTOLAKIS: Sure. 23 MR. YOUNGBLOOD: Fault tree, they're the end 24 states of your event tree. You don't maybe talk about them 25 as goals, but they are things that you address. g ANN RILEY & ASSOCIATES, LTD.
\ . Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 l-
F. L' 127
-1 DR. APOSTOLAKIS: So if you look at the flow of a
{ 2 PRA initiating events, plant damage-states, and so on'and so 3 one'you.can say that's the-top part of the diamond tree. I 4 'think'the title is-misleading. I mean, the diamond tree 5 does certain things. It's goal-oriented. And I don't know 6 'that'the PRA does not address performance issues as well as 7 this does. 8 DR. KRESS: I don't have that much trouble with ( 9 'the title because I think I view this as somewhat of an 10 influence value. 1 11 DR. APOSTOLAKIS: Well, the'PRA is not intended to 12 do this, Tom. So to say that it doesn't do it well, I mean, 13 yes, 14 DR. KRESS: Well, you didn't let me finish my l () 15 statement. You look at all the things that influence 16 -achieving your top-level objective here, and you notice when 17 you do that that there are things that have a possible i 18 strong-influence on achieving this top-level objective that 19 are not well addressed by PRA, and that's these things down 20 here-at the bottom. 21 DR. WALLIS: But isn't PRA much the same thing? I 22 mean the PRA is a tree as well, it's just not called a 23 diamond tree. It's a PRA tree. It has the same sort of
'24 idea, what influence --
25 DR. KRESS: There are some things in common,
. O. ANN RILEY & ASSOCIATES, LTD.
(,,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 L
t 128 1 that's for sure. 2 DR. APOSTOLAKIS: Let's understand what addressing [ 3 means. So you think that.just putting down that programs 4 affect activities you have addressed the issue? Well, I can 5 do that every day. 6 DR. KRESS: CR1, no , no, this is a way to say oh, 7 look at these things -- I perceive that they affect each 8 other Now I've got to do something about them. That's 9 addressing it. You're just identifying things that are not 10 well addressed. 1 11 DR. APOSTOLAKIS: Right. And what I'm saying is I that the diamond tree does not address them either. 1 12 l 13 DR. KRESS: No, it just identifies them. l 14 DR. APOSTOLAKIS: Well, PRA has identified them ( 15 for years. It says, you know, we don't model organizational 16 factors. 17 DR. KRESS: Yes, but you identify them as, sort of 18 in an ad hoc way you say we know organizational factor are 19 important, but we don't deal with them. This thing sort of 20 puts it down on paper and says yes, they're important, and 21 _they affect these things and -- 22 DR. MILLER: It gives you a structure -- 23 DR. KRESS: It gives you a little structure, more 24 structure to this thinking. 25 DR. MILLER: A structure approach identifying O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
129 1 those issues which are not - - - [ } 2 MR.-YOUNGBLOOD: I think that I'm getting the 3 message at several levels that there may be some indication E4 here that I~was trying to slam or criticize PRA, and that 5 certainly wasn't the intention. If we could return to this 1 6- point in a few slides, I think it will be very natural for 7 'me to clarify what's meant and maybe for you to discuss that 8 too.
'9 DR. APOSTOLAKIS: Maybe you can say methods for 10 identifying' performance issues'that are not well addressed 11 by typical PRAs. _That's different. Because I don't see any
- 12. performance issues that are not well addressed by typical
.13' PRAs here. Because you're not addressing them either.
14 You're just saying they're important. And any PRA analyst () 15 will tell they're important too. 16 MR. BARTON: Moving right along. l l 17 MR. YOUNGBLOOD: Let's return to that. This is 18 just a picture of a diamond tree, really, it's one I could i 19 find that looks like a diamond. As far as I know, the guy 1 20L that thought this up was Niall Hunt, but it's actually -- he 21 doesn't bother to write things down in papers, so it's not 22 an easy story, but he, Mohammed Modarres, and Marvin Roush 23 were-working on a tool to improve plant availability, of 24 which safety is a piece. 25 DR. MILLER: This is an outcome of the goal-tree
/~' .
'} ANN RILEY & ASSOCIATES, LTD. -(1/ Court Reporters l 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 1 130 1 work? . 1
-2 MR. YOUNGBLOOD: Yes.
3 DR. APOSTOLAKIS: Of what? i 4 DR. MILLER: Modarres and Hunt did what they 5 called goal-tree work to analyze systems. This is an 6 outgrowth of it. 7 DR. WALLIS: There's nothing in here about j i 8 regulation? 9 MR. YOUNGBLOOD: .Not yet. 10 DR. WALLIS: But regulation is going to -- diamond 11 tree's going to. help you to tell you where regulation should q 1
-12 ' intervene?
13 MR. YOUNGBLOODi That's the idea. 14 DR. WALLIS: Okay. (f 15. MR. YOUNGBLOOD: The next slide we've seen before. 16 I'm showing it now again because the levels of function 17 system train and so forth I find it useful to keep writing 18 those down and then putting things opposite them and drawing ! 19 belt lines and so forth. That list is just sort of the 20 spinal column of the diamond tree. 21 DR. WALLIS: Where is the spinal column on the 22 tree? I don't understand -- 23 MR. YOUNGBLOOD: Well, if you -- 24 DR. WALLIS: I don't. understand the link between 25 the conceptualization of the diamond and the
'), ANN RILEY & ASSOCIATES, LTD.
J Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034-
131 J 1 Econceptualization of the level. ( 2 MR. YOUNGBLOOD:
) It means that you have a lot of 3 -flexibility in how you - -I don't know if it's going to work 4 to show these together or not, but if at some point on here, 5 'maybe a goal. corresponds in-some to some function and then
-6 you nave subgoals being systems that can perform that 7 function --
8 DR. WALLIS: Okay. So there is a relationship -- 9 MR. YOUNGBLOOD: There is a relationship. And on 10 the next slides.actually I've taken out that list of -- that i 11 same list of levels of the diamond tree and sort of then 1 I
.12 festooned the page with bullet items from an AIT report on a 13 particular event at a particular plant, the idea being that l 14 all of these are criticisms made by that team of -- they're
() 15 either observations or criticisms. They're things that that 16 team chose to write up in the context of some event. And 17 some of them are observations that some system either didn't 18 work or was degraded. Some of them are just observations 19 that there were a lot of components that were inoperable or 20 leaky -- leaky valves. There were human actions that were 21 criticized. There were criticisms of performance at the 22 supervisory level. 23 DR. KRESS: Your point here is that these bullets 24 were things that are indicative of poor performance --
- 25. MR. YOUNGBLOOD: Yes.
/ ANN RILEY & ASSOCIATES, LTD. \ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
132 1 DR. KRESS: And that your hierarchy here tends to .#h. 2 capture all those at one place or another somehow. 3 MR. YOUNGBLOOD: Yes. It seems to me that if --
-4 that properly done, and we don't really have standard 5 _ guidance on how to do these, but if you -- that in principle 6 you could do a tree structure like this and design a 7 hierarchy that was reasonably unambiguous about where you 8 'would put things. And then everything that made any sense 9 to talk about in a performance context you could find a 10 place for it.
11 DR. BONACA: And-by the way, I mean, this is isn't 12 a real part of the root cause analysis. I mean every time 13 you have loss of function, you go through steps of that kind 14 and you get back always, if you do a proper job, to ( ). 15 supervision or some -- because you want to look if there is 16 a failure at that level. Hopefully you find failures 17 intermediately that do not propagate down into 18 organizational issues. But it's interesting how that's the 19 same structure of a root cause. 20 MR. YOUNGBLOOD: Well, that actually it's very 21 likely that Niall got it there. 22 DR. BONACA: Absolutely. In fact, you know, I 23 mean, you know, you want to work it out to the point where 24 you're looking at the organization factors, and the question 25 is'where do you stop. And hopefully you always stop ANN RILEY & ASSOCIATES, LTD.
\ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
m 1 f 133 ci- somewhere higher up in this-sequence. 2 Anyway, just an observation. ( 3 MR. YOUNGBLOOD: Well'-- 4 DR. WALLIS: Now is it your intent here that the
- 5. human actions and the supervision appear in the bottom half 6 .oflthe diamond tree then? .Is that --
'7~ MR. YOUNGBLOOD: Some human actions --
8 DR. WALLIS: Seems to be below the line in this. 9 MR. YOUNGBLOOD: Well, as you -- that's right. 10 Human actions actually, as you pointed out earlier, really 11 . belong at several levels on this thing in a functional 12 sense. 13 DR. FONTANA: I think that the diamond tends to 14 confuse things. This gives you a sequence -- conceptual () 15 sequence going from the bottom to the top, and. things that 16 affect it some from the' side. 17 MR. YOUNGBLOOD: Yes, this is -- well -- 18 DR. FONTANA: I think this is a little clearer. 19 To me, anyway. 20 MR. YOUNGBLOOD: So you think this is -- 21 DR, FONTANA: Clearer. More clear -- 22 DR. MILLER: Isn't this a diamond, though? 23 MR. YOUNGBLOOD: Well, this -- actually the -- 24 L any -- this is just a list of categories, and then I sort of 25 festooned this page with those things, but the placement on 1 ANN RILEY & ASSOCIATES, LTD. O. j Court Reporters ; 1025 Connecticut Avenue, NW, Suite 1014
- Washington, D.C. 20036 (202) 842-0034
e 134 .
.. I
'l the page other than -- the horizontal placement on the page
-2 means.nothing. It means --
(} 73 DR. FONTANA: That's right. The vertical does. 1 4 MR. YOUNGBLOOD: Yes, the vertical is intended to; 5 yes. 6- DR. FONTANA: Feeding in at various -- 7 DR. KRESS: It becomes a diamond because there's a 8 number of these things. 9 MR. YOUNGBLOOD: Yes, yes, but again this is-just 10 stuff from one AIT report -- ! 11 DR. KRESS: Yes. Right. 12 MR. YOUNGBLOOD: Maybe there would be some other 1 13 event where all the stuff was here and it wouldn't look like 14- this at all. .( -15 DR. KRESS: There is another dimension on this 16 vertical axis, and that's the level of perceived 17 intrusiveness in regulations -- 18 MR. YOUNGBLOOD: Yes. Yes, And we're -- 19 DR. KRESS: And I think they need to keep this in 20 mind, too. 21 MR. YOUNGBLOOD: That's right. 22 DR. KRESS: .The further down you go, the more
- 23. intrusive you are. And that's v..y T think you put human i
24 actions where they are, although they -- you know, they tend j i 25- to fit a lot of places, but --: 1 i i
~ /"N . ANN RILEY & ASSOCIATES, LTD. I lg,) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 i Washington, D.C. 20036 (202) 842-0034
135 1 DR. FONTANA: I-think.your intrusiveness is 2 .- perpendicular t o the paper. l- DR. KRESS: No. 3
;4 DR. BONACA: Oh, no, no, no. It is deep down into E the. human - -I mean -- and-I think human actions and 6 supervision is, well, really at the base of that, but is 7 also much less scrutable, because it's much more complex, 8 behind layers of, observation that you make from the top 9 ~down. Anyway, but that's interesting,,'that true sequence of.
10 ' root cause. 11 MR. YOUNGBLOOD: Well, so, two comments I need to 12 make'about this. One is that when I say that PRA doesn't, l 13 .you know, issues not well addressed by-_PRA, what I mean.is ! 14 -thatLitems that I can put on this diagram are not basic () 15 events.in a fault tree. That's really all that I meant to
-16 imply, and I don't mean to criticize fault-tree analysis in 17 saying that.
l -18 DR. KRESS: That's actually what I took it to 19 mean. I don't know why George had -- 20 MR. YOUNGBLOOD: Well, if I ever have another shot 21 at it -- l L 22. DR. WALLIS: Because all of these are human l' 23- actions. 24 DR. SEALE: Some people have to have their i 25 feelings on their sleeve. ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 i (202) 842-0034 I l
r 136
~ 1. DR. WALLIS: All of these are human actions, so
() :2 what you're pointing.out is that human faults are not very 1 3 well modeled by PRA. 4 -MR. YOUNGBLOOD: Well, specific human actions are, I l .5 but if you had to go back to a higher-level example, if i-6 there really is a large number of valves that leaked and l 7 unavailability of parts is kind of'a maintenance backlog,
.8. and'somehow they weren't catching, if there was a rash of 9 Enot capturing test'-- failures after maintenance, in a 10 sense, PRA -- some cut set in a PIU4 might have a large g 11 number of leaky valves in it, and at that level it's in I l
12 there, but in a typical PRA you wouldn't necessarily take I
.13 the trouble to model an underlying common-cause event of all 14 those, or you might.
I 15 You.might not do it, but this comment of the AIT 16 was sort of trying to signal programmatic conditions. The 17 fact that if they think they saw -- by large I think they 18 meant-more than they would have thought and that meant that 1 19 something underlying wasn't working right and PRA I think 20 does some job of capturing the fact that if a large number 21 of valves leak you may.have a functional failure but the 22- actual prospect of that happening maybe isn't terribly well 23 quantified in all cases. 24 DR. KRESS: Well, it's difficult -- 25 MR. YOUNGBLOOD: Yes. ! - ['D ANN RILEY & ASSOCIATES, LTD.
- \m / Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
I l l 137 ) 1 DR. KRESS: -- to put an importance function _on 5 ( ~2 _ human action. What we are~after is how important are these 3 things to the final risk product and somewhere down there 4 you have to have.it modelled in your fault tree in order to 5 get an.importance factor out of it. 6 You are just saying a lot of those things just 7 'aren't'modelled in the fault tree and how can you get an 8 importance' factor out of it, so how do we know how to deal
-9 with it.in performance space. I 10 MR. YOUNGBLOOD: Yes, and I think we have a 11 structured way of deciding that that stuff is important and 12 that PRA plays an important role in that once we decide what 13 challenges the thing has_to meet and what systems we want.
14 We can go back and walk them down and pick up the stuff that l () 15 is not in the PRA, but the PRA is more guide to success 16 paths in that sense than it is an actual list. 17 DR. APOSTOLAKIS: Sure. I think the PRA does a 18 pretty good job from human actions and higher. From human 19 actions and below you have other models like Athena is 20 trying to put some structure there. Jim Wreathal has 21 published figures that show how supervision and line 22 management deficiencies and so on -- so there are models 23 down there. 24 MR. YOUNGBLOOD: Yes. 25 MR. ROSENTHAL: Let me remind you, what we are O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 ( l
138 I l 1 trying to do is figure out what we should do with j r~s 1 (J) u 2 performance-based regulation and part of that was saying 3 just where should we go, at what level should we look in 4 figuring out how we might use the square. 5 Now if I am talking about, and we have had this 6 discussion -- part of my new title is Human Factors. It is
)
7 something I care about. Nevertheless, if we have a 8 performance goal of .95 diesels, how did I get .95? Because ; 9 that was a risk-informed number that got me the .95 and that j i 10 becomes my performance goal, and I am measuring .95 diesels. ' 11 I am meeting my goal. I know that I might stop at that 12 level even though I know that human performance is affecting 13 the unavailability of the thing. For those things which we 14 think that we could measure at a performance level that 7 15 achieves our thing, we won't look lower, and so I don't have 16 to look at human performance as it affects my diesel if my 17 diesel's performance is adequate. 18 Now if my concern is how this crew performed in a 19 high stress, rare event, dynamical situation, well, yes, 20 then I have to worry about how am I going to come up with 21 something. I could monitor that, but we are trying to apply 22 that rigor to it. 23 DR. BONACA: Of course you have always the 24 challenge that programmatic and organizational failures may 25 affect common cause because of the very nature of those and "D ANN RILEY & ASSOCIATES, LTD. (O Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l 139 1 the moment which you commit not to look at those issues-then () 2 you have to have some other reliance, and certainly one 3 could be that you rely for example on a proper corrective 4 action program that is looking at root causes effectively. 15 I mean that is important at some point to get some 6 judgment there that you are not waiting until something 7 happens to make a judgment that you had some deep-seated 8 common cause programmatic failure. 9 Okay. I can see the challenge that you have 10 anyway in moving up to the top part of that. 11 MR, YOUNGBLOOD: I would like to make one more 12 . comment about this slide is there's a whole dimension of 13 requirement that this doesn't address and that is 14 documentation, because the way this thing is wired, it is () 15 about stuff other than documenting. It is about stuff other 16 than showing the regulator what is going on. It is just the 17 sheer safety performance that this tries to capture, so we 18 could bend this or adapt it in some way to do that, I 19 suppose, but I just want to make the point that that is a 20 dimension that is not on here. 1ML With that, I think I will try to pick up speed. 22 Before getting to the process steps, there is one more idea 23 that needs to be thrown out, the idea of allocation. This 24 topic has to do with the fact that you wouldn't necessarily 25 want to set your performance goals simply by the numbers O ANN RILEY & ASSOCIATES, LTD. 1025 Connecticut Avenue, NW, Suite 1014 Court Reporters Washington, D.C. 20036 (202) 842-0034
140 1 that you had used in your PRA. If you come in with a core i I ( ) 2- damage frequency of 10 to the minus 5 and you believe that 3 the.mean failure. probabilities you used are ambitious, maybe
- 4. you think.they are right, that doesn't mean that that is'the 5 thing that you want to use to establish -- to think about 6 performance here, because here it seems to me -- of course, 7 this is policy -- but it seems to me that-there is a level 8 of safety you~would like to drive to and the thing you need 4
9 to answer is what numbers do I need in order to achieve 10 that, and that that is a distinct exercise that people -- 11 and I believe of course that the licensee would have the 12 lead on doing that allocation. 13 Another idea that we sort of saw in the literature 14 search and that I think has been discussed here in other j () 15 ' meetings is that it is useful to recognize a distinction 16- 'between how you license the plant from a design basis point l 17 of view and what you care about overseeing or requiring in 18 the way of operational measures. I think there is a 19 distinction that is useful to remember there. 20 There is a slide here now about -- 21 DR. WALLIS: This is one of the disconnects that 22 ' runs.through the NRC all the time is the conservative 23- depictions of things that make it okay for being sure you 24- are outside somewhere. It doesn't help you at all when you 25 are faced with some real situations. O
\s,/
' ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p,-.-,--. 141 i 1 MR. YOUNGBLOOD: Yes. Yes, there are in a sense n f V) 2 two things, two things wrong with it. It makes you worry 3 about the wrong things and it may not make you worry about 4 the right things, but I remember one meeting here where ! i 5 South Texas came -- I am sure they have been to many -- but 6 they were making the point that actually it is good to have 7 robust systems in your design basis, so they weren't in i 8 favor of sort of junking that, but then recognize that 9 something different goes on later. 10 This slide is simply -- in case the point needed 1 11 making, the point is that in risk analysis you have modelled ' 12 your CDF in terms of initiating events and mitigating system 13 failures that the allocation process -- somebody decides 14 what we want to live with and what these numbers need to be
/~~~)
( 15 j in order to get there, and it is those numbers that have 16 some bearing on setting performance monitoring rather than 17 the numbers you may have used in your PRA. 18 DR. FONTANA: Does it lead to a legal problem if 19 it has different than a design basis? I am asking these 20 guys, I guess. 21 MR. ROSSI: I don't know. 22 MR. KADAMBI: I would think that those are the 23 kinds of problems that once we understand the application of 24 performance-based approaches a little bit better and 25 actually apply it in regulatory space, we would have to face ANN RILEY & ASSOCIATES, LTD.
/]
\_- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
142 1 some of those questions. r .
'2'
) DR. APOSTOLAKIS: This allocation process, is it
- 3. going to be done here on a generic basis or isn't it highly
'4 plant-specific?
5 MR .YOUNGBLOOD: .I would think that it would.be
'6 plant. specific.
7 DR. APOSTOLAKIS: So the. plant would do it? 8 MR. YOUNGBLOOD: At the very least plant-type 9 specific. I could imagine types of plants having generally 10 similar allocations within a type. 11 MR. ROSENTHAL: Well, let me get back to -- if you 12' would put the former slide up -- you get back to the point 13 that Dr. Rossi has been making. In the maintenance rule it 14 is the plant that sets the goals for their equipment at the () 15 system, function and train level and component level, so it 16 is the plant-that is doing the P of C given the maintenance 17 rule, so that is already in our regulations. They are doing 18 it. 19 MR. YOUNGBLOOD: Okay. Let's see how quickly I 20 can get through the steps. 21' By report process here I mean that there is a 22 series of steps in the report that I will try to walk 23 through very quickly. 24 The output of those steps is intended to be a set 25 of performance areas where really a scheme, a O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014
-Washington, D.C. 20036 (202) 842-0034
143 1 performance-monitoring, a mix of performance monitoring 2 2 requirements and prescriptive requirements that put together [J - 3 is supposed to' lead to the level of safety that you 4 articulated before you entered the process. 5 -Step one, " Build Safety Case." You could say^that 6 that has some relationship to risk-informing, and safety 7 case is a phrase in fairly common use, and I've sort of 8 indicated here what.I mean by it. But as part of that,. 9 you're allocating performance, deciding what success paths 10 you want to take credit for, you want to take credit for 11 bleed and feed, fine, put it here. The objectives CDF has 12 called out here, I think that if you were going to have a 13 requirement, some kind of requirement on some kind of 14 defense-in-depth other than CDF, I think that's where you () 15 would put that'in. And again, that's something that the 16 licensee does. 17 The next step, at least conceptually build a 18 diamond tree, I put conceptually here to make clear that, 19 you know, a diamond tree could be a vast thing. I'm not 20 sure -- we haven't really learned what piece of it we need 21 to draw or what rules of thumb to shorten it. 22 DR. KRESS: If you have more than one safety 23 objective, you would have a separate diamond tree for each 24 one? 25 MR. YOUNGBLOOD: Well, no, I think the way goal ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034-l r-
~
144 1 tree people think about it is you'd have them all -- because ; 2 some performance elements might support more than one (us) 3 objective, and your allocation, as you allocate performance, 4 you'd want to be thinking about everything -- 5 DR. KRESS: Combine them. 6 MR. YOUNGBLOOD: Yes. And then -- I'll actually 7 walk through this. I'll wave my hands-at the next slide to 8 describe what step 3 means, but given the level of 9 performance that you want, and you could of course write 10 numbers on your -- if you had some train level availability 11 goal that you had allocated, you could sort of scribble that 12 on your diamond tree. 1 13 So now you have a picture of performance and a 14 description of what performance you need from various nodes (Dj 15 to get the performance you want. Now go downwards through ( 16 the tree, try to put performance monitoring requirements as j 17 high on the tree as you can, and if it doesn't work there, 18 for some reason, take a step down, because you've got to get 19 the assurance somewhere, and you keep working downward 20 through the -- I 21 DR. WALLIS: Can I ask something here? I looked 22 at your tree, and you've got all sorts of boxes which say 23 maximize reliability, minimize probability of failure. 24 MR. YOUNGBLOOD: Oh, now what page are you looking 25 at? k\I
- ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
145 1 DR. WALLIS: Operation of controls. Well, these f 2 are impossible extremes.
)
3 MR. .YOUNGBLOOD: Oh, you're looking at --
'4 DR. WALLIS: You can't maximize. It's a 5 meaningless statement. The tree is--
6 MR. BARTON: Where are you, Graham? 7 MR. YOUNGBLOOD: He's looking at the report. 8 DR. WALLIS: The tree is full of exhortations.
-Maximize the probability the operator will quickly detect.
~
9 10 You want the probability to be 1? That's maximizing the 11
~
probability is make it 1, just a meaningless statement, a 12 MR. YOUNGBLOOD: Yes. 13 DR. WALLIS: It's full'of statements like that. ) 14 MR. YOUNGBLOOD: Yes. I think some of the diamond , f 15 trees are. 16 DR. WALLIS: I'm sorry, it's just that --
- 17 MR. YOUNGBLOOD: No, that's okay.
18 DR. WALLIS: Just don't use the word " maximize 19 probability" like that, it's a red flag. , 20 MR. YOUNGBLOOD: Yes. 21 DR. FONTANA: Don't use the word " minimize" 22 either. I 23 MR. YOUNGBLOOD: This is a little gaudy, and maybe 1 24 'it's underdeveloped -- 25 DR. WALLIS: Let me tell you why I got into that,
/ ANN RILEY & ASSOCIATES, LTD.
C Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
146 1 because I said look, after all this sort of religion and f) V 2 picture conceptual let's see what a real case looks like, 3 and so I looked at your real case, and it's full of these, 4 again, these exhortations, which I don't see are useful. l 5 MR. YOUNGBLOOD: Those are. I don't -- there are 6 things about those trees that I don't necessarily like, but 7 as I say, there's not a standard format, and they may not 8 have been drawn -- well, they weren't drawn from several -- 9 I'll just -- let me just stop. 10 DR. WALLIS: Well, let me get back to the question 11 I asked you an hour ago, is to show us the usefulness of the 12 approach. 13 MR. YOUNGBLOOD: Well, what I think you would like 14 to see, and what I would like to do, is a pilot. And we n, qj 15 haven't done that, and we can maybe, well, if I move faster 16 than I did, maybe we could have talked about what ingredient j 17 a pilot might have. 1 18 This slide is intended to be sort of marked up. 19 If you wanted this box to be a 10 to the minus 5 box, 10 to 20 the minus 5 probability of failure of some function, then -- 21 and you could get that performance by some combination of 22 these lower boxes, maybe one of these boxes would be a 10 to 23 the minus 3 box, and the other would be a 10 to the minus 2 24 box, 10 to the minus 4, 10 to the minus 1, assuming, of 25 course, that they're independent, you step down deciding how /N ANN RILEY & ASSOCIATES, LTD. (m,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r a l 147 1 much performance you want and how you can get it. And you j [") 2 might decide at some point, as you decompose your t \> 3 performance, you might decide that actually this box you can 4 actually establish that it's performing at that level by L 5 some sort of monitoring scheme. 6 DR. KRESS: At this point you're talking about 7 what the licensee would do. 8 MR. YODNGBLOOD: The licensee did the allocation. 9 I think the licensee would do the allocation, I would think, 10 but it would have to be iterated with how the regulator saw 11 the monitoring scheme. 12 DR. KRESS: The licensee might come down and say I 13 want to emphasize the performance of this thing, but NRC J 14 might say no, we want you to put some on this other because A l
) 15 of certain defense-in-depth considerations or something, j 16 MR. YOUNGBLOOD: Well, yes. I guess 17 defense-in-depth would have entered before you got to this 18 stage in the safety case, and you would have allocated 19 performance in a way that met whatever those objectives 20 were, CDF, defense-in-depth, and, yes, at that stage it 21 would have forced you to do a different allocation. So, but 22 given those constraints, the licensee would do the i
23 allocation. j i 24 DR. APOSTOLAKIS: I have a problem with the use of i 25 the word " allocation," because this implies that I'm going [~N ANN RILEY & ASSOCIATES, LTD.
\,_ Court Reporters i 1025 Connecticut Avenue, NW, Suite 1014 !
Washington, D.C. 20036 ' (202) 842-0034
c ; 148 1 to design a system and build a reactor now. And what we (} 2' have in reality is 104 plants out there operating. So the 3 utility will'not really allocate anything. The utility will 4 come back and say -- if they have to do this, they will say 5 well,-at this level at least my PRA does something, it tells 6 me something, tells me what the unavailabilities of the 7 functions are, trains and components. This is the way it > 8 is. I'm not allocating anything. And presumably it's 9 acceptable because I'm allowed to operate. 10 Now the question is which parts, which boxes here 11 which I have already assessed can I monitor and establish 12 performance criteria for. That's really the question, not 13 allocating performance. The performance is already there. 14 MR. YOUNGBLOOD: The best estimate - or I () 15 shouldn't use that phrase here in this audience -- no, the 16 mean failure probabilities are already there in some sense. 17 I believe that the numbers that go on this diagram are more 18 generous numbers than that, that some people -- for example, 19 I mean, an implication of this is that if you invoke 20 something on this scheme, the regulator will get interested 21 in it. 22 DR. APOSTOLAKIS: No, but my point is really one
~
23 of approach. I think the approach you are taking is I'm 24 starting with a clean slate and I'm allocating and I'm 25 looking at things and so on and I'm saying no, you already ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington,.D.C. 20036
.(202) 842-0034 1
r 149 1 have.what is already out there. So you look at it, you say l
) !.(
2 well, this is the function unavailability they have for this 3 plant, system trains, components, now what is it that I can 4 use as-a performance measure'to convince myself that.this is 5 what's going to remain next year. 6 DR. BONACA: These are criteria. These are not 7 ' calculated -- what he means, I. understand what you are { l 8 'saying, you're right, you know, PRA, but assume that for ' 9 example you're expecting a certain availability from.a 10 component, okay? You're not going to set your goal to that I i 11 availability. Probably you are going to give yourself some ' 12 margin to that. l 13 DR. APOSTOLAKIS: Sure. ! 14 DR. BONACA: And I think that would be for the I () 15 licensee to propose it -- l 16 DR. APOSTOLAKIS: Yes. 17- DR. BONACA: -And then you get some box out there
- 18. 'where you say well, you know, I can't monitor that, I would 19 rather just go down below a level and deterministically 20 commit to something else, okay?
21 DR. APOSTOLAKIS: But the starting point should be 22 what I have now. 23 DR. BONACA: I agree with that. 24 DR. APOSTOLAKIS: I'm not allocating anything. I 25 mean, this is it. This is what I have. This is what's ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r 150 1 feasible. [ ) 2 MR. YOUNGBLOOD: I think I would admit that as one 3 way to implement this, but I-have imagined that some 4 licensees who have a lot of alternative core cooling schemes 5 in their PRAs would not want to have all of those schemes 6 -pop up here and get requirements on them that they would 7 maybe say well, you know, to meet 10 to the minus 4 I don't 8 actually need all those success paths, I don't want all 9 those criteria, you know, being monitored against all those 10 criteria. I'll just bet the ranch on a couple of good 11 success paths that I'm sure I can satisfy the more generous 12 number that I can live with and still meet 10 to the minus 13 4. 14 So I saw it as a way of giving discretion because, (f 15 as Mario says, it's criteria we're talking about here and 16 not modeled risk. 17 DR. APOSTOLAKIS: Well, I guess my complaint was 18 that I.didn't hear you emphasize enough the fact the 19 facility already exists, and that you already have a lot of 20 these numbers. 21 DR. SEALE: Yeah, but it has brought you down to 22 the point where you now can deregulate certain blocks on 23 this diagram based on an assessment of what the overall 24 contribution to risk is. 25 DR. APOSTOLAKIS: But I am not allocating i ANN RILEY & ASSOCIATES, LTD.
\m- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 i (202) 842-0034 1 l
C ' i l l 151 l 1 anything. l 2 DR. SEALE: I didn't say that. I said you can
}
- 3. ' deregulate. ,
I 4 DR. APOSTOLAKIS: Yeah, I agree. '
'5 DR. SEALE: I don't put as much inspection in it.
6- DR. APOSTOLAKIS: Yeah. 7 DR. BONACA: The use of the word " allocating" is j 8L confusing.
.9 DR. APOSTOLAKIS: I think allocating means 1C 4 designing something, and I am saying this is what I expect 13 from this system, from that system.
12 MR. YOUNGBLOOD: Okay. The next three slides are 13 just figures in the report and they are just there for 14 purposes of illustration. This is really where that _l ty \ j j 15 . beltline came from. What is shown on this particular slide j ! 16 is probably a bad scheme, the most intrusive scheme 17 imaginable, where the regulator is looking at stuff at all 18 levels of the. tree. 19' DR. WALLIS: I don't know that that is necessarily
'20 bad as long as - he doesn' t do it all the time. It seems to 21 'me it be very good if the regulator says, at any time I can 22 come in and make a spot check of something. That keeps the 23 licensee on his toes. That is not necessarily a bad thing. )
I 24 Otherwise, he will.just work on the things which are l 25' emphasized. So it is not necessarily intrusive to have the I ['Y ANN'RILEY & ASSOCIATES, LTD. l
\ms ~ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 L (202) 842-0034 o
152 l 1 ability to check things, as long as it is not done to 3 f'~') 2 excess. \.~ / I 3 DR. APOSTOLAKIS: That is not what NEI wants, I 4 though. They want you to do that only if some performance i 5 measures -- I j 6 DR. SEALE: Or only if you indicate that in a 7 certain maintenance operation, the level of CDF has risen to 1 8 a point where you need more assurance that you have got 9 another -- 10 DR. APOSTOLAKIS: Well, Graham says that we should 11 reserve the right to go and -- 12 DR. SEALE: Yeah. I 13 DR. APOSTOLAKIS: And NEI says no. NEI says only . 14 if you have a good reason to do that. That is a different, r~'N l i 15 I mean -- 16 DR. WALLIS: Everything else which isn't somehow 17 measured in this way can go to hell and you don't have any 18 right to go and find out? 19 DR. BONACA: Although the special program that was 20 presented, I mean had all kinds of flexibility for -- 21 DR. WALLIS: It has got to have flexibility on 22 both sides, I think. 23 DR. SEALE: Okay. Now, you have found us an issue 24 where we can have some comments. 25 MR. YOUNGBLOOD: I hope it is semi-clear by now /) . ANN RILEY & ASSOCIATES, LTD. (._/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
153 1 that a process could lead to a big mix of performance things ( 2 that'you.could monitor and some prescriptive requisements to 3 , reinforce performance,.where monitoring wasn't going to 4 work. One thing I would just like to throw out briefly, 5 because it. appeared in the report, some people took an 6 interest in it, is the idea that.abductive inference might 7 be a good way to thinh about the problem. The issue is we 8 are getting -- we get a boat load of numbers and how do we 9 think about what they mean?
.10 .And what this, all that this slide means to 11 suggest is.that in some ways it is analogous to medical 12 ' diagnoses and that work is going on in that community to 13 formalize how they think about getting a lot of information.
14 DR. WALLIS: You still have regular checkups I) 15 whether you are sick or not. 16 MR. YOUNGBLOOD: Yes. Yes, there is your 17 monitoring. So, I have a couple of summary slides here, I 18- am not certain how -- we have talked about things so much 19 that I can probably zip right through it, and then maybe we 20 could a take word to talk about what a pilot might look 21 like. 22 The process was biased towards finding the most 23 performance-based possible scheme, and, again, we didn't l 24- argue the merits of that or debate it, we just tried to make 25 a scheme that would drive that way. And we tried to look at /"N ANN RILEY & ASSOCIATES, LTD.
\-) % Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p-- 154 1 areas -- well, there is that phrase again. We tried to look
.x) f~) 2 at those areas that I don't have a good phrase for, that 3 don't leap out of PRA as having been well treated.
4 The process involves, first of all, formulation of 5_ a safety ~ case, and I could just sort of toss that off in a 6 page because this was about performance-based and not 7 risk-informing, but, obviously, there is a lot behind making 8 .a safety case. l 9 -Then we described a process for identifying a 10 combination of monitoring measures and other measures that 11 would somehow combine to satisfy the objectives that you had . I 12 articulated. ' 13 The example we did suggests that the physical 14; parameter temperature is not necessarily a good way to D) q 15 monitor a functional performance. No need to dwell on that, j 1 16- Diamond trees -- 17 -DR. APOSTOLAKIS: Is the message here, Bob, that 18 the prevailing view that one can have performance-based i 19 regulation without it being risk-informed is really false? 20 Because I think risk-information was an essential part of 21 your presentation. 22 MR.-YOUNGBLOOD: It was. I took the view, I think 23 in principle you can have performance-based without 24 risk-informing. And-I guess, formally, I could say, well, 25 maybe your safety case wasn't risk-informed either, but
/"' ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
I 155 1 somehow you could talk about performance. Without a j#') '2_ _ complete PRA, I think of availability, you could even work
%.)
3 with availability and have it not be risk-informed. You 4 could just work with a gibberish set-of unavailabilities, 5 and then go about performance-based implementation of that. 6f DR. MILLER: We have performance-based without 7 risk-information for a long time. What you are really-8 saying is risk insights help you do performance-based 9 regulation. 10' DR. APOSTOLAKIS: No, he is saying you can't do '
.11 it.
12 .MR. YOUNGBLOOD: Well, the process I described had 13 a safety case in it that, to my mind, began with a PRA. 14 DR. APOSTOLAKIS: The example that NEI did and () 15 Scientech reworked convinces me that every time I see now a 16 . deterministic performance criterion, I will have to become 17 .very skeptical, j 18 DR. KRESS: What is this now? 19 DR. APOSTOLAKIS: Well, excuse me, but that is 20 what the example shows. The examples shows that what NEI 21 proposed made no_ sense in the probabilistic framework, that
'22 you had to go deeper and look at the initiators and_the 23 conditional probability of core damage and so on. So, I 24 mean one message I am getting from this work is that you 25 can't really have performance-based regulation without O ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025' Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 8(2-0034 I
i 156 1 risk-informing it.
'2' MR.-KADAMBI: I'would like to make sure that the
,3 message of this report, you know, is what I believe it 4 should be, which is that there is a methodology that we had 5- not become aware of, even though it was really the diamond 6 -tree was done on; behalf of NRC, actually, some time ago. So 7 this was something that we became aware of that could be 8 useful in thinking about performance-based approaches,.but 9 the broader thing is that it would help us in dealing with 10 the broad range of regulatory applications that we in this 11- agency have to worry about, including all the materials
- 1.2 issues.
13 It could well be that there are many NMSS 14 licensees who don't have any kind of risk analysis at all 15 . who are -- you know, we may be able to regula*.e them better '
~
16 if we use.a performance-based approach, as opposed to what ' 17_ is likely right now would be a very prescriptive kind of 18 approach. I think in principle you can have 19 performance-based regulation without any risk analysis and
.l 20- this did not -- nothing in this work I believe contradicted 21- .that.
22 'DR.'APOSTOLAKIS: No, I thought the example they 23 did contradicted that. , 24 MR. KADAMBI: Well, that was just an' example -- 25 DR. APOSTOLAKIS: But you didn't show another ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
157 11! example that did work. () i 2 3 MR. KADAMBI: -You'know, as I mentioned, this is a very modest contract. It spanned less than a year and we 4 really were not able'to answer a lot of questions and in 5 fact if you look at that report in the Foreword we try to 6 identify many important questions >that we were not able to 7 answer.at all. 8 :MR. YOUNGBLOOD: No, but the fourth attribute'that 9 you don't want to have a problem by the time you trip this 10 criterion, I think it can fairly be asked whether there is a 11 way to explicitly address that bullet without a-risk 12 dimension, and that.is certainly the way I approached it. 13 DR. APOSTOLAKIS: Well, you can address it perhaps 14 int looking at how many additional ways you have to achieve () 15 the~same thing without looking at the probability. 16 MR. YOUNGBLOOD: Yes. 17 DR. APOSTOLAKIS: The pre-PRA way of looking at 18 .thingc, you know, the number of events in a minimal cut set 19 without the probabilities. 20 MR. YOUNGBLOOD: But that would to me -- scenarios
'21 being part of risk though, I mean that's -- if you had a 22 logic model --
23 DR. KRESS: That is still being risk-informed, 24 wouldn't it? j 25 MR. YOUNGBLOOD: Yes. It wouldn't be PRA-based -- i N ANN RILEY & ASSOCIATES, LTD. s Court Reporters 1 1025; Connecticut Avenue, NW, Suite 1014 ) Washington, D.C. 20036 (202) 842-0034
e 158 ) il DR. KRESS: Where you get into trouble is where [ '2 you'get down to the level of talking of things l'ike QA and 3 safeguards and' management issues and things like that, which 4 this concept is just not going to help you at all and you ' 5 will-have to decide on how you are going to set performance 6 indicators and levels there, if indeed you feel like that is 4 7 something~that has to be part of your performance measures. ' 8 Like.I say, use the PRA where you can but when you 9 get down to these levels you have got to look for something i 10 'else. 11 DR. APOSTOLAKIS: I agree with that, but the whole 12 presentation assumed.that you had a PRA -- everything has 13 probabilities in it. 14 DR. KRESS: Not the whole presentation.because O . 15 they - : 16 MR. KADAMBI: To some extent that is true, but I 17 believe the fire protection example did not'specifically use i 18 a'PRA, did it,. Bob? I can't remember the details. ; 19 MR. YOUNGBLOOD: You are pushing me. Certainly -- 20 MR. KADAMBI: That may not be a good example -- 21 DR. APOSTOLAKIS: It is not. We did review the 22 NFPA standards and I don't think the committee liked it. 123 You;know, they thought that they were going to propose two 24 parallel paths, one risk-informed and one not risk-informed 25 and the one that was not risk-informed was nothing, so I
/ ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036
'(202) 842-0034
1 159 1 really have doubts that you can have a performance-based --
.[') '2 I mean you can go to a little thing, somewhere, an issue and G'
3 say, gee, .if I went a risk parameter I don't have to do the 4 rest. .Okay, maybe,.but is that really what we are talking 5 -about when we say -- 6 DR. BONACA: I think I agree with you, George. I 7 believe that right now we are looking'at everything, the 8 whole diamond -- many part of that. What NEI is saying -- 9 we want you to look only at some high performance indicators 10 up there and that's it, and the bridge to'doing that, it's 11 risk information in my mind, okay? 12 DR. APOSTOLAKIS: Yes. 13 DR. BONACA: And it is again I think the only way 14 to enable us to make those judgments is to say we are not () 15' going to look at causative factors because we can monitor 16 functions and have sufficient margin is to have the 17 understanding, and I agree with you that without risk 18 information we're not going to get that. 19 DR. APOSTOLAKIS: Well, maybe we should tell the Commission that, because they keep separating the two. 21 DR. BONACA: Clearly I believe that many of those 22 performance indicators, that you can poke holes in them the 23 same way you did here, okay, with pretty simple analysis. I
.24 mean it was well-structured but it was pretty simple. The 25 PRA analysis showed that that was inadequate.
s' ANN RILEY & ASSOCIATES, LT! . L Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
1 1 160 l 1 DR. APOSTOLAKIS: I I thought that was a great part j l
'2 of your presentation.
[} G Lest silence is perceived as l 3 agreement, I disagree completely that the diamond tree 4 hierarchy is a useful way to organize discussion. It is i 5 really an influence diagram that you are trying to develop 6 and it will be declared as useful only after you actually do 7 it. I mean it is so easy to talk at that level, you know, , 8 programs affect this -- but try to do it and you will see 9 how difficult it is to develop an influence diagram or a 10 diamond tree which is a renaming of the thing, to actually 11 show all these influences. 12 You are talking about modelling a complex 13 industrial facility and you want to bring into it everything 14 that management does that affects other things. f") 15 I mean if you could do that, it would be a great 16 guide, but at the high level of course it is a useful way. 17 I mean actually the most useful part in my opinion was the 18 vertical stuff you showed -- function, system, component 19 down. The tree itself, I think the report over-advertises 20 the usefulness of the diamond tree, and talks about it as if 21 it is the special theory of relativity and we have to find 22 out who proposed it first. I mean if you go to the report, 23 there is a whole paragraph as to who proposed it first. 24 I think it is a simple idea. Decision analysts 25 have called it an influence diagram. The top level is the ("' ANN RILEY & ASSOCIATES, LTD. (_)) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l l 161 l 1 value tree. The bottom level is the decision part and I
~D 2 think you should reduce the emphasis on it. I mean you can
[d . 3 claim that it is a useful way to organize discussion only 4 after you have done it and demonstrated it can be done, l l 5 because the most difficult part in any decision program is l 6 built in the influence diagram and you renamed it. That is l 7 all you have done, and you say it is useful. Yeah, sure, 8 conceptually it is very useful, but try to do it. 1 1 9 DR. KRESS: On that note -- 10 DR. APOSTOLAKIS: It sounded a little harsher than 11 I wanted it to but a lot of my colleagues were impressed by 12 the diamond, and I wanted to make sure that the record shows 13 that there is disagreement. 14 DR. BONACA: But you are not unimpressed. You say rx ( ) 15 simply that there is -- 16 DR. APOSTOLAKIS: I don't think it works. 17 DR. BONACA: -- renaming or something else. 18 DR. APOSTOLAKIS: Yes, and that something else is 19 very difficult to do. 20 DR. MILLER: But it still may be a very useful to 21 organize discussion, even though you can't do it for an 22 actual situation. 23 DR. APOSTOLAKIS: No, I think the vertical thing 24 that you showed is much better -- but that's okay. I mean 25 it helps people. (~') ANN RILEY & ASSOCIATES, LTD. (_/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
162 1 DR. FONTANA: There's more ways to skin a cat. 2 b[~h DR. BONACA: But the vertical accounts for the 3 very nature that you start with an objective and it opens up 4 and by the time you get to the middle -- 5 DR. APOSTOLAKIS: People have called this a master 6 logic diagram -- 7 DR. BONACA: I don't care how you call it -- 8 DR. APOSTOLAKIS: -- and decision analysis is 9 decision tree, value tree -- I mean -- 10 DR. MILLER: All of them are hierarchical -- 11 DR. APOSTOLAKIS: A lot of stuff, hierarchical 12 approaches, hierarchical decomposition of a problem -- 13 because, you know, this may fall in the hands of a 14 non-nuclear person and then we are really undermining our O ( j 15 credibility if they see something that is very familiar to 16 them advertised as a new discovery. 17 DR. WALLIS: George -- I agree with George mostly. 18 Now the thing that concerns me is this is an important thing 19 that the Commission wants to get done and you seem to be 20 still trying to figure out how to get to first base. I 21 don't see a plan to implement anything or anything like it, 22 and you are still arguing about how you might conceivably 23 think about the problem. 24 MR. YOUNGBLOOD: That has a -- can I just jump in 25 and make one technical comment before the Staff takes over ANN RILEY & ASSOCIATES, LTD. {)) x_ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 ( Washington, D.C. 20036 (202) 842-0034
163
- 1. 'and responds to that, that the theory, the observation that 2 we should really do'it for some problem to really show, that 3 has been a consistent theme, and I certainly agree with
]
-4~ that.
5 The report as written contemplates a soup-to-nuts, 6- do the whole thing at once mentality, because I think it is 7 hard to be sure when you go about risk-informing, it's hard 8 to just take'a piece of the risk analysis and then believe I l 9 that you have done.it -- 10- DR. WALLIS: You're right. j 1 11 MR. YOUNGBLOOD: -- done it right, so in moving l 12 forward with a pilot, which I of course would love to do, in 1 13 an incremental way. It would be nice to identify a piece of l l 14 the problem that moderately cleanly decouples from the rest j () 15 and then do that, so figuring out a piece that decouples 16 from the rest would be an important step, and I think if we 17 had already done that step, we could have already done the 18 things that you are talking about.
.19 DR, APOSTOLAKIS: It seems to me, Bob, that if you 20 indeed go ahead with the pilot, it would behoove you to look 21 at the literature and influence diagrams.
22 MR. YOUNGBLOOD: Oh, yes. 23 DR. APOSTOLAKIS: That is where the action is.
.24 That is where people have spent time understanding what is 25 going on and developing mathematical theories and so on.
ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1025 Connecticut Avenue, NW, Suite-1014 Washington, D.C. 20036 (202) 842-0034
r 164
'1' MR. YOUNGBLOOD: Actually we didn't not look at
()
\~/
2 it, and I agree'we may have written more than we should have
.3 . about the diamond tree, partly because hardly anybody had 4 ~ heard of it, and'-- l
.5 DR. APOSTOLAKIS .: Well, I had.
)
6 MR. YOUNGBLOOD: Well, you are one of maybe six 7 people. 8 .DR..APOSTOLAKIS: The authors plus me, perhaps? 9 ;I don't want to finish on a negative note. I
'10 thought the analysis that you did of the NEI example was i
11 very good. I learned a lot from it, and I think your l l
'12 discussion of the hierarchical level independently of the 13 diamond tree was actually very good, but not very new to me, 14 but the other stuff was really new and I really enjoyed it. !
15 MR. YOUNGBLOOD: Good. 16- DR. APOSTOLAKIS: I thought in other words there i l 17 are good parts to this report. I don't know which ones you I 18 wrote, Bob, but -- the Executive Summary -- 19 MR. YOUNGBLOOD: No, no -- 20 DR. APOSTOLAKIS: -- but part of the' diamond tree 21 I must say, you know, if we keep it among ourselves perhaps 22 it serves a purpose, but I can see decision theorists
.23 looking at this, saying, you know, the nuclear business is 24 going its ownLway. I have done the same thing and I'd never 25 quote it that.way. It was a nice tree with a decision node ANN RILEY A. ASSOCIATES, LTD.
Court Reporters
-1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l l I 165 l b 1 at the bottom, the value at the top, the objectives, and now (} 2 that I think about it, it looks like a diamond, yes. l 3 DR..KRESS: Well, George, one thing.that I liked 4 about it that hasn't really.come out is that it provides a 5 . way to be sure you'are looking at the' performance and 6 covering every branch of this influence diagram in some way. 7' DR. APOSTOLAKIS: You use the right words and I 8 .liked the same thing. 9 DR. KRESS: Okay.
- 10 DR. APOSTOLAKIS: But'I'must point out that what 1
11 Bob showed us, with the vertical lines there, he could have ! 12 .done it, in fact he did it, without the diamond tree. l l 13 MR. YOUNGBLOOD: Yes. ' 14 DR. APOSTOLAKIS: But that was a useful part. () 15 DR. BONACA: But the diamond just describes it -- 16 I am looking at this purely as a pragmatic tool to help me 17 making some judgments, okay? 18 DR. APOSTOLAKIS: .It is not pragmatic until it is
- 19' applied.
20 DR. BONACA: I understand that. In some examples 21 that I saw and some that I have been familiar with, I just 22 played a little bit and it was pretty useful. That's all I 23 am looking at. Certainly I have not tested whether or not 24 -it is complete, if it is totally effective. That is beyond 25 my interest at this stage. (~') ANN RILEY & ASSOCIATES, LTD. s% ,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
l 166 2
- 1. You know, what is important to me, however, is
() 2 that it'showed me in a quantitative way why my suspicion of
- 3. some of those criteria I saw in the NEI proposal.--
4 DR. KRESS: Shows that you were right to be 5 suspicious. 6 DR. BONACA: Yes, in a quantitative way. 7 DR. KRESS: At this point I want to turn it back 8 over to -- 9 DR. APOSTOLAKIS: Wait, wait, wait -- 1
'10 DR. BONACA: On the components -- you know, 11 quantitative means simply looking at the contribution, 12 looking at the concept of margin --
13 DR. APOSTOLAKIS: Yes, I agree. That was the 14 great part, and the diamond tree has nothing to do with it. () 15 DR. KRESS: I think at this point we are running 16 out of time, and we need to hear Mr. Kadambi's plans for the 17 Commission paper, so -- 18 MR. ROSSI: Could I break in just one minute, 19 because we clearly did not come here with a plan that we 20 wanted you to review, but Prasad, do you have on a viewgraph 21 the questions for the ctakeholders that you could put up? l 22 MR. KADAMBI: Yes, I do. 23 MR. ROSSI: Okay. We did not come here with a 24 plan. We are at the point now of collecting information to 25 ' respond to.the Commission's request in their SRM I guess by ANN RILEY & ASSOCIATES, LTD.
\_ Court Reporters
'. 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
j l 167 l
.1 the end of May. j
() 2 3 Now at the beginning of the day I indicated that we had a number of questions and I asked people to sort of { l J 4 take a look at those and see if we are asking the right 5 questions when we are trying to develop our response to the I 6 Commission. 7 Now, here are the questions that we have focused 8 on and rather than read them, maybe you ought to read them 9 yourselves and then make comments on them. Because I think l 10 much of this has been the subject of discussion today, in l 11 particular, the -- well, all of these, I think. i 12 .I think your comments, Dr. Apostolakis, have, in l l 13 many cases, been addressed to questions that.are up there. l l 14 DR. APOSTOLAKIS: Well, a procedural matter now, I () 15 obviously, .you had some input from the subcommittee, you 16 know, the transcript is available, or will be available. 17 What is the plan now, that you will develop a plan and the 18 next time we will see it will be June? 19 MR. ROSSI: Yes. That is the plan. And that will 20 be after we send a paper to the Commission. 21 DR. APOSTOLAKIS: So you feel you have enough now 22 guidance, input from the committee? 23 MR. ROSSI: Well, we have as much as we are going 24 to be able to get. 25 DR. APOSTOLAKIS: That is a pragmatic view. ANN RILEY & ASSOCIATES, LTD.
~
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 l
168
- 1. 'MR. ROSSI: Right. I' mean we have everything that !
2 we are going to be able to get at this time. 3 DR. APOSTOLAKIS: But that will be a plan, so we l 4- can still-talk about it in June. I 5- MR.'ROSSI: Yes, it will be a plan and it may be-6 something that ties a great deal to what is going on in'the 7 . agency in the area.of risk-informing things. l l 8 DR. APOSTOLAKIS: Fine. l MR. ROSSI: 9 And, as a matter of fact, Prasad, you i 10 might put up the planning for performance-based approaches. j i 11 DR. WALLIS: I have comments on the question. Do ! 12 you want comments on the questions? ! 13 MR. ROSSI: Sure, that's fine. ! 14- DR. WALLIS: They seem to me.very preliminary type j (6), 15 questions. I mean I would have difficulty responding to any 11 6 of these without a better idea of what you guys are up to, 17 what you have in mind in the form of regulations which are 18 performance-based. These are questions based on some 19 hypothetical thing I.have difficulty visualizing. 20 So if you would come up more -- perhaps more of a 21 discussion, more of a specific thing that is visualizable of 12 2 what performance-based regulation might be more like, and 23 how it-might specifically change my life, then I would have 24 a better way of answering the questions. 25 MR. ROSSI: Well, we have the example of the t
- O ANN RILEY & ASSOCIATES, LTD.
\s ,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
I i 169 ' 1 maintenance rule. We have Appendix J. [~') 2 DR. WALLIS: Maybe say I just don't know it.
\J 3 Maybe you should make that. reference then. You should say 4 here are some examples of performance-based. If this were l 5 extended to some other regions, or something. l l
6 MR. ROSSI: Yeah, as a matter of fact, the other l 7 viewgraph that I was suggesting he put on the screen, I 8 think makes that point. 9 DR. WALLIS: Okay. 10 MR. ROSSI: Well, why don't you put the one, I 11 planning for performance-based approaches, Prasad. 12 MR. KADAMBI: Okay. 13 DR. WALLIS: So the two would go together then? 14 MR. KADAMBI: Well, these are essentially the ("% v) i ( 15 elements of the plan that we have come up with right now. 16 This was going to be my last slide, as, you know, the 17 Commission paper itself presenting these with some schedules 18 as being our plan at this point. 19 If I can just maybe quickly talk through these. 20 The Commission wanted to make sure that we were well 21 integrated into other things that are going on, so whatever 22 we do, we do together with the other offices and make this a 23 truly agency-wide effort. And to do this we need to 24 appropriately recognize where the similarities are with the 25 revisions to the regulatory oversight process, the Ch ANN RILEY & ASSOCIATES, LTD. (,) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r2 1 170 l' risk-informed revisions to Part 50 and other NMSS activities
'2 which I don't fully know yet, but we will be finding more }
3 out about, and make sure that we are not duplicating things
- 4. that are going on, that is going on elsewhere.
5 We want to learn from the prior experience of the 6 maintenance rule and Appendix J and, you know, this will be 7 part of the plan to incorporate the prior-experience.into 8 it. i 9 We want to participate in pilot projects, those E10 that are going on elsewhere, and those that we might want to 11 initiate. 12 I'm sorry, did you -- 13 DR. APOSTOLAKIS: I think -- yeah, sure, I mean 14 this is important to do. But there are some basic 15 questions. 16 DR. WALLIS: I see, a list of activities. 17 DR. APOSTOLAKIS: Yeah, these are activities. 18 Like the question we asked in our regional letter, you don't l 19 seem to think it is important enough, but shouldn't you be 20 asking people and trying to address that question. You l 21 know, who would set them up and how? And the other question 22 that I asked earlier, what is performance? And the reason 23 why I am asking is because the NFPA standard that was 12 4 advertised as performance-based had examples that said, 25 yeah, you look at the two pumps and if the distance between
~
ANN RILEY & ASSOCIATES, LTD k',N,) Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
171 1L the two is X feet, then it is fine, that is a performance { () 2 3 criterion, which I didn't' expect it to be a performance criterion, it~is not design. Right. 4 So, performance, it seems to me, has an element of 5 time in it. zYou know, you monitor something over time that i l 6 might change, right, _ not how you design the plant. So a 7 definition of performance someplace would be useful, j 8 And then the objectives. What are we trying to 9 achieve with this? And that I think'will come when you try 10 to integrate your work with what is going on in other parts 1 11 of'the agency in terms of objectives. Do you subscribe to l 12 the cornerstone approach? Do1you want something else? You 13 know, these are.the kinds of issues that should be debated l 14 - right now, because if you disagree,-then those guys that are A t y ,/ 15 working in that area should know about it. Because the last 16 thing we want is, you know, to try to risk-inform the 17 regulations and then five years from now we have a complete 18 mess in'our hands with different objectives in different 19 parts of the agency, and then, of course, we will blame 20 risk-informed regulation. 21 MR. KADAMBI: Well, no, I mean that is the reason 22 why I began with, you know, make sure that this is an 23 agency-wide effort. 24 DR. APOSTOLAKIS: Okay. 25 MR. KADAMBI: We want to keep it that way. But ANN RILEY & ASSOCIATES, LTD. p/ N, - Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
y 172 1 . relative to some of the other questions, one of the l' ) U
-2 activities definitely that we will be doing is developing 3 guidelines, you know, to identify and. assess issues and 4 candidate performance-based activities. I mean these are 5 actual specific ones.
l 6 DR. APOSTOLAKIS: Now, before the guidelines,
'7 don't_you think you need a set of principles?
8 MR. KADAMBIt. Well, I mean it may be that -- 9 DR.' APOSTOLAKIS: You have heard that before, 10 haven't you?- 11 DR. MILLER: It seems like that what we talked
- l. 12 about this morning is where I was. starting all this. As 13 George says, lay out a set of principles in some sort of 14 document, and then start on this.
() 15 MR. KADAMBI: If the ACRS recommends that that is 16 the way we go, -- 17 DR. APOSTOLAKIS: The ACRS cannot recommend 18 anything today_because the ACRS is not here. This is a i 19 subcommittee. 20 MR. KADAMBI: I understand. 21 DR. MILLER: If you started with a blank sheet a
- 22 paper and go over the process you have outlined there, you 23 are going to end up with a sheet of paper full of things l 24 that aren't very well coordinated. You need to provide 25 leadership of some type here.
ANN RILEY & ASSOCIATES, LTD. '"k Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 a _
173 1 MR. ROSSI: Well, we are certainly going to start [ A.s) 2 with the attributes of performance-based regulation. I mean 3 there is no question about that, and we have talked about 4 that today. And we are going to talk about the work that 5 has gone on in terms of the revisions to the reactor 6 regulatory oversight process. And there, they do indeed 7 have objectives laid out. They do indeed depend on the 8 cornerstones. They depend to a large extent on the use of 9 PRAs in what they do. So we will look at the thins that are 10 going on now. 11 Now, we believe that those -- we have no reason 12 whatsoever to believe that any of that is misdirected in any 13 way whatsoever. So what we are trying to do, I think, is 14 see whether there are other things that we should be doing O ( ,) 15 in the area of performance-based activities over and above 16 what is ongoing and is using PRA. And what we heard today I 17 think that we got a lot of input in this area. We didn't 18 get any, and have not as yet gotten any specific suggestions 19 on what we would focus on if we were to look at something 20 where risk was not a major contributor and to developing the 21 performance criteria in whatever we monitored. I don't 22 think we have gotten that. And the things that are covered 23 by risk I think are already going on. 24 DR. MILLER: You are looking for an example where 25 risk is not going a valuable input? ANN RILEY & ASSOCIATES, LTD. (Oms/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
p i l 174 l 1 MR. ROSSI: Well, yeah, because we have got many l l l ( 2 things going on in the area where risk is a valuable input. I 3 And, generally,,where risk is a valuable input, the 4 objective focuses on the core damage frequency. It works 5 back to initiating events and mitigating systems, et. cetera. I 6 I guess in NMSS area, it focuses on doses, it works back. 7 DR. MILLER: But that is not a risk. 8 MR. ROSSI: .Well, I suppose it is -- I think that 9 is what they focused on. 10 DR. MILLER: I think that is more performance, 11 isn't it? 12 MR. ROSSI: Well, yeah, that may be. 13 DR. MILLER: Go back to Part 20. 14 MR. ROSSI: Performance, right. Yeah. It may be O 15 less on risk. bj 16 DR. MILLER: It says you have to maintain these 17 things. 18 MR. ROSSI: Now, it may very well be, apparently 19 there are some members here today that feel that way, that 20 the areas where you don't use risk in developing the 21 performance-based approach are limited. I mean that may be 22 the answer. I don't know. That appears to be at least one 23 member's view, perhaps others. 24 DR. KRESS: Of cou'rse they are limiting, but the 25 question is, are they important enough to develop some t) ( ,/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
175 1 ' guidelines on how to treat them'in a performance place? I. 2 think if we look~at the influence diagram and see what is d[~D ~ 3 .important even in risk-based, it probably would have to
~4 include -- yeah, they are limiting,_but they are important.
5 Probably you need to develop some guidelines on how to treat 6' them in performance-based. 7 DR. SEALE: Well, whether you call them diamond l 8 diagrams or risk diagrams or whatever name you want to give 9 them, and whether you are going to be worried about being 10 accused of stealing them from somebody else, or starting 11 over, or whatever, I don't care, the thing that came out of 12 this that I saw was a lot better basis on which to begin to I 13 identify the' level and the kinds of things that intervention 14 oriented performance indicators would have to address. () 15 DR. KRESS: Well, I think we are talking about 16 things like QA, training, institutional factors, safety 17 culture, safeguards. 18 DR. SEALE: Specific to certain systems. 19 DR. MILLER: Well, why don't you try training? 20 There is a lot of information there, and a lot of 21 performance criteria in training. 22 DR. KRESS: Well, once again, I think you are 23 going to have trouble determining performance indicators 24 that are different than just the process, because they have 25 the appropriate process in place. Q '\ ,,/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
r 176 1 DR. SEALE: Yes. ij ) 2 MR. ROSSI: Well, you can look at examination 3 results. 4' DR. KRESS: Examination results. 5 MR. ROSSI: Simulator results and that kind of 6 stuff. But that may be already being done. I mean that -- 7 DR. KRESS: My point is -- 8 DR. SEALE: That is not where the screw-ups are. 9 The screw-ups are with the guy on the maintenance floor. 10 MR. ROSSI: We have the maintenance rule to look 11 1 at that, and.the maintenance rule is indeed 12 performance-based and, presumably, if that is where the 13 problem is, it shows up in monitoring that is done for the 14 maintenance rule, j 15 DR. SEALE: Yes, yes. It is not in the training 16 as-such. I mean not what we normally think of as the l 17 training program. 18 MR. ROSSI: Right. Right. j 19 DR. SEALE: Is it in the maintenance areas. 20 DR. KRESS: Well, I think there are some important 21 issues in just performance orienting the risk-informed part. 22 DR. SEALE: Yes. 23 DR. KRESS: I think there are still some important 24 questions that were raised here and need to be asked. I 25 don't know if you guys are the right ones to address those ANN RILEY & ASSOCIATES, LTD.
}O %,/ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
F 1 177 1- in'this particular program. But there are some good () 2-
.3 questions that came up that I think need addressing. And maybe that is -- maybe you ought to look at that, too.
l 4 MR. ROSSI: Well, one'of the things that I think 5 we can say, based on our interactions with industry l 6 represented is we haven't -- I guess we haven't heard 7 anybody stand up and give an example.of where they believe 8 there is some particular regulation, or set of regulations 9 that are overly prescriptive, that could be made more 10 performance-based and reduce the burden without reducing. 11 safety, that are not being covered in the efforts that are l 12 going on in risk-informing the regulations. I don't think i 13 we have heard that. So that is an important piece of input, 14 I believe. ( 15 DR. KRESS: Yeah, when you do hear that, you will 16 need some principles to guide you on deciding whether or 17 not -- 18 MR. ROSSI: That's right. But then the question 19 is, how much effort do we put into it until we have a real l ! 20 problem to work on? j 21 DR. KRESS: Ahead of time until you have j 22 something. Yes, good question. l :23 Well, I think we have reached the witching hour. 24 Are you just about through with this? I 25 MR. KADAMBI: Yeah, I am really -- as I mentioned, l
,f'} ANN RILEY & ASSOCIATES, LTD. \/ Court Reporters )
1025 Connecticut Avenue, NW, Suite 1014 i Washington, D.C. 20036 ) (202) 842-0034 l c j
178 1 this was going to be my last slide anyway. One of the, I () 2 guess, important points to be made on this slide is we are, 3 you know, thinking about modest resources on this part of 4 the program right now-. And we do foresee that if something 5 useful comes out of the plan, it will become incorporated 6 into the normal agency activity. So it will either become 7 institutionalized or it will be sunset, depending on. 8 whatever comes out of it. 9 DR. KRESS: At this point-I don't anticipate a
~ 10 letter because' subcommittees don't write letters, and l 11 probably our next time we will hear about this is when you 12 do have some sort of a Commission paper, or a draft plan.
13~ Maybe we can look at it then and make some more comments, 14 and actually have a letter. () 15 DR. SEALE: And that will be June. 16 DR. KRESS: And that I think would be at least 17 . June. 18 MR. KADAMBI: We intend to meet our schedule j 19 relative to the Commission paper, which will be done and l 20 submitted to the Commission by the end of May. 21 DR. KRESS: The end of May. Yeah, you need to go 22 ahead and do that. 1 23 MR. KADAMBI: Right. l 24 MR. ROSSI: And we did get what I feel to be a lot 25 of good comments today, and a lot of good input that we can ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
179 1 .think about. And_the kind'of things that we got in our ' f) 2. other public meetings, even though there were some kind of 3 what.I would say,~1ack of input, I think that sends us a 4 message also. 5- DR. MILLER: I think an exercise maybe the staff, 6 I like.to:see'the staff do, just in your offices, take your 1 7 attributes'and apply them to-a current performance-based 8 process like some part of the maintenance rule. Just see 9 ~ how they stack up. Not something you are going to write up, 10 just. force yolrself to do something real. Because all right
.11 'now Professor Wallis is saying, we are doing a lot of 12 talking, but we haven't done much.
13 DR. KRESS: Well, I think he did that with the NEI 14 . proposal as a rule. I-mean that basically was an example of () 15 doing that. 16 MR. ROSSI: Yeah, I think that has been done, to 17 apply the attributes. And I think you can apply them fairly 18 quickly. 19 DR. MILLER: But the NEI proposal was not an 20 ongoing performance-based process. 21 DR. KRESS: But it was the form of what a rule ! 12 2 might have been, that is a way to look at it. 23 DR. FONTANA: Are we about done? 24 DR. KRESS: Yes. I am getting ready to -- 25~ DR. FONTANA: I looked up, one of your slides had Q
\)
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
180 1-abductive reasoning, and I looked it up in the dictionary, () 2 3 and a third definition is a syllogism where_the major premise is certain, but the. minor premises are probable.
'4 That.goes back to 1670 or 1700 years. But the first
- 5. definition is illegal carrying away of a person. Which one 6 of those are you --
7 DR. SEALE: Where is this? 8 DR. KRESS: It is the fourth definition, you need 9 another dictionary. 10 DR. FONTANA: Abductive reasoning. 11 DR. SEALE: Abductive, I see. 12 DR. KRESS: With that, I am going to declare this 13 meeting adjourned, i 14 [Whereupon, at.1:02 p.m., the meeting was () 15 concluded.] 16 17 18 19 20 21 22 23 24 25 ANN RILEY & ASSOCIATES, LTD. O\ Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034'
i REPORTER'S CERTIFICATE This.isLto certify-that the attached' proceedings-
-] before the United States' Nuclear Regulatory Commission in-the matter of:
NAME OF PROCEEDING: RELIABILITY AND PROBABILISTIC RISK ASSESSMENT AND REGULATORY POLICIES AND PRACTICES-CASE NUMBER: l l l PLACE OF PROCEEDING: Rockville, MD l-l were, held.as herein appears, and that this is the original i transcript thereof for the file of the United States Nuclear ' 10 g j Regulatory Commission taken by me and thereafter reduced to l
-typewriting by me or under the direction of the court reporting company, and that the transcript is a true and E accurate record of the foregoing proceedings.
h M Mark Mahoney Official Reporter Ann Riley & Associates, Ltd. t I l i
[} V INTRODUCTORY STATEMENT BY THE CHAIRMEN OF THE SUBCOMMITTEES ON RELIABILITY AND PRA AND ON REGULATORY POLICIES AND PRACTICES 11545 ROCKVILLE PIKE, ROOM T-2B3 ROCKVILLE, MARYLAND APRIL 21,1999 The meeting will now come to order. This is a meeting of the ACRS Subcommittees on Reliability and Probabilistic Risk Assessment and on Regulatory Policies and Practices. , I am George Apostolakis, Chairman of the Subcommittee on Reliability and PRA. Dr. I Kress is the Chairman of the Subcommittee on Regulatory Poiicles and Practices. ACRS Members in attendance are: John Barton, Mario Bonaca, Mario Fontana, Don Miller, Robert Seale, and Graham Wallis.
'1 The purpose of this meeting is to review the staff's reconciliation of public comments on performance based initiatives (SECY-98-132); plan for pursuing performance-based initiatives, candidate activities, and related matters; and NUREG/CR-5392, " Elements of .
an Approach to Performance-Based Regulatory Oversight." The Subcommittees will l gather information, analyze relevant issues and facts, and formulate proposed positions and actions, as appropriate, for deliberation by the full Committee. Michael T. Markley is - l the Cognizant ACRS Staff Engineer for this meeting. O The ruies for participation in today s meeting have been announced as part of the notice of this meeting previously published in the FederalRegisteron April 6,1999. Jl A transcript of the meeting is being kept and will be made available as stated in the Federal Register Notice. It is requested that speakers first identify themseives and speak with sufficient clarity and volume so that they can be readily heard. We have received no written comments or requests for time to make oral statements from members of the public. (Chairman's Comments-if any) We will now proceed with the meeting and I call upon Messrs. Emie Rossi, Jack Rosenthal, and Prasad Kadamb!, RES, to begin.
v O S E E E H H T T C R C I A B A M E S EF . O M RH c R O E R AA n I C- S EB , P Y D,R Fh Hc P B R r of,Ae A N U 9 O T tiEn e t O S 9 A chRe D 9 e rC ,ic I S L i E T A R 1 U DhbS i S nncmd O 1 G A L C 2 E ao T A R oado O U i r B- O L R isBal v ,Kb g G I E T R A il E P E Dadn C N A
,ha u sna serY o R L it s N O C A I U osPt T N Ro .r M A T
F E
.RNeb R N O .E R
o O E C J E C F S I F R E F E R O P P O -
1
-. "t O h i
g s r e v _ O _ y r t o l a u g e R d e s a B-e c S n S a E E m H V r o C I f r A T e O A I P R E T o P N I N t P I L I h c A T O T U N E a o r D E O C p S E p A A B-R n T E _ G a U C N f o P N . D N A I D s _ W N I _ U t n G M U E L e R _ I V O C N O R m I _ R N e 2 1 F E G I l E 3 t R V K S 1 E E O C E 2 8 M P A I 9 T B T 9 R R N I 3 Y- E O L V 5- C D F E I E M A T R L C C C S O G E ! A N G R / G O H I A O F E T E N N T F K N S A R M A A A I T U R T L M H S N S S P O o e e e e e e
J S O M E T S Y R S Y E F R H E O O T L N T O S U A S R O L H E I S UH T E I C C G E C I V N W O N I D S R A RA E P N E ER I T E H HB T H I T T TS V G N FR I F T I I W O OO C S A _ E A R M I S S T E V E EC N V D R I TS I A O O N E T F A L S I I I V I E L I N S Y J R I O BN BA V X I O I T SE SM I D T I D NV N NU O A N F I E OT OH O L E S . PC P S S D M U P E SE N G P M E N G EF A O E A N A RF R I T R N E E EN T A R M M A HY H N O O M TR T E M I D O T R F R F FO T FS R C OA OS O A D D E E E NL NE O R N N OU I G OS I S C S E R R T TA DE A A P E R P S NC H E E I I AIF T L L R RS C D CE NF G N S N S N SN SN OO EA EE I TM I S O S O S DS I DV I AA I V S S FS FT RR E E E EY I L EC I GG R L L RA RE F EO BN B F TR AA AE NP I o o o O e e e j
-~ 0 N A L D N A DR
, N I
S P K D 2 NE D S T 1 E E E NIR N E AW H S S DO O .E U C A A I HF S M S E P A B- B-T AITO T HE S D G MR O E I EI N R E TWNG N V G SD RO OF P C C N E DIOISRO N AU P N F S A A A MTAI ETNP BL I M I i E C
- NE I
D M M E LI ARAI : N K H E R R CI S C t PC ER Y SD S O O MOIDP R NO A A F F NE I I SSMO OU A M T RO B- R R E ASNOT I S RH TR E E P RAOR AL T S NP C P CI OC EP R F A P SCF U ER RO MA N D O EEDDG HU EEE RE ER EY LR M A E M F D I TSTVR DA P PP S R R S N P PO N RSCIRL E U I E P DAI T MT O O A E EL I F YLA O F F Y V"N EEI L O CIR DAR I MVV R N P _ R EI T AIO I L L U E I ET G T L D LY E I T T RIST AG P K RU K A P )T V I A ON C TE E S AP D O "N I O I NL F E A NR S R PN C PGOIC E I U N HF ED A E R I A GG I EF E N B UNI TL P H NE K RA ME R O R E 0 L A SGA TE U I UR LX T EO N R I LD I S PT I RM S ES RA CB - C N R E P D OL _ C GT T EE ", O O NE I P TO A M EOI T N 1 CTN I O A FH I R SS 6 LC T P FE O S E RN D AA 0AI LN S E AK T Y LUR EDSE BB - 6GS
- I A N T TA ES EE 9 N PT WM A I
ST S RO S VA I RC YSE I ER L H SS I H EF FBAP? T AT H DN C RC WO P W TG R NA E " F ", ", CN A O E TEM I BA AM S ON NE
- R 2 . 4 EI RIN UNC N I
R TR M DO A 3" 4 L P 1 S OONI Y NO O ENC - E 1 I DA
" I ADE G EF C" D PE 8I 8 T (S
8S MEHO RSTR MR S E S )2 S E CTA 9T YV I 9 Y-2B 3O 1 U P SP E1 A I GI CIT C 1 2 6S COA F D BLT E , UI SS B- ER EC E 8R 9E 9I REUH - SD SDE T P SA S YT Y D ECOG SE I (C AO SY S". F CAP NWS I AM R NNN RR TP" ER EN C E A A R C O OOA U UO SS EGD MWE I STM I I SP S S O SI T E S NE ROV GF SAR CAR O SA T SA RV EN OIT RE T I I NIDM O UR FOHO TI - M LO I UF NR C F L FU F L FU F A AK MLO R RS MGR E E A AG AG MIT R CF E TI R OEE H HE TE TE RI SI NN Po I S" CRP TWR SR SR SI N 0 e e . e
- e *
~ D D E N S D D O U L A D E M T N EM F DE E O EC C SE NN N E N EN R O MR A C N I UI M O3 F - 1 T ES T N OHM I D S R AI TC R A A N SR NIS R E DIO T AR T AO I M I A ET K SD F P R R E NE O
)
S NE SH P YS ES R E H RIT S F D"? E I F S" L I CS ME P T A EEM I RI S NWR LE ER L DBE OR O I R L L E P Y E DOH ON C PD P U T L RO A AF N DN A T A NT T ET H I ED TA MNG I I M AE N PCI T AT HIO N S NA YRO TTI N E AK D FAS ANW PU RO I I H NS N TST I EJ T P AN WDA DE C AR I A I TNE A L S NRM EEI I OE E L D E AM O PC R DT L H CM S AI T LER R AC I ET O I EE A B /O P CS T OMC W NO I T TIF I CIS D P NL I C TAN S AE A E I A FRA N EL DE HN OSN D L B FAT TE S S OA A E OSB S E 2 3 L EB R APPE I S UA NB A PA I N S A ( T E OE S CO C SET T GRO 1 S B-XR H TB I NEI C C I 8 EP C DN A M BO SY NDI TCT I E 9 EO A . EA MD N EA EU RA C OE KME O LS O Y-C E HT TE RC PN S A R O C A C UA E OW GM GS U E MLC ECERY N A M S FL PA 8F L H W F SH RN CO I I R OB AD 9 R P U ,N T Q O E HSAA A I 9 T" F . AC 5 N EU EO N AS DA EML F YE VG 1
, PT T AI OTR EO R RU R HT SE I
GM SN 9 E EA NO L MW OO TA .D TR NP EAC OG FE P LSL CORR I T EI R E P HH AT HT 8 9 UEO EP T RO EC AWD 9GUH SA R UPPI RE C A TN R E FY N 1 ESE ED OP F R SE PP OBA 4 RSKA RE SA EI TC O NR M N S I PS E PN R DAE OA OI O DE Y A EFS DYT FA EI P RA
,H CF D NL E ABA M ASI FB A - DS P RS OL T
RIC AO DB LI RHAT TE EA OR CP AH KN OWT S,NC ZB RO NN TOP E ECW
- I TIO I
PW OIO D 2A NE P F RHC E GCPS N,ES I OT TA E",C NC N OS TN 3M 1 R ONAR OA NT T DV YR i G A O PM A SO 8 9OF CA EE DN EIT CE i
- N D S R GS I
EM RRI N E T D E I YR t I SC ED LD E E R FT NS OI U CE FO F M M FE I U E SS SUM FF E A AR D R CJ EN RLE OREC E P F L S AP OB HO CC N E DEE S N TEOA T SIM I FO TC AI D EPMD NO I SPTP O e e e e e e e
E E
- d n T H a N
'O V T m s s _ I S t p- ie et E S ns i t M E N oyceki s c l E T D T E ns niarf L N I V N E H i nf oo c f odn P . A E H E ta s M ES O R r emh e na C _ R M T cr ,is o , I I E P e t t OT I F _ P M M dt sae TIN I N T O nn nt ns s N A R O aio ri e e SU G O T F C saamc t l niue o r t ET I S I S S amipp t l CR T Y G i s D" A m, FO I l I ps A C N , r eeis r E FP S L U I L I K W erl waph i r ea l S OP I O osmt e A S M O G N . O N L peirt B- S E P I H E soa E AE i N RIO B rt avI nf i k m E C O A T eit r e C RA T D T c c .oh O l I C O S N u atywt N GI T E E T N O nefeef o A OR R A S R T O I T o es sam am M RP P L U AI BD N I S A t nth f of o l, r ur R PO R T H
- E S T O E P G ]4 ES L I O it aai nl ct et ioip be etc F
HP G A I E 4 C'N M U x s R TA S R - 1 N V M Q l ipd pni anf l ellu E R AIO R D I U O P E B- 8 9 MS Q C E H aimaf r o n t r etse O O FI N V P Y- RIS E E T f et r eph t D S F O C OM N H N l eedf odss , I Y Y 1
- FM A T bfad hea or i
Y SE R R E I R ulet Cc D AH E O t D O O i S S F sioRa A BT N O P C A O E S n ht Nn s E L T A O T
,E R S S eWdt ot R A S L DH E E E v
o . htohaia N C A U R M ET P N R r pe)sedg t l u E I S G E R MG A I L P eimdrr ee H NE E P S RN P T X vot ci HH R A [ OI E U E aiiv hst sr od W CC R r oips e E P F T NN T O S e pl i a D T A O O E E I H D A pebeb N T I KM AS a rcbb al - A ER C T SE W T pi o at ec , HP I H E OR E N isgri opu n W TP A _ I RP L R E hl s a PA E _ B P t of H O W ine oym R "M NI T EP A T N gnl sfoiur o afo l r N OD E E D L E OR ES H t S H 2 I p t aqe T1 - S eAcep E VA T RO U cpl ipely E D B EF DIS N' n o , pot C ve . M O P" L E O cgar R E T S D deepmN aR L O AE I . I S OC N _ PN n(hdil uu e W UF F S ast l t N TN O E O S O SA I F h I T M sinhedo ni t dt I KM S T S AT F iova s nyab K I E I A HN M it cti ys S ER V L HH T O n t F E aind,d EO E P I x WC R i S FT C f er eioete S F RM - U dolpi t m la E O E AN E e t cmf inr ogu O SR EA . TO H T H H a h e oef e D EE HX T" T SC T Tr cdir n " RP TE O e e e e e
- e
M D E O E T E S E G ET CU S A E A NT Y B AT I R S C R DN U MS O R R E A O RN , H C O O H S M EC OO FC N O F T R HN A - I I LO TE R F I S N T O B F EL I R I S O M A T R l L t- L PE D N P P A B S E EI AS T O A O E P MW I I C Y T E R T t - F D R T; RO OA EO E E A S E C AD
/ TH MD S
N S M I WT A I XN EN OS ON E . CEE B- R E A CA HY TA N E P SM NS A ENR I L S E RR AE E RI E . C N W N H EO MS U TC NN E A T 1 F RYL I MIND LON O C S S
- t R O A N A NO I I TI L M
R A E ME AP F RAA EI R; FLC A S T R N EI Y L LE _ R E A -ES i O U UH T AE HW DITM E t i F B PC GT R L PE T t O E U I R )S ST I OR H LA E,F EY P S E i EN SS TCO C C BS RB ND E R A i LE BC EIN SI Y ET I HA E I E H DG AI SM TCU WIR T DT T G L L L ANO I AR N SA NI N N EP F AK UD OE I D ED I DO O I CN BM E I UD W S DE I E A 1 S CM O LA A T AD t XR V T NM L A . C - L CY EO O L SU M C NE NN F F I I T L T R , R I XEI C A MIO ES EER P I S O RI L ELN EN I RIS F OI (C TT H VP M KH A I BA R E I A I E OC H E A RG CI H ER DD RWIN YID T FE T L F S O ,T AU C A RD E BG EN SHA E WN L MG EY T A RIN VI ; EI SW EOU SY R TKY I A PR A UD R S BLE MRS l RR CS N A R ER O H HO R SU I AE E EO C ST O A L J T E TD RTR I T LA I P ECN BNIS C SN F RR I t A B L R MI OOH I LEA ACO ML AU O AU TG C RG SE N AE ER I o o o o PR O e
- e i
~ H Y T R
~O S A L
E T C E L B O T E E B A S H E " A L U N F S T U C A F E S P HY
. I L C A H E E G CLL B K" Y C M C E F U ,R D A A C RO I
HA PY E O O A H TLT WT M C R N COS R T N E O T NWA S P U I P NE OM EEB - E H P U O O WUS A KI SIC ACME C A T YE T E I P A Y D LR AR R FF RN I R E N P E LC O V M A I T UN EE YM FA R O . O G R OS I H G I U E RR P TT R CA EU T DH ~P AS P A _ W R B NT OO ^ A LE UT M M O G E N U .TF D T N _ AR GG I I O , F S TDE OL A NI UP LE E S E RI N R O O YA TH E H A N T I L P C G D RG" E EER B- DE F P LC I AN BY E E " E TRO C A E D C I SA O H RA / C S R R - N R C U S M HN ED N AC B A N NM I R P A S O OS O NRT A W . P PE M E A C P TS E AB EAF" R C E E SNE H CDL S EL OD E O N F R
" H E EH T
I A CE AO T R MC F T S U O H UC M R F R MS O T S S' ROEA O I HNN E R T I E NH CF E O P ON S ) E I R S U P Y 4 _ FE I _ S N MS AO R ElN F W RM A ( E N QP R O O EI A R EH K O EE W T E EA I T I R P SO S H P
" EL S U C R D A EE P U CI R E B I L EE C DT AE D"R AH FC D I
R HS LR I ET OI U ETA LH NG S I . L i" H .B AC LP SA L iS NN N AT A P _ AE TSE - . M NE CE EB C T NTN C OIYC YISS D N NC OV UE I T iN UE I i-I I eN I I N EE N TIT BH Y DE A I DP DtE D H I C YI - I R E T T EU GM M _ AFTE A T E NE I I AR OS i L DQ AR R _ RASV S OO iD I RC TS I NN I B DE HIUFO TS RR E AV I X ES C QR EE EDN SF EO BN HR HNO P M P P I E HR E L MO OEE - TP TACI A TD I TP F EC NRP O e e e e
- e
- _
~
,llJ N
Y 8 AY E O B D E G N I 9 1 L 9 FA OLE FS O'F L O R _ S SD TF ", S I I M R T NA A 3 O P N 9 . E T YA1 P O A E 9 R TI - O R N M9 M SD N OA S R P I E1 EE E T D P R D LY VH T LT T AS LEO S O E ER
",A O NA UVT A F T GTD I
E E 2U VI Y D H L 9 NEDLL EAE C 3 N RIT TA I I P 5 JA E R M N UA I I NNL E H A O RN ALM I E I T S C CI TC R EDR E E S
/"
GT BN OIO O S EL R R A EH N USI O L U W RG OTN T SA DB O T UI DA G N S P D X A N R 9 9 N C RE A E R A SE 9AN AC DNI C E . . AV 1 SA O TS E HM DO HA H NA H H T I LI CR EY CE T AMC C T MU D EO UR RDN T S RN C TF OI R E8 RT N T SO AI O MGI DF 89 A J1 OS E F S T I NT NR 9 E O2 F E S A NIRLA I
- CA L AE1 S R6 RS SRD AU I
SP , P9 EA DOU O E 1 Y WG ELP EF R H Y- PC BN T E S HPO DORE CC O T O RRS SXP O NB I S RE DT E C OM E AS D ER ODE I L R S T PEC BRE R EM AO C O GISE S P ESO AED N ST E O W ST U E RAR THI I R R ,BP STW TU P F T CLT D N R - RC E AM O LA WEN ER EU A OS S DR EU JS E CO I TFM P I DN ES OE CN VNI T P O SN O I T RR SI F EA A I RG UAG AE P A F RM C E YFN SD N I H E R RI CT NT I FEI TT HS WO EOBL NI ODT TE I NN CU EFU E NOE UE I S I R D A C E C PRE P R UIT E E MA M E L E L CM E N EN N L O RIN S J RI O SA OA A PW FMPS
,S ES RU PS NOE NOIC OCI ER R S R".
7S RE RT N TD E E EY D R DR 9I S C LRL 9M FH HI E T H Y LT 1 M OC A CE RH N C AD B TA ECP AO I , OS EO TO AT I ROE NI H WYTHU N U C NR EH ERS R NSE HNK EI ED HIVKN J E EP ST TPU E T ELTCAA TTAI N T H TP EI FPA CTF I NA I RW AAC N TPS I N ASO I O e e e e e e
T O H G I S R E V O Y R O T . A C L N U I G E H 2 9 R C 3 E 5- D T E N R S E C
/ A I
C G B- S E E R C D _ O U N N A O _ M L _ N B O R O O G . N F N _ O R U I T E O A P Y T O T N T R E E S H B E C O R A R P O . R R P D P Y A B N A F O S T N E M E L E O "
E O C N S E S A A I T M R S T I V D N A N E - O T LT I M F G M R O AC N L NA O E P R I P OG I N CE V O N TN R I E CIT N F I I E E I O I S DT H R LP S E G DST C BI R T N AXO I S UR _ A I PC E D T A G E D E K E S D P NMN H L I D I DOA I T ER O N C E P H A I NR , G S Y E T EFS N 2 3 K A C G R A P PE C XC EN A R l Y F Sl GR
.L 1 T N Y
I LS NA S 8
- S I Y B EE EI H PE I
KS 9- R F R RT MIT AE _ Y E I T Y OE I I MC SV H N T FPN EE C EXO I I E T E R BEI S DT LN O UN O S D U T NS NC AA RU OEI S D I O N A N SG T A I S M I S TM E N WE ER M SE AM NI T R Y EI T D S NA R CIV N I RO I LC S T I A MO C E E NS T S FI F T G MP DL I SN U OC N ORE I UE EE D A I CO C T H GS I TM N MD D E I AE N E HI NM G R I N R M RS A O N O UI OS GA T THR I F TU RE OB S OCF DS I RQ FV - R TIHSS . VS OE RE E NWT OE PR NR I TT UA PC D A E RC PR PI LN L N LSE D PO OE NI T A A M U PEML R R P RH I N ER E AC R MO E O I TI V T U OH I D V L O O I DW I Ft H C E F R MO C E K SE K I LS O R V E P BSDA NV I O W OA M UENT OE OO SB NP I I SRAS CR LT O e e e
- e e
. N ,
O S S O E T _ G V O N I S U D" E S C R DL NE AS L I P ( C O A B- N S E E L M E S E E R E C UH I F E V N T T C . R I Y N H I T AD S V B T DE E CLE I A A T O U S I S C I _ E M T A NU A V R O I N GB AR I O T I RE N G T S EE N A F R E D E OC TH I I T E I T S SN NT O I P A EA G-AA T I I N LM
" V B- N UR I
I D O T A E MH O T E T C T C RO ET U S " D A A N S . GF HS M P NR O N I A B- E M D E D D AE MS I DP E T R FA R F E S N RY R OEP R E C R O A A OL AG NP S - N B- FA GI N N D L A F N E S L RN EE OA I O RB ST S O C O M O .EA I N SI S H E R O K S A ODPT L FF DO ET UN E T,E I L K F I R M OE RD CO SI E SF S N I A R " R EE I T T DA T E : N O D F T AN S P D E F O ENE F R D L I E HO-B I N S E R H TI T OE EU RD E DG PE
- NAS T NES O I A W P i
SV OR S R t LN CO I I
- F M N AO ERM RO L
TEL E PT t A S B O FO UPN C SC O PC SS E U P SI P . OITMO P E X E NO CB AA ON S I HEH .YAI OB FAR T F _ PO S SHSSTT ECIS M RO ORP AE OIT I NCNMLN I PT HB RS F OAOAB E RC I SUE T) CST S S SS I P E O TOT ARARA M UIT TC I I I AO DU EQ G GLE OP F N A P Nl LPLOA PL SE B I I N CE Y At - E T HILT OT EPERV EK EM RARFAI RSM T F I I L RA CO P PU OR SC TAN I L1 M PAL Ot : A OO NUO M A SS S o n oooo ND OQD I e o . O e e
O S T N E E T M A
) E J D R X
I I D I U S D N Q E N A E H E C R C P P D C A A N I O A R R E E P T L S N H U E E P G U A I R S G P D S E S D U E R C I N O S E N S A R A V A S Y O E S G T B- N S G I V E Y E S N G I C :R 0 T I N T SO 5t N A K I K C N T Tr I D A R A A A R Aa N M O Y O M R OL P FUO M ( E A Y . E L U W C N O F FGT C S FS E R D E R T EES N T S
. I TI T E N G E R YR N E C G NI R A A P O C RO I R E N EV U E L R F N OIS E E J I DI T T E A F TI P O I I
C U i M S O E GCV R t O T A X E F i R E F E AAE E P M D I M O C G TE R S T S N R E E D R M N NRDE R I N I E O EI O O S S O O U I W T L E NA O N I D S C T N Y R TMI R RI V R P I P L I LB E - E G N S E A C U S OT N O DE C N I R L N E C O E CO N FC H I I I P E T E E UN R R T T G EI N A A T K GA P E A S A ZSI S - R A A E R E NV KS I I O P T PM E T O D E P I OR C S P O T G EISM R C S LO N R _ A O RRN O I T D EF A M O M C I T R L VR H R E I N I E Rooo C N I A P O H EE DP N E O F C N S U I O e e e
- e e e *
- e
e O l Lu 1 Q
] ;
0) Q. O 3 M O m
)
l l l O l
1J l - d a d n - etnt se n o n a nis o ae en r it d oiy O sm ne m oir le t o ie dt aa rl t u hd lt nde aa edp et uyor r s cl r e it up t shdn s hn e cop laqm i cno o ic mfotlai u rd g ei : aaip r o ,i ft l bpeu f e r n n t e a arpeddo spuatr uir puq agw eo t r e c D oeyrle nrd eue s at cn a t er scp e o lycaI p i f atsut . m N A k s f ohob t f d r el ke i ph e nine i aialptn y r mtarb euhoi e ,v wc t R hed e : cmmie rf K ut dcees r t r E t r og K it r se nk s
,t a R upume ner P f of ei n R i t n e gisl O o s sa d A
P yr e n eb O imdir r eeer s la e W nde a nr n f t io ah ehdhnf vo gotont E M Te .a r W E t t e dt,io t E nr ibde r ea r l u T .ds n e ntsh e M u eteh g M nec A shao cc er l ae n I H t c o n el ai t A omn os lye ais R nt e nodt o W mel pa R e a u se t rt F inel iva ns l er gu ae gt s es F rSsus Y luGet E r clye r a y aqa f s bnc h H nk Y l dcyoo R st.ntea T icd eiue R s el it wtr i olat e O esas or cicisp bqs O aPuyn T c x u esnp N t o beb a T b A qlnn A stncc" eoa I
- s iRr ooa S d e A sPf sm L neruadek .
E t etoc cn n L n o ede s e U emo e besr esa o r L en a pam U ih it t a res n si v G sor veubw P H xcr et o G dn inde n E harpdnec s - e n e m I E ni R: C s af odrgd h a o n ar O C ceb a aep ni m r A ih kr s p e R TT iord t af N t r T eimlyr sr NN t oedma I R O otn /o N n cgcit m e s nl o EE af nnroh luseaf cu R w ed i geh P P e m mn ea E icdreipc ex a RM r it t
,r se s p RE l
G P ard R dse nAnsin e e ou i n g nie t "r de N A f ri y ue eqm R a .gss i i UV t t hot r ei it r v /o vi I Z L r or or U e s n yl c n rean us t l t CO R cp scdo ape an r p I A t a ef C n a e oa e EP oobda r d N T luiv -n i E dmf r p ,n n" pls a adu l esie c HM t A N gpk H i ur TI aact eo c ir snmw eis r ri gi nsa t t t G E ds cr T ,u e in r FR t t e n er o eel e i , R M sqemig mte saomf n O E m ph ri F O n o it eiei r e r r n OO i i nmt i ai pno i it od nd w r uk a E R f l aiqeo vut t S F mrluqsn r a eegeiirb SS t C in ae E upedn et H ecs e lcr n "t m r s e ao
- n gi rr t T N k ci stia R s,d EE dunh eano I ist ri n ni U f orehu s CIT laf t,ien icec f r -
N aia T y pish e s CI no dnf gi ri A omm t r A d yb T. ly s. l UN io e t sreido f ntee N oba b ee ds t .at n n SU idue- s y oeb na ev T vk el W ido it E t nt o ER ar e oshg tet h crTn is i O sn no e t H enniye r eoct og r L a v T r umtiut n H T O ht ie nsts o gd L rda c elat lai r P letyi t ne i ei i v mii t t ep gusua eh O esl N l n gi NP hf t ea p p kl F h ai Tbw l O hmecrin Tir oe it OO Wsat maac h e e
- O
;l)jl l i' t
y c e d n e r fo wle b h t r c a n t o ~ h a of ef r o e a t a p it es hre n ede a t O t ir h nl icnnl t gd de c n s e e t e vnrop e r ee eiu nol
,t ea ht .
r ei - , o u i r cr mi l r e. m rm tom of n pescd e t io cml r sre emha f o sr (if ai otycei e t l q e oneaa nn a eeev sr ar l utsde r t ptrhee iepeboie i s nso t st dn d sf n , y pmdeo aee pixdo ue f l u erdt cn eee r -e onf t bmhseedcc v l o u ein r r et e sf l todsie n s i
) ya s ero aie nnl w ahaa D anf d fdo u eoiaoa l i t i
a atas vpn eh h eTmn e T ge im pl u ahiamibt c a m . y n,t N e r nn s catnse r l r) pelco sdn o ab t y y o r yt ioia n O f ea c n n oferd ei ebaca l t ie er al mde p p y rli oiccm t C doem t ha l t aau pfie m t H" t l o r swiwstp aro sf r n e scut et ne r t ( nf i l e C al drier ,eo nt n r sot . R e omtna retsy A a e ccdl eted ais e anie pleoc s i sc i h h mccug r s el anbm a t s E lanhq t e ufo s O t r a e ce r u i3v) a o unns n l u se (o et (, es uc ,t P gnl e ye i r c t n emhntis et R r e b c an Sn c ge
.a l
A sn P d bnboe y ar dn )p e aioa s eiua t mleie se sm l j r d P vt e ch ae webef c e P n ar ac otso u ul bg t r ho ias d nl ,c t ded nhte eed A a ut )i ce e sa2hnilay gf i r s s e r ega s f ut aa vnh c D c a h (, eeh ur o l se np c ct c cf e e uhim s akeiheha net r ecl wlct i c E i u oodt x H ea serf ost e a m )m s cni nawn,cu l ala ks n d s oe W bmsioo S r t e amt or g i r a snnt yeeind t t o na anf t A o (1mr emr o ht ter (o er ai f s ovmha n E H
- t e ciret saio ec yn B-f e
r s: a r of r ieo t oi s ey lpe et mdl e psee H T u ntsn cam t eca E perf t aesrl ep ycb a e s la un e sedr e T P i C suppr a er c hanu ea a ulgni- lu i t r eb t N I E D-ngt uqo oi c nc sed r ef der N hid it st r ee/owcun edniaar nse hy t t ha t cief nia-s sf ame e t n t e A l l l i n a el ap baas age n n air uf yy S f oa hl yn o ese ego mer l l N iwku M r hs panet O t E diet s sha u sinr ceir f o ur af n R t L I nRhnt ewfolisr t ehn ic wis st ndr ea E t enid t alordlycpTi t loa n o gl e P S a esgesd .hmov O l onae a .u d di i I C n i F ht fo t sanc t ein nm h Poet r nintot N eiul eio t enn i R eeetyant a emlc e en ere yt dra e r N E nh ri c fe en epm no o bihe l e E ot ei tl i ci mt t s st c srm mn s ct s h , e e I R F saAew yf l t P ieinaio f irnf o cono adpr E p d l f ar hs Sieoe waelet Don P D
. t ha t
cr ogme e yi f t t mo n
'sccdcbn l
E aorinr prted i ei o a yi o G F alcba su aht st H opede dl t u f n i ed C ,l l f t N anic t n a poelueeaiao r r t be s Rnoht oosd I O h gct ac eic T pcmcdhf n ,s eee s c na Niwcn ec ef e e f t ab Z I S 'h i rnaf si pr dpeyn r e t F an yd i a aisli r ,t bt ah in ha s t c e hn f t N E t eat n hod O onpmhgt r de hb c t f l f ube t t o n A V t
,t ta alas eiscec aar oa oei S n eiml i
dgel a t s i t G I T nret xiug al h C l u gskh c yi h ewml a oi r e t n mlnns ue egu eaie i l t gns pi t el yen i R - wh i A I eiyr st n m pr s r etar o r T r k ahoirt es nr i ao ac miyTq e f O R nlah ssh S des m pin oomi t netsn n f de et .n E eut bet I ec s n le gfeityi naai E - e amscem coukd i P f eit saet R a ot nhmdeto eluor xt i aa l H M dramemin E bif - s k s bm sat y ats T f opeb hyseg a , T eioos ct t awr or efe l eo- r idaf hr s I t pldt ans c e C c en sbohwr s t cf r n het a ndei ape t E eegmfehn t A a xd e oi t pt s er e a gr ec o l H cfr nye ega i mrymeeemt en m diu it uy nit n R ot rt siauir t o r o e) hn e i T o aisdim f t s nr A fr l a usl i mf ia t r oi v nmnai s l c N c e n nhio euar aeaslu ng sebr t f t r c ee i er ef a ecafec H l f ukr pgetet soyof O h uue T n qd m i rp se C Ar e mni e d (4 cr se) o e t pj Ao b fet n sh n poh t n e ee oi t sa a t
'Ddor i cieh l e t O o chp apois
d e ,
. s d e e lu n c ie.
( e i l p e) , f O i t e n s d pi ei l a s c t n l uae
- pde na n at a y e, t oda t sr cyo d r (3 o m s
e sde snb eay c sa i t sd at sms er o inSio ia s st nt c t ysai i l c sm u e mbi s r oh : c e egahmior bu i s n ce uroc
, .l r
o ee ef cer ws " x n f co npe H dvnn yok a cfeoed i l spa e e v e r e p a r nd a ae C rbagn rf i e sm mf s oa er
) prmo A a , mn a pd ro r i m gi p s 'e of e
t u mfo n f o r r ec on e O l u 7 n- er v n ne sr g6oeonom t ri e ur s c ep i eml R i i N petp n poe e rdige u)d r inf si rap cr P i e e l O ham l at uuh cs o P t o nc ae n pla4 a d e d v (, w a io i hp f l eo C ( t t n are tc ot A h5 c9etsa r , ne e er t enr e ahiu adn a r c pou D t c q s reo oshhgsfondan , l R ,
.t e e
'e si s E r
pmoinsiama t msu t E gor ENC
. ee ec S p e d n gi r r e t a uf x cw P R sdr n u A ainir ateog efo o A
( s?)N ee chs o B-dne i ak t i
, smif r r a cer pu r
o n h P t sdnr y l udh i t l
, e ae ehteve e o Fed hn sei ae r it e c t
t e ri n w2s n nsC E bssmicf
- n g. .t t a
C eueha hds s i era er z mu o ssNeR N cccw fsj en el ni cs clu a er H inr t cac nsrionobeik c l o od n oiwa m et adf rh nu s ssd A i ps ailaio ms on enhoh da-l W h cp ue n M r t cisi nhn t pr pinmc e cda oan , at aoo t t E mh R f et norbat al oyis e c on oo r f emep r oas maic isc ed H e rl ca h ri cbt n O pemprpt h pn desao gns a s n cetewe i t T t ot op a ee F ,l ef m oht )2 sy ad t e nie N scdo rt a R m n sny r e"d esai( dn p r ise c na e- ec pI n e E i o uhn I ene nno a r P os aih seirnia s se , gat t a S t f a d eno m O de )s. o nb nt r
, i r E ht m elu cihn e n t t
, i
- nodi otvirig cio f L aat n s t apuus D k sciaat eoini t nce t
etar P ome e r bt c uo E incl r t c o err pl sup e I p peew ch eof M ae uha Amp se ptnr nf gt mco ng n C at ph i
=e
, c n , oy r R : r n rdt er aoas iei er n N ds it aeiet .t o h oid f mis bdw O
f I eada m (.bc c r ,e n osr a des o R swr a ,op i r r otsl is f a F a pn o egsipefob r " isb emt a-r r y luad ob k P l b- o f N sieer eucwi a o dn h - a
.t r e gnp r t I
n ne e nt mpa see cr G cel.ap n( pa pdths arogn K A ac nc ees f or adm ei r r o nb N ae i adps , pisa S d d ,admahp
" l I
Z mhs ou a t nn rh I e s egmf e nr ps ie nmh a r t I r ot eo t n eov et s R a mio ohlel bh cof N f r e mnio mn o s saye
" B - f rk f oar ep nat ss l et bat sfr o eee A pd s c t
sh rl D e n m pic o lue a ips t c i va u G a,an ot ea et or sr a sel t g o E n a ie - kludiolahu sr ept ct n ri nc et sl i t cyc a R ns dn e ah t ugi n N m i r n s abt o "eCmee oe e o rt O ie t osm cctue m ea er rlen I B r r Roht e r t ddeero eocc s ), i c f e hNf r t ae mhe sn E pr er n sto e M P et t soin ng sb l uaort n t nfie H s p pf o e eh d O ki
- eeh t T i nrensn ovie ic l mcs C d nssdct i uo swo u i l
cd an d t oo or to tcoe oai r c E e ibprl r mea m c , c f aos n) ehu c f i oreta yti (1 t t d)h f dupn eopa o o n me o cirtr s i l e? n cd e ipdam l H f n c gcceto ol t n pi nf e p ce poa pit o e pedr T i
- i n eim,lo m )5 n se d n
acnp e r n aieo r k k s c gd e e r( ah u s uph seaessf r N is ae cdseen u u evt d y t go Af at Adbp O Rmh
" t sjudda r
en vi t t dd t eeehdoe N omt e o i O e 2ima n v
~
l ' e y i n l eo du h g h e _ h pr k ut u or t O l t l sh s cnh o? t s po rd e oh ot e a sc r a aw t r s ct t h dg do aa r h ol l ou p h w es i _ d peor r c t yW i pe co t r ae apap u al l ta a p s it n ub sd oo rf l e r v o hdr ya s a u?e ouol pn pa ey r t sstae eoi a mno t dc ac bo nal s obu gb a t a md o et d eC st ela _ i y ce sa sR hu _ a e uht cg t cre-c r e o e b e aN ae _ l unr t a eo ba r b t or gaen l u s ns t de t i e a r r po emha r r t g ed l ui ch n t pt a ca do om r e r i cee oc r aty nf e o r r l c d mic de ae r o dn n er on f s ph er ri S e t c nk aro a foi r c f se ah _ R eat p a i t , ep e bt nre a e Cw ah l E u aAg r r Rl a mc ps e - e D r a ci s t oC n n o Nn oo at e n nv ri e L s .?is n eo? f r np aer O dyn us s htdia i l i p ec m r o e c a i H h otuod diet s e t ndi c hi wfu f ot E t e r- ass r iaf e nla s s f s r t K l d ) ar er pof A mfou - gr e d? ase tata n ah e me r n eh gi t f T ks sed srd oc dwb - oe nw a f oe S i r e ea t i d ,ts fr g h s ng R gns d v e ntoo e e c ae oi n t nf e e e anc ph era io t O sd ag i O i ydt h b un sf e e t l u c - e r o po n l l I F pnss eio b ele rh i l S pasi a yr ew o? yn s h u oc p N e bt a sd i t seo
,l wrc t u ns a he ntidk si f he ra d vi oy et O i surdl i
oia t ai e if h r I ec ad c h clu t r ct ee t i it t o T ad o eg apm dn g0 r n S i t l eolu t ef f e s de n5 E u styc o p s a mer y r ei s ad i vt c ea , n i or U i f pl ( sd oh e ie r a f i cwh c aar eee t at yu l s s pP Q d n aeh i de i rh s u ca l uf r a ae t n r v et t cab gi t e el mR a uiw i sc t sa uo- en r eh r p a nF i c sea b am r r e t pc l df t ye oC iaf R f e spn aio r l y0 n gt yep f e ci v a aa r n oletn ob t i ass i t v 1 _ i i sla t o so n aac t f ndm i it be l uon i t crm t sero t i i nu ce dst n gpa ao _ mio n ar f dsi ao e l eq eb l rd ef p en f e sr a fro ra i ebe c - p y ph rI y n hi - baon ef ie f f nIf t n? ot t k eh v c me po func o a t a a vc ei ds l u i r a u a m sat s e? r st h ei a ng r ca oo?s t hse esdl y an cis eme r r u ee h gr e e en ht sst es e oq nu hf a wn i i r t t f e avd hf eh wr c h reoa r e oe t r e he eas e r ei t ofo o Tapr Hi c l i spt rh s Aca Abw Hep f r O e e e e e e
+
i ll l
~
O 1 "t h g t i s d i r m t oev h c S w hc O . o a ry Rn .h 2 o o Ec 9 r ,o r d . 3 p a p t t nP u o yloIn99 c 5 l u A HD Bb gH1 ,9 R n eg s . , d C a R h oMk r
.b e t nC e u n o E 2 1
O / G fo d t uNm .o e s Y e T Ni l r E s se A R,D P r WIE A p R t a d oF RS
. C U en B l bi o n Nme e-c go n
uB l l n oJ . E a Y m r W f o R r e P O
~
J l
O d t o r e e 2 s d e . o a e s e n n s p l a o e a a t o p ed i c e e B )2 e t a d i t e u l w r g e 9 e m o ht b o e c 35 c o n s h D nce s ";n g n n - a mb e o t i y s a R r y nh i yc e oe d t at t ni tv m r /C fh r t A r t o n i l i i e c R pt a b ci j e f o G e e pr P mrop i xf b r : ef e O s nfo h i e E l c o e c s .f r y P R i pleb
~
i h t mtn e r t e t tob
- s. w ai nta oa ef se r o U o ss em i
n h w npo ur mr gSt a o T (N wolp b i " o s a ms mno t s el h t o a at e g r g gu c h t h s r e oe i frr c e n a e t a p r O a g o i wh g c n a g e ot ua r miTR sd y r i y ui n v s a e a : s p rs mle sB l t i i p e i r a mce r e r eh e r t o c s e cd A Ov i ma st n d e d i s n d et s ts s mtumpe e rd oe r p l o n a e o ot e n y d ol i i t t c e wioc mtoto n u pe t t t oN r A r e n l a s ul e t a e r ns o o er cn f t o r e gi eh r e i us r c ta e qe e id oar O a l c n o r w e t t eb e oul r ui rd o t s u s _ s u c s)
, b oj ye s e ed ge s t us _
t g as t o r s pA n s b e omc nd a do oac r r e e a ya e ge s nf i t oog t n v i yO m R r t of i t : t o s l wa i p wt ci e ut c ge cd e e oug n s ge o gh i i n ef al l r Ho doi emRs a e c oi l l E e H (o i s P a PK h r _ T - - P - - - _ O _
O 3 e y l l y c t.e e a nf a a w w d s y a e l n e te l a i c a c s xi i n ed p o e y n t a m t i t a i o ht hi c n m t o n l u g t s e ua e r t a s at o u s d e - s n i rd e n i e s a e s i t r o c sp c n b-e a c o e r : e e c rr n o i n r n a s ie s o n u P e u a l c f r r s s s n pAfios a - s nt e s s r o o r e o i f n sR e e d i s i o rd sP ps e i l u h wel pu e r y ph ful c n T f t t e o s w o db ot da de evses l e o s uC n o t omrne e t os s e d e s e r l l y /y r i s wh c wr od a a s t n od t d i u o f s a t n m c HocoHad s pt e m i t e a t o u D St h P S o l
~
e i v
. t p
O i r c s e f 4
? r . . o g P f t s
s s
, n i
n O n s e i n o i n o e c s n o . i t o a r l e e m l aa t l aac t d a n i . t a g , t n p r c c c c e o i m u ii ni f ii ni f i el ri l e i f i n m. u qr v t a q h c h c i l n e . i x e c pe c e ai l p n E R e e p e u uar mS I TS TS RS QT o I I M e e c s c n y y y y s e n a m fl i t fi i l t fi i l t i l t n e r e a f r o ob s i a oba sl oba sl fi o b sl a i t v ht r si l si si ce eO m r P e oa Lv a o va L a L a o va si o va Lai r
, fl u ,
f n f f n f n f n ERo f o o s s oU y oU y oU y f na y oUp e s eet c c a i a an s e e , nnt r p r c n no cn n cn no c n,R e ei o nn e l e u ei ei n ei too t a r e e me m m s ut ut ut ut qc e t t ra P a a x e qc e nu r qc e r n u qc e r n u e nm r o r ii nnl aa p g o r EM FF FF FF r ui FFT E r MMImr P o F e O t e c n a n t n e s l a n n o i a m r o l e i t o m e n o p n no a s r m a
. i t s ur o c
r f r v e nl t s i a m o lI mi t o g t it p P e L F u y S T r C l uc A P r t s c n Fa o r I I p p A I s - l - e R P- - v Ai S iY f E ,v - e - A- _ L - L 3 t _ a e3gCO4!3sO o$2* h W _ O w
O s n s 5 i o i t n mr el a t l a t a s o p styc a r v s t n e t g si t u e o g t n v n n e r p mn ei e v o s e ai e c r h t a p r v e o O i l d s f t ,t t a o p e oi nnd i y e v u p ht go aeh h n t o t t r " a l e r i s mc e y o i fo s e v e f e b l t p h e rd n o a f d s aoo sd e eh o e s e c s c i "p gi ss v ni am s e i g i l t ua t nr e I n mf s s a l o i b a at t e o id o r o l t bo"h n pad foy i d c c l e e r c n i l r pfi s gt no p c e pe n i a pd o "p oei r n u t x e l a s m yu r q r e ei f n e ua t r O a d e oe r f c "e " ;t g pni i o c ne t i n ot n a a u a t t x cf o ah E R i fa y l mt aeq ud n s" t mio g sb e a oi ng t a p d d h" eb c nh n ita "y o yi t a i r s r e o m e r sd d n a e nt s r wh t o a mg te r al u e gt n v c w u o yt gt e u e " c a su af D i t _ a v t a r tn c y gnp mat nh fo e )p e t r , u s yi v l s v u uq t st
- i t _
a nd t a u s et n oai _ v i o e e l l a wge a e s e e nl t pc a o t ch s u t d i nh f r gv no ge ot g ed ge n l i nn a e n m e t aD d uro i gr pwd e s t a i i e u c o i oiCt t a s vgo r R s gi f r l ni ei r n l ed oMIn (P u a d c e u r nt eii oe r b mn l vt s t n ug eu o o a e Od i s Dsh - - EvMpOc m H - - - -
- O
e e r a g om Ca O t o r e D evo r " l u c i e l a e F aR nc w7 ge on it et t e
. 8. i un s P e. d qe t ne o s ve u ct g c
gein n s one e t v 's ]- 1 e t i ndt a o ag emt ue nar r m e g3. 7 i nci i t t i m t man ci l i l a pet t s .m g a e s F mlA y rf i rD t I s - e on a e u g n dt t e yev Mo C r c i i i g e t si l c a eb n ruba se g t e i t u n i l i ot q e Far u o n m i t phi Ro n s t pc s urn t r t c i 2 3 n r s e e nu' e d t n iF r i u c O I l i a n r i yt c a n F i g s i n e r 5 t a l u i 4 i t i a n i n t F ns e i as r Te - u r y i s u t l el i ga e r c i nF m l ut n S ie a v F e s ctn
,i s e _
n a v l ai o BE _ mt r ra _ oe _ Np O O
. t - .
o ) E I 3 2- 3 3 7 D E E E E l 5 2 e C ( 7 3 1 3 3 3 P d o q M F e r 6 E 5 E 4-E 4-E l e D l 2 1 5 6 p C 1 1 m q a e r 4-E 4-E 2-E 2-E S F 3 8 8 8 o E I 1 3 4 4 m o r e n F i L - s W t n t C o r l C u s t u o f o F f o L A k e c a s s o sR s oH O T R l B L LR T o
ll , 0 n l a n 8 y a n n o r s e o a a i v ig t f di n g i s m d r o pon "u ri t e )
"e nd o e m a on s r t u bu g
u t a e u a c e npe S et l a m c i h i r v a i e r r
- " c e d n d pt a t
a f o n g n )
, e r
o" , t n y i i o F o t p r yi r i D o c a u a ke t e i t o t a C i r a t a i c i l ii t l u ( t n v e f e r b n c yi h i n n C a ol c n g e" g o i t u ma c e u "m a r d nde s ess i we i e s e e q vrr be es n c d c l e ( 0 n e a n p fr D si e gu c d di t r o a t a e m a e gC t a r gc e l ud a m u rr s a f a o o r l a fo , "p mo a ys t s umt wng al u o v r r tud e e y i t e l l ah t a s oer f np o a e i l pid t i t r t e r b y o at r n e e s n ad e od e v nh e"o oc b ot o tor o ur evl aues e P i i t at n n t t r pi a a cl a e l ue i s r s r l i t l u fi a e e vb o o a n s f e ht ci r d ni c of e l a o- cb b e t u t u ot n l yf ht oc n g c e uiir r i t e at ci o d a i r l f n n e md t n t n d e eid oim l e nf l b n i po r o o of pai e k r sf o i r CCCd t k t e mtc a u o - - - mb a rh oe oh mir S oF e S S pt S Tp 0 l l
; ;lllI 1 O 9 l -
l e e m o W t s t c u o A f o e o N R s r o e P r t c m s l e g e A a c ) p s ns e h c a t i . a i t ch c s a p l a y e a o r t n h y n l p e s o e v T T t e a r opp r p a e s m y O e y t nf f ot h a r o
"t n
u B s u o e c r a y t a a c s d Hp a l p ool r ta ui i f I e s ( i c pl g n e e s e m e n pu a g r i g. c g s n e i el a r e r r c c p d r e n "g a d Tian n trd ue i r a n d d hc i m p p r A n r r mli u ri c t n ue ompa o a of o erf r f r mie r eS s t oCM e aHpU- - - P i D - - o
5 8 ) 9 l l 1 a O e h ht s a 0 1 e u o e r r T R W d r d n te a f n s A o e r n m r o a is
,a i
d r o e D M Vs i t n h u T H ( 5 H
, T A S E
i t c a sr P I mo S S T mt a g L S E [ /I I V T ac ri gd s O C C oIn r r o G- C P A t a U O B U S S S M i d I c n S A l a L R n A o G i t O S a G S O R / E i z n
/
I H P C a S S T I g r L S L V A A E O I T C E O Ga P S S I T I V I
/OP E
T B E T C A U S C U C A S NRO S L A S M A R G K N P R O C O G O S R L S P A H O T S G- A E - P I B S T I U S \ V S T I E T C C C A U S O - ILl!
e i v t p O i r I c s I e f
? r . o g P f t s
s s
. , n o
n s O en n o n o e n o i t i s i i c . a r e m l aa t l aa t , n it . a g t n p e l c c cc d a c e t o m u i r ii ni f h c ii nif i el l ri u e i f i i n n m. e . a a c e h c c qr v l l . ep e i ai p n x e E R ep e u u ar mIS TS TS RS QT o I I M e e c s c n a y y y y s e n a m r fl i i t fi oba i l t fi oba i l t fi i l t n e v r e f o ob s a oba i t ht sl sl sl r si l si c eO m r P e oa L va ov L a a L a si ov a si ov L ai r, a fl e ERn o f u , f f n f n f n f o O s oU y oU y oU y f na oUp e eet cc a i e s y ,R s nnt r e cn no cn noo c n n c n n e aa n s r nn e l ei o t p e m s u
'uqct ei ut o ei ut ei t ut e a
r e e me m t t ra P a a x e e r n u qc e r n u qc e r n u qc e r ui nm r o r r ii nnl a a pog r E M FF FF FF FFT E MMImr P o F e O t e c n a n t n e s l a n o n _ i a r m o e l i t o m e n o p n no a s m a r i t s ur o c r f r v nl t s il a m mi t g o t it p e e P L F u y S T r o C uc HA P r t s c a Fa o r l p p A I s l e AL S iY P R v e A fN _, y L . L 3 _ t a e.gc CO4 ! 3=Oo$ 2 h W O _ 4 1 , r
O f o 2 t n n 1 t e v i t o s - r d i d n t n e o p h a e o c s e ne m fS i r oI f o l al v a t u n L e n iar ev b s o ae ytioV ode aR R t
;t an om t l id d ar mio r at eS el mt sla&
T ng rI o e ol oo luL isTn t Cd Pi s iVa niE e I FR UrC A i n s l e a r C u d R d y e f o r o f d e t f to ceo r v e t n y ok r a i h Opr on o i o i t bl ee s s ot hy S inco e it Nth s s o nso i m N i d e b mta t al npted f in nt s oy c i i a a uh nt io n o um a e o wo eyt i f t o sir t s s ee s t luiqr e o a r o t v a gv t e o u rl pau r u ar Gc t t c oen n rl a a o adf iaor r a r l mce l ie p Ebi p nFcp e P v u Lv v o a px e F oe _ p _ s s A t n n o n e H e i t o ) c 0 m n n c i s d e n j _ g e i o A i u a gI t a v n O I . g s y r Da n r g i t m r p g S T m a e p n o o u c f o lg e C m u S ( r n H e y i s a )s P r o t lyk n e e f g n f nR e1 o o n i f n ie oi o w o i v1 nR f ( in3a r s n o it t c b r s f o os tr f o it cP la te o b o s o l S aC j-C Re r e l o o r s l l y yR i p e v e t l ng on V C d i Rli f a b a i l i b t l i eg mn t ir e e t iar r u r p e ci f n n ov a f o p u o t a L u e a i s s u ote o r at ot s r i q u p r s v pa p p s ea kc re br a p r e l orh l i m a e d a i a u nf a r a Lt A F p Uo Lf o i n v o I n i c h c o r ly e t s R a S m n1 oR 1 r I L e t r ot r o n o i i t cg e V c n t inpe an i i R a omv yt lerea H l b e a t iu n es nr e t mui s o pn ge it mp o l al s n h f r iav mifa nie oa p t r oo t r ki a n t s t os it c po em c a o n O U Pt e Aoc Lm
l a O d r u e c o i t et a e crrev e s i
, 3 1
rl s t po po r d t r ape c p o k o e n c c t or e a c i l a& r r. e t ne ec g luin iag R Sd vn a n Fi s Ci u nsir T Rg a r A I mtCeee n s e u ai e r lur n r u HFg iata r i d e c n f s t o d o r E eni mn* d e P e eo" g t a p* t ' a o u q ne" a r s
- N d e
a Mo* t t s I c e p s ) / f A de s e g n t O s e c u r i r r t en oo nn n u mii n e 0 O a i t n d e C e nC n nC eI g a a st r n e pe iu - m o O g . r c P r nS a f o ( E O r M m e P & a," ve i f w o eer rt i i
'e ce l
s v u e rd ec r e o N L
%si mfT t e e u pr e l e /
g y e v s n ef o l g a om r t no p n e s *
- t r
o n n o ol 9" i nl u f r i e l t a oid p iais L ev r ed et
*
- m n v it ce l
a lu op iap F a r r t a uga wbi a s s t i a rV a "
' teo
- a ,t
- uco i
i i t nd cy t i i c sd e pa v T I ladv e ail mba Olon i " er letr h bopo - c o w e g 3 r P re r t s v i in C
"g g -
np a r e or n o '" nu t i o r t af r o i t s odts n e _ i e e pk o c e u q
"t
'n d ce ed ioi r t d pm a " _
H fl o eo ri n f a od e E" a w nl e r oe citt ip i ut s k ut I* d s s l iau e ci at
- v " Opes qde u g O E '$
t F q La
( l l O e : r e
' 4 l
s e e a s ei f sd ht c a r a e e a c g e s c r yn yn va yn t ot e e t t u eifei o st hl a e e b f t a a a s nt g )y i u t fa w s sl wge s o e r g u gol i i l r i cd i t e e i ndb s na a et o t s c e i l i s t t r nai l d y i l n ba ev e r e c )y vaa ol t a e a l i t b ei t s r m i a c at t l t yo c a r ve ff o i t
;e fl i l i l aj b ob ais n eb s o a pc l
a t va a pa u at ubiat i t or e n u o y o c d oc( r pe ecb t h p i gt a ef t s ai mta c c s m; i ya p nnt l h t s n it s e e a aa sylye is"w t ei a fl ymms a u et vu s et o O c ad sl r orooh r ai n ss ib t d s e l o e ol ht h utaff ue r r e b i l s v el e jn a t c "t e bede n e l gpptai e s t a r b a ot r o n m )s mof e v e t s A o "t ntet fr n e r sh a e y gluo t oet rh s e enr e c po nm ai oeit ni el i s wsgr s i f e c wms s ab c a wr e n ve t opobi t s gs unt n f r 't o a t yi l a l h a o t es i e rh fe v f mits ry rysat e b uv ew p a s n a nI o s gnt us o men ese ae "ts?" t e s u i t ' n oa vh ui t a n o at a s emluI i t g t et cjmB ot l uh i
- l l oor -
g Dd e ANf eW-( r R (" - O
l 1 O 5 1 e c n a ~ e s o t r a
~
s t o et s p h m Q y c n n i r
" h T 7 Cn ,a e Q o o
i
- " I m I t c
s f
- l e r Cr s
me I r e nS l i r d oi e s y P e ~
' M o @ S fr uq t e e N ht s
e l a g ' P w cPR e P t s n
' l
"
- t e a A n s S e
" i
*
- l i
t a " h T Q
- t a
s n c '" l e I ( - Q s u Oito o d q w s c
" o e
- i h I
( i s n l l e A
'" M F
r v*
~
T l e 5e q d l a e v r ~ s i ' d o F r d t e M r o P F ' i h s b s i l b o o g l o o T i t c ) C
=
b Y n u o s d m T i s y P e r ( q s F o r
)
=
l a y A l a o e r i s y s c C t e k i i n T F hl dr A d e l a n ToP r (q h a k e . A fe e r c n s N n a i R n I F l B o y c i t i r n e b s e v r n e e e p r a O G P P
O e 6 1
- c
) .
nhd at e mir r wiud F fot n eD r n qe a e c et e r d nCe a( c y ps i si e t v h gs c n mse r n a c r a n o nio n a opr s p foivi n r i r d e nc u gtoe v s r c o e ef t r i h gs s s n e e e pjr a i s s t na i r ni c s
,b e oP e ei n e
a ef
,moima oih t b c o
r s e e e n . i y e e r t fE s Td o ri r e oog mc i l iTd t u gr n P n ne i e qe i _ b taA e f t e a ab o cn ht O p sC rd oh - t r a ct oO ma a mgvn h ed t t e s i s . o p t neL dL i Do uiat f r ope s t a n ei n o l adAd pe e r e ht ri r c s mta n s e c e e ,n l i ut n p s e c a s e r o R C s a s et gn s B a
)
yd l p dr r mciul a pr o o r qa pet l y e l e wor rf f r a ne6l t l m ai r n g p oeh e fe a l a hi e r t u uq p e wn oidr tntievt p c S c u e e r d ot e i oinr a . qs c a pc r - De r t d u ki t l i I nP o r n c o ol od n e s u = u C s B - ( Wml a E pa r s O
O g
)
e i n de cn r s a o om pr i t mfo I r n t s Pe n o - _;
+
4 e ro mi t :::< M _- e 4 e r n
- +
_- n f. r 4 io u
- +
- +
f .- ae 4 qM
.- SP 4 eo :::+
O s$ p llm$ + R T :::+
- +
y e ic el va :::+ t n f. t ae e MsP i i r pt m a l
@$s' M i r r c
sP ac e tc e I r t i C " J P o h e N c c s f. s r ee n f. ae r ( S a r LP sP - g P kt ) e c n e n a r d i s f. s r n f. n ee aer m o LP
@sP fo r
i t A J g$3l
,/
r e n e derP oo c e r f. r ot o E or M an e MP t i n on i k
/ \ MoMo A m t g or n f.r oT -
MPe Nl a n fr % i t c . i l [ c v e I ! a r - r P ' P i
" f.er t e d MMPM N o
D te !
,i
\
( a c l o d r o e - l - [ t A i n o n M e O t s I c n a d n m e e s r n n fo s s s o o r a n o m p i t c P e B i t c t e s m o A n y C n a u S
/
s n o F m i a u T r H
- 4 . P :;i , :
7 e1 v e i t p i n _ v O i r c i vr o e t s s e u f s id t r c on r S s o s a f a n P s a n s el e sm e f O t n e e r s n o s n o e s n o i t o a c o o wefe n r t o m A c s r t - m e e d l aa i t l aac i t d a n i . t a t n Pd eie pmr t e e r l p i r u tc e ii c c nif ii c el rl i cg ni e
- m. t r r p u e i
ni f m c sciu e es i f . h c i n e : i a a e c ep e h c c e qr v l ai l p eepe u q x ERS e le TS TS e p RS e u QT u a r mS I I I
, t o x rqmd NePr e os o n e
R e e c n y s s i v a m fl i y t i l t i l y t i l t y e n e r fi fi fi e ? t r v i o ob o ba oba oba ht e p f r s si l a sl si a sl si sl si i t c e eO d u i r P e oa L va ov L a L a o va Lo ai a v r , fl f u n , ;i st t c f f n f n f n f na . ERo r t oA s o oU oU oU oUp eet c ca i t c g e s e s y y c
, y , y e s nnt an nn
,R cn n cn noo e c n e aa n s Fi r
i n r u no ei ei o ei o ei t ar t nn ee me m e a l ao n P m s a ut qc e ut qc ut qcn ut q cn 'me ror t nnl g r t ri ut s r n e r n a e e e ii aa p o t e x u u r u l uu d E M FF FF FF r ui FFT E r MMiP mr CQ n - O A e c s n a t n l a l n e s n e m r o m n o n an s m i o s v f o l e i t c e n p o a r t t ur o e P r e v e n u t s y i ra m o m i uc t lg o r i t t s c n Fa L L F S T C HA P I l l w' s' w ' A Q Q N Q f O b*$h.Q y5eM % $t.g$ ,E$g ,gg3.M g i n - r o t _ i n r o sl s o t r e ao M l uih@e t nr C gol e .t RMA l O ^ 3EOC '
!, gg
3' s ehe v t i t p n n i r i v o e e O i c s vr m st r c m e u f id on e r f r P Ss s O n t a e s n s n
- m. .
s o n o s a s el e c o ve n f a t s mr rtoief i e r n u o i o o e o t e q s e m A i i c n i t l p e r de l a a cc t l aa t mdel, a t a . n e Pd eie pmr t e e c c c g
- m. r r p l
ii ii i ri i ni t R t m n c ni f ni f u f e . s a a le h c c h c qr v mieue in l . : e pe c esci u e e ep e l c e ai p e ERS x e e e p u ar mS I
, t o qmd i
v TS TS RS QT I I Nex Pr r eso no t p m i r c s e c m s e n y y y y e s a r mfi t i t i i t i t ne r m l l l P r fl ob i fi oba fi oba oba v e ht
?
e fo i s a sl sl sl t c d d r e eO u l si si si si n P e oa Lv a o va L a o va mo L a Lair, a v fl f u n , ; i st t a f o f n f n oU f n f na E R.i o r t oA s s s oU y y , oU y , my oUp e s e"t cc a n"t t c g an c n n n e e ,R cn noo s e Fi e l p u r o c n n ei o c n n o t a"e s n c m a s ei ut qc ut qc ei ut qc mqcei ut t e a r r t n"m e e t m e ra l ao ri ut s o r a e x e r n u e r n u e r u n e r ui nm o r r i n"l a ip ^ o g l t uue E M FF FF FF E MMImr CQ P mFFT P e e s e _ c mtn O n e c n a m n o e n s l a n o mp i f r o e l i t m e n o an nes m a r i t u s ro L r v cl n t s i m mi g t i t e e u y a o uc t o t s c In Fa r r s
. P L F S T mC HA P e
d N o + + + my + + Y l e O o E $* $ >
- A = :: < cgob ,Eu2
- . y* ) '
v e
= " - "
= " - "
L m= " - h a " l l( 4 . " l: % g m " i H " f m o r g r n o y m o s r e i r t l a st l t a o es ss l t o ug h gvr e ec l m uine neoc t i g goc n o Rgiei e HLD m e RMir LP M i O a E*m*COehggg*oE
n _ eI _ v e p n i e t i O t s i r c s i v v r e m m u s st o r c on e - n e r u S s f o i s a d f a _ e P f s a s m u u
, n o
s el e s mr en o t s u O ne e s t n n m , n c _ m s m A r i o o m e c o . i t a o o v rt it ef - e i e e l aac t l aa t u , n it . a g. t n Pd eipmr l r de c mdri ela c e e r o i n i m c t ii ni f ii c c ni f l i f ni m t e r r p s i m u e i e u n : e ec sciu e e i a e h c hc ce a uqr v u l l . c e ai p q x ERS e le TS e p ep m eu u ar mS I
, t opeqmd TS RS QT I I Nxe r es on o pr e e m
R m u s e e e m u a , i v c n a y y y y s s e - t t i t i t m i t n r m i l l l e p r o fl ob i fi oba fi oba ufi oba i v ht e ? e i f s a sl sl t c d e eO sl r r si l si a si a msi a u Lo av Lo av e oa mov Tl t c P L v fo na a fLnaair, ERn o f u , ; i st r t s f o u f n ou f n ou i oA eyc n,Re t oUp eet e r s e e s y cn n y , y , s
, cc nnt n s a cg a n cn no c n e a a e Fi n
ei o no i n u r t nn P ei ei n oo ar u ei t eemm l a o _ er a m s ut qc n ut ut m ut e r o t t ri a a qc qc aqcnmr m ut s _ nnl d x e mu e nu r re nu e ui r r ii a a po g l t uu e _ n E M FF FF FF e FFT E m m MMImr P CQ _ A e c n u m - O t l s a mm n a n m n u m e n s o l o o s m e r o e l i t m n p n a n a i t ur o s v e f r e e P L v c n u t e s y i a r m m u mo mi uc t o I r g o r t it t s c n Fa F S T C HA P I L y y y y m m _ uy u 4 m _ Y h _ g _ i cu# j. (ga q (o goN g8o
- g62.d _
H u u m f O - - m - n e e A _ g . i n y m u m s r e r e r l n s d wo h g o e l u i r ysOl oLfH t ve i I n ol r e y e WtAlyA t o t l aa oLl c t r e n s o nom M l u m.t i nh e a m u t l a e vl s lel s e o ogr gr ge eb ur vove i .i RNMHD m et n er m u m RI LPh O ^ 3g2.LOm*m3 g 0,g i
O i s g e c 1 2 s o n n n e i n e r g 'h o s f e i a t a n
? d ht e i n l i Rf o o i a
c w? ns v m e ) s i d o ui r n . t e o t c o e s e a m gi n e dr u f a i v o
? : i g d m
r c e t a a o s b gwAi s s n H n _ f o n d e l l yof a H e oi t o i t o
)
,sa _
c aDl n e l n e? r s c a n v e _ i r a s n a r nws _ e fe s e a ei _ m r i n t l p s a _ t s blopa _ fo ht In al l n _ . hi x x _ t c s,eD n E i _ r e e p wta a t t ,s _ r e e : h s c e n _ r o _ O t a i v t u o dWb t e e o B e fa ui ( t l af p r a c y r b n? e _ l g tc a eh s t el d t ht i s
- t i dd a ,e x _ b
_ a t e u k n r a peh i a s ot h f l uis s _ b _ o n d i ht s r s t ek e o n oe _ r p i b o mo i s ci nl o (w ho t _ s n o A _ i to s i s o i t e rn t cdp y s y ps e o r e e e sh _ H h t a p mn nf g t l l n e e a Ina w yk t oira e _ r s w s c p c fo o l _ fu o i i r l h J e eS d ht ? l o as a pt s xo e r n H s u nn e e t i a c nwie o o DHN o _ _ h _ T e h o l y y vi t d e i t l cl J b gi a ioGpMuf o _ - sl d t s a b a P ona - - Ah t O
j1 l o e y l 2 2 l hl t t e s n w e drie c g y n n ai fc a l l i wf a m r s ous c r o e t i p o i t r y fr n u s gh nt t e o a ii vw t o f p e c me r c e i r t n o n e n a gM d yt n e s a e n o m r ht a r i e su n e b si r a i t fo p nc s s a gs a i t n e c r n e l o a n oc e n c l o e p i nc y e v n i m, a l a e r e gn r e e m g u e s i a r a e fo rl b r n s tr s m a r fo i m D re ui s qs r i d u s ad n ef,o o m u r o P F f e r o g n e p f e p o l i c n d t o o e mt p e e a d f r r S s O de d n o o i s i t s e m n e e ts i o s a e e mn s c e xh e c i s c nh t a ae o S s s s y s f i r ed mu e o de m P t e me e o sl i c s t fe a r u ph r s o a s babo d t s a t t he d e e- pr r f e e wo nt o e e i d c a cf oA p n r f mht or n l pa no pR oT o ta d os a r mg e uP s ti n tr kt C o on toi ri n s n p a e d n o r n l a oi o S Wp t u~o i mmc t t f r t d r a d ed Ss e de pwde e ss s r a oi l e - - nt y e sl n er e F c DS _ t or ted o I nme a nd I a P r - - - o - - - _ r
O y f 3 2 n a o - a s c s w t c e si l e m x xfu p i o a a e s _ s a c yr e u t t u _ i l t t u a s s o o s i a s e e _ b r m r n a e p i
). mg o i
o b omlah r e e i n m s e t c d g a pt gg t e s u ns e hc - v r nn t nr yl c i t ch c o a t s oo n e a p r n e uoit l e s l l a a Ha l l y s o e r p v e b n n (y ul a op a O O i i ggh gi t p y "tn c e a r t r r n y a a r e r h c r y o a r a t mmr rf a o e t o p f o e a ool r ta t a ui i f c m ht a min o h o p l p pl g e n g r e s i a i pu r sf e s c a g i s m g ts t r u m nd e u e nTc l e i e r n "g gr e di s a c rp rd i r S ge c n ue a n f nd i h l a n t t c n p i p uf si oe z c r rul i ue mpa l ed p yi mia a n rf of ut S r ri oCM o r a e r e me Dgir e s - - - . a v e o HpU x e h - - Eb Tto O
;' : l j
O yn go a e 4 2 oi s e t o e o a l c o ht n a nmh _ c hc n h r i n c fo o s a f r e n oof t i e ot r n l e e r "g ec h pe c pv s vp e i n n o a a e n l n o f na rd t a o ompeh _ el s r ps t _ i e e a o a o g pm t _ a v ef v twe e l r r e e n oo r r r ) bd vpi r pf o e s e s d e h n i t h t e u s a ct ih u ui b n iAu d woys nt s O O i t gR b y mlai n nP al e pe c y oc i " t d s s n t r ( t ri a ei n a i i d d or pp sd o a pd m m t oedl e e mro onr a,foe p r _ m u t ndd o e i op p r r e t a r mnl e t a o t r n et e pd - S u o e e f o t et nf n pc c e s s vh o mnn i i s a tet a a uer e a r a nwm _ e mme b r r b p b fi hed l c i yffoo l ye i si net e ar e e r ah p ~p M t o oggm s oa M P i s t s O}}