ML20148T309
| ML20148T309 | |
| Person / Time | |
|---|---|
| Issue date: | 01/29/1988 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | |
| References | |
| ACRS-T-1639, NUDOCS 8802030275 | |
| Download: ML20148T309 (188) | |
Text
{{#Wiki_filter:1 Acesr- %.59 R,a N A _ O LNITED STATES NUCLEAR REGULATORY COMMISSION IN THE MATTER OF DOCKET NO: SAFETY RESEARCH PROGRAM MEETING l l O LCCATION: WASHINGTON DC PAGES: 1 - 135 DATE: JANUARY 29, 1988 ..................r.. ;..r........................................................, ,q nf ..,4.. h l 1 ^ \\.,,L, s a f .,'JT .t h'. ui d' ' T' e ll ~U< i} lA n o n jq flge. l o\\ ^ u,iid f l l Heritage Reporting Corporation OfncM Mmertm 1120 L Stroot. N.W. W uhingen. D.C. 2C00$ (202! 628 4444 0802030275 000129 PDR ACRS T-1639 PDH l I
- - ~. i' ) 's 1 PUBLIC NOTICE BY THE -1 2 UNITED STATES NUCLEAR REGULATORY COMMISSION'S ) i 3 ADVISent COMMITTEE ON REACTOR SAFEGUARDS j i i 4 s 5 \\ 6 i I 7 The contents of this stenographic transcript of the 8 proceedings of the United States Nuclear-Regulatory 9 Commission's Advisory Committrse on Reactor Safeguards (ALRS), a 10 as reported herein, is an uncorrected record of the discussions i ^ l 11 recorded at the meeting held on the above date. 12 No member of the ACRS Staff and no participant at i 13 this meeting accepts any responsibility for errors or 1 j 14 inaccuracies of statement or data-conthined in this transcript. 15 l 16 s I i 18 l 4 19 20 2 21 ) } 22 j 23 j l 24 l 1 25 W () Heritage Raporting Corporation (202) 628-4888 l, l iN,, i i k + -.-,llL,3,z A
1 UNITED STATES NUCLEAR REGULATORY COMMISSION 2 ACRS SUBCOMMITTEE MEETING ON THE SAFETY RESEARCH PROGRAM 3 4 t
- Friday, 5
January 29, 1988 6 U. S. Nuclear Regulatory Commission 7 1717 H Street, N.W., Room 1046 Washington, D. C. 8 9 The Subcommittee mee, pursuant to notice, at 8:30 10
- a. m.
il BEFORE: CHESTER P. SIESS 1 Subcommittee Chairman 1 12 SUBCOMMITTEE MEMBERS IN ATTENDANCE: J. C. MARK ( + 14 CARLYLE MICHELSON D4DE W. MOELLER 15 PAUL G. SHEWMON I DAVID A. WARD l 16 FORREST J. REMICK CHARLES J. WYLIE i 17 JESSE C. EBERSOLE 18 PCRS STAFF MEMBER PRESENT: f l i 19 SAMUEL DURAIGWAMY 1 } 20 ALSO PRESENT: 21 ZOLTAN ROSZTOCZY (RES) { LARRY YBARRONDO (Scientech) J 22 ROGER MATTSON (Scikotech) i ROGER BLOND (CAIC) 23 JIM PITTMAN (RES) 24 l I l ) (]} 5 i i Heritage Reporting Corporation j (202) 628-4888 l d
i
- O a
1 1 PROCEED I NGS 2 MR. SIESS: This is a meeting of the ACRS 1 i 3 Subcommittee on the Safety Research proDram. I am Chester 4 Seiss, Subcommittee Chairman. ) i i j 5 The other members of the ACRS Subcommittee that are 1 6 in attendance today, starting on my left are Mr. Mark, Mr. j t 7 Michelson, Mr. Moeller, Mr. Shewmon is out for the moment, Mr. ] 8 Ward, Mr. Wylie, Mr. Remick, Mr. Ebersole. a l 9 The purpose of this meeting is to hear from the j 10 research staf f on their proposed methodology to be used in l 11 prioritizinn the NRC research activities, and for them to t 12 present the results of a trial application of the methodology. O 13 The cognizant ACRS staff member for the meeting is 1 6 14 Sam Duraiswamy, seated on my right. 15 The rules for participation in the meeting have been i 16 announced as far as the Notice in the Federal Register on 17 January 14. A transcript of the meeting is being kept and will j 18 be made available as stated in the Federal Register Notice and { l'9 as usual, I ask each speaker to identify himself or herself and f 20 to use the microphone so that the reporter can get it on the I 21 record and so that I can hear. j 22 We have received no comments from members of the i 4 23 public in writ ing or any requests to make oral statements. f l 24 We have an agenda that Sarn prepared. We will stop at l 25 Noon today. There will be one hour for lunch and there is c:) i J Heritage Reporting Corporation (202) 628-4888 r t
i CE) 3 1 another subcommittee meeting beginning at 1 00 O' clock to which 2 you are all invited if you are not already planning to attend. 3 The representative from the research staff this 4 morning will be Zoltan Rosztoczy. I never can pronounce that. 5 He will represent the staff. And are there any l 6 questions about the agenda from any members of the Subcommittee J 7 or any comments anybody would like to make before we get j i 8 started? I see nobody volunteering, so Zoltan, you can have j 9 the floor. f l 10 Ch. Sara has got something. This is for the 11 subcommittee. At the last full committee meeting we were 12 reminded that we owed Congress some comment on the research 13 program under the law. We have not been relieved of that i l 14 requirement completely, and Sam, I guess with some help from I i 15 others, has drafted a letter along the lines we talked about ] 16 last month. He is going to pass that out. We' ve already got a 17 it. And it will be discussed week after next at the February t j 18 meeting. But he will give you an advance copy. And if you i t 19 have any comments get them to either me or to 3am or both. [ I 20 Okay, Zoltan. j 21 MR. ROSZTOCZY: Good morning, Mr. Chairman, members l t t j 22 of the Committee. I am Zoltan Rosztoczy, the Deputy Director i 23 of the Regulatory Office. The subject for today is research t i 1 j 24 prioritization. This is not a new subject to you. A number of I i 25 times the Office of Research has presented to you our thoughts i ] 1 Heritage Reporting Corporation (202) 628-4888 I L t [
O 1 in this direction. 2 We are off to a new start this time and we are 3 planning to describe to you what is our new start, how are we 4 planning to do the prioritization this time. 5 We are not starting from scratch. We are utilizinD 6 all the information that has been learned from the previous 7 starts. 8 We would appreciate hearing your comments today. 9 Certainly we would like to use those comments for our final 10 process we are going to use for prioritizing our program. 11 The prioritization effort is a Joint effort. The NRC 12 staff is working on it jointly with a contractor. The main 13 contractor is Scientech. 14 As you can see from this slide, there are a number of 15 people involved from Scientech. In addition to that, we also 16 have the support and the help of Roger Blond from SAIC and Mr. 17 Humphries from Southwest Research Institute. 18 My Job this morning is to provide an introduction to 19 you, and this presentation will be followed by Mr. Ybarrondo, 20 Mr. Blond and Mr. Mattson and Jim Pittman. Jira pittman is here i 21 on behalf of the staff. He is the project engineer for this r 22 program. Division Director Bill Morris probably will be l 23 Joining us for part of the day. He is right now tied up 24 somewhere. 25 The first question that we have to face is why do we Heritage Reporting Corporation (202) 628-4888
i T O 5 1 need prioritization? The purpose of prioritization is to 2 provide guidance for planning and for resource allocation of 3 the research activities and to facilitate communication within O ( 4 the agency, meaning with other offices like NMSS, with j 5 management, the Commission, and you gentlemen, the ACRS, and i 6 also to facilitate communications with other Government f 3 l 4 1 7 agencies like DOE, OMB, Congress, and the various state 8 governments. 9 This also helps us in our relationship and j i 10 communications with the industry, including EpRI, and so on. l 11 We have also been urged by other entities to provide l 12 a system for the prioritization of programs. ACRS has been one l l 13 of these. OMB has urged us a number of times and also i 14 Congress. b \\ l l 15 A recent viewpoint by the National Research Council ) i j 16 also touched on the importance of prioritization. Let me i l 17 quote from the report. i ) 18 "Some research programn must be based on a safety-19 research philosophy, a long-range strategy, and a set of near l l 20 term priorities, with the priorities linked to the philosophy ( 21 and strategy in some transparent and meaningful way." l 22 It is our goal to accomplish exactly that. [ 23 MR. SIESS: Let me interrupt you at his point. i 24 Because I have a slight problem with the objective of 1 I 25 understanding is. From what I have read, it looks like the i O I Heritage Reporting Corpoiation (202) 628-4888
O 6 1 effort has been to prioritize the existing research projects. 2 Almost what the strategic plan says. 3 Rather than prioritizing the questions of the 4 research r.eeds, that i s, the interviews were with th9 research 5 people, not with the potential users. And then the projects 6 that are ongoing were evaluated on the basis of their 7 responsiveness, or somebody's ideas of their responsiveness, 8 etc., etc. 9 Was that the limitation that was placed on this? You 10 know, if I am running a program and I see my budget cut by 25 11 percent, I need to prioritize what I am doing so I know what to 12 cut out and what to delay, et cetera. 13 But if I am starting back a few steps, I need 14 somebody to assign priorities to the questions before I start 15 the research. Do you understand what I'm saying? 16 MR. ROSZTOCZY: Yes. The purpose is to draw them. 17 This prioritization is not solely for the purpose of budget 18 cutting. That we will use this, but it is expected that there 19 will be other uses, too. The goal is to prioritize all 20 research activities in order to accomplish the agency's goals. 21 We start with the agency's goals, we ask the question what 22 research is needed to support this agency's goals. 23 MR. SIESS: Who asked that question? 24 MR. ROSZTOCZY: I described to you in some detail -- 25 MR. SIESS: That ouestion was askad somewhere? Heritage Reporting Corporation (202) 628-4888
c 7 1 MR. ROSZTOCZY: We started out with a st ategy plan 2 which represents the agency's goals ard we generated a set of 3 questions. You will see those questions. They will come up in 4 the presentations. And then in terms of how the trial f 5 prioritization was conducted, that is only part of the overall l 6 effort. In addition to discussing this with the part of the 7 organization that performs the research, it will be also i 8 reviewed and discussed by the users and then it will be 9 factored into the information before the actual prioritization 10 will take place. I i 11 MR. SIESS: I' ve read the report and I guess it is ] 12 not clear to me where in the report there were questions asked 13 about the agency's goals, where those questions would develop. j 14 Who would do that, the parel? J j 15 MR. ROSZTOCZY: That was developed by the contractor l 16 and us in a start-up meeting, in a two-day meeting on that to l 1 17 try to pull these up, and they finished it up and they will I I j 18 show you in some detail a little bit later exactly how they I 19 pulled them out and what was the process. f I 20 MR. SIESS: Okay. But it's not in this report? { a r 21 MR. ROSZTOCZY That's right. That's right. I think l 22 it will be in the handout that you will receive today. Keep in [ i 23 mind that the strategy plan I believe has not been published 24 yet so it is a pre-decisional document and you probably will i 25 get a copy of it with a pre-decisional stamp on it. I i I L I I l Heritage Reporting Corporation t (202) 628-4888 ) I 1
(:) 1 MR. MOELLER: I think another aspect of the overall 2 procedure which would be helpful, and I gather it will be 3 covered, was whether you were seeking problems of a technical 4 nature, you know, deficiencies of a technical nature, within 5 the research program, or problems of an administrative nature. 6 I notice, for example, that when you read the ovarall 7 summary at the end, many of the problems cited are policy and a administration, or so it seems ;o me, versus the technical 9 nature of the research. 10 The main effort, the current main effort, is going in 11 a technical direction to prioritize those programs which are 12 providing technical information for the agency to accomplish ) 13 its goals. The research office is also involved in number of 14 other activities which are perhaps technical but in a different 15 sense. 16 For example, rulemaking activities. And those will 17 be eventually included, but we are not including them in the 18 first round. 19 MR. SHEWMON: As I understood Dr. Moeller's comment, 20 it was that you had a procedure hure for evaluating technical 21 procedures or technical information which dealt primarily with 22 is it an important piece of information, what will the schedule 23 be, is it relevant, but if somebody fed you garbage you 24 wouldn' t know it. There is no way of evaluating the technical 25 content, only the merits of getting information for the Heritage Reporting Corporation (202) 628-4888
9 1 1 questions given. And your response did not speak to that, os I understood it. 3 MR. ROSZTOCZY: As you will see in the presentation, j 4 one of the evaluation criteria we are evaluating on is the I t 5 usefulness of a given research program to a resolution of the l 6 problem that it is supposed to address. So specifically that 7 evaluation, that part of the evaluation specifically addresses ) ) 8 the question that the research program, as it is planned, the a j 9 data that is on the books data, whether that research program ] 10 is going to be useful to accomplish the goal that we are trying j 11 to accomplish. That is the area that we will try to factor in 6 l 12 the rather important matter of how good the present research i 13 program is. I i 4 14 M R. SHEWMON: But to decide in advance for $3 I f 15 million, somebody gives you new Reg. 1150, and then where are a ] 16 you? 17 MR. ROSZTOCZY: The purpose of the prioritization is 18 more to look into the future, today, so that we can accomplish i i i 19 our goals this year and we will be able to accomplish our goals i 20 next year and the year after. So the one that is already done, i i j 21 that is the last significance to us, but nevertheless, we do l 22 factor that into our decisions. 23 MR. SIESS: You missed the point. You cannot ignore l 24 the past when you are judging the future. You are saying that 25 if this research is done, and if it is done well, the results (1) i Heritage Reporting Corporation l 1 (202) 628-4888 4 i
h 1 will be useful. But if the research is done and it is not done 4 2 well, the results will not be useful. So you don' t really know 3 whether the results will be useful until you have them. 4 Somewhere in the process you have to make a judgment as to what 5 the probability is that you will get good results out of that 6 research for that contractor. 7 MR. ROSZTOCZY: In a conse, that judgment is made in 8 this process, and is supposed to be a living document. The 9 present judgment is that the given reseatch as it is today 10 planned is goinD to provide good, meaningful, useful results, J 11 and it could get a relatively high rating in that category. If 12 next year, after seeing some of the progress from that research ) 13 program, it becomes obvious, or becomes questionable that it is l 14 going to provide useful results, then at that time, that will j 15 be factored i n, too, so that how the process is set up, it kind 1 l 16 of systematically forces the evaluator to address that j 17 question, and they address it at that time and periodically i 18 when the priorities are reviewed. But yes, it is a judgment. 19 MR. EBERSOLE: Mr. Chairman? l 20 MR. SIESS: Yes. q 21 MR. EBERSOLE: Zoltan, with the perspective of the 22 last 20 years, I would like to have you make an observation 23 what you think about the intense focus, certainly prior to 24 TMI-2, on the hypothetical large LOCA and the extremely i i I 25 expensive complex therrachydraulic ef fort that was made to try i i Heritage Reporting Corporation l i (202) 628-4888 i
() 11 1 to solve that, in the context of looking at the original event 2 as contrasted to events that would sneak up more quietly and 3 wubtly on you. For instance, the loss of force convection. Wo 4 would not have a roar in the plant due to a large LOCA but 5 really a great silence. And you lost supplies and lost force 6 convection, and the plant would make its own LOCA due to the l l 7 absence of heat removal. I have always looked on the enormous j 8 effort put into the large LOCA program as being 9 disproportionate to the significance of it from the very 10 beginning. l 11 MR. SIESS: Did you do any retrospective 12 prioritization of judging 13 MR. EBERSOLE: Not yet. You could do the same sort 14 of thing in looking at the progression of core melt as it 15 hypothetically goes down and takes the bottom of the 16 containment out, et cetera. 17 MR. ROSZTOCZY It is an important part of our 18 overall effort to periodically review and ask the question, 19 where are we going. There is a tendency in an organization 20 when a relatively important, relatively large program is 21 attempted, there is a tendency to let that program go its own 22 way and maybe dictate its own costs. We cannot afford that. 23 We have to be careful in that area. And this prioritization is 24 part of a periodic review to take an account of what we are 25 doing and why are we doing it and make adjustment as it is O Heritage Reporting Corporation (202) 628-4888
( 12 1 needed. 2 If we do a good job on this, then I think the 3 likelihood that we will be spending a large amount of money in 4 one area without a proportionate return would be less than it 5 has in the past. An ongoli-) change in the agency is a larger 6 emphasis on planning, strategic planning into the future, and i 7 to periodically review these plans and see if they need to be f L 8 adjusted. 9 MR. PITTMAN: Zoltan? l 10 MR. ROSZTOCZY: Yes. r i 11 MR. MATTSON: Just listening to the questions from l 12 over here, most of the questions that have been asked, you are f 13 going to get a lot more information on in the course of the 14 briefing. The report is not as complete in describing what has i 15 occurred as tha briefing is. There is lots of opportunity for 16 questions later and they will go better if ye.u get more j 17 information in front of you. 18 MR. SIESS: Okay. Let's try it. But let's keep in 1 19 mind that you are going to get questions. Although we are 20 working in a methodology, we are constantly going to be 21 examining the methodology in terms of what it says about 1 I i 22 projects we know about. l 23 MR. ROSZTOCZY: We appreciate that. That is a good 24 check on the methodology. f 25 MR. MATTSON: You won' t see that until almost the Heritage Reporting Corporation (202) 628-4888
C:) 23 1 last briefing. 2 MR. SIESS: That is the way it usually is, Roger. Go 3 ahead. i 4 MR. ROSZTOCZY: In terms of the background, I 5 indicated that this is not the first time that we are faced i 6 with prioritization. We had some previous attempts. One of 7 the attempts was comparative prioritization, that individual r 8 research areas and individual programs were compared against 9 others on a one to one basis and then some difference between 10 them in terms of importance was reDistered in terms of numbers, L 11 whether one is more important than another or less important. 12 There was a set of people with reports like this and then those ) 13 were sumreed up by a computer program and some kind of an 14 average rating came out frorn this. i I 15 Later prioritization, one of the problems of course 3 l 16 with this approach was that it didn' t leave documentation i 17 behind of why those rates were given. It left the rates behind 18 but it did not get the documentation of why those rates were 19 given. I 20 Later effort tried to correct that and provided 1 21 somewhat more documentation. In this case the documentation 22 was prepared by the proponents of the program and there was not 23 sufficient time and sufficient discussion to arrive at that i 1 1 24 consensus on the reasons why certain program were important. t l l 1 25 And based on that then this was done also on an individual Heritage Reporting Corporation (202) 628-4888 l l
i 14 1 contract level and those were summed up then in order to arrive 2 at the priority for the larger levels. It turned out that 3 quite often within a given area we had a number of programs and 4 4 some of those are high and low priorities. And we tried to 5 average those out. It wasn' t very useful. 6 As I indicated, we have noticed certain difficulties j 7 with this previous prioritization. Our goal now is to try to 8 address those and develop the new ones in such a manner that we 9 don' t have these shortcomings. 10 Some of the dif ficult ies were that the information t 11 needed for the prioritization was not documented in a 12 sufficient manner. Some of the documentation which was O 13 available was not factual or did not represent a consensus. I 14 Criteria that was used in the rough prioritization was almost l 15 entirely risk oriented, d idn' t take into account some of the 16 other agencies' risk reduction. Quantification in terms of l l 17 risk reduction was difficult. Individual programs came up with i 18 estimates that they will contribute to the risk reduction so I 19 much but there was really no real way to judge it, whether t 20 those evaluations or those estimates are correct or they might 21 be overstating or understating the actual case. t 22 The part ici pant s, when these individual forms were 23 filled out, the participants had detailed knowledge of the l 1 24 programs that they were involved in but frequently they did not 25 have anywhere close to the same knowledge of some of the other (~) i Heritage Reporting Corporation (202) 628-4888 i h
10 1 programs that they were comparing against. 2 As a result, some of the assessment ended up somewhat 3 partial. 4 In the last prioritization, the panel participated 5 and therefore in the prioritiration from both RES and NRR 6 simply didn' t have the time alloted to them to study or discuss 7 the program in the detail that one would need to arrive at 8 useful prioritization. l 9 The time period that was set up was sometimes 10 interpreted differently by the different participants. And l 11 finally, prioritization of research areas only we found is not 12 sufficient because some of the sub-areas, within a Qiven 13 research area, some of them could be high and others may be low ( 14 priority. 15 Based on all this experience, then we set out the l 16 ground rules for the new effort and then we -- l \\ 17 MR. SIESS: Excuse me, Zoltan. That last statement 1 18 am still trying to understand. prioritization of research 15 areas only. What do you mean by research area? 20 The prioritization back in 1983 which was done by the 21 research division directors compared only the research areas, 22 which were the main entries in the long range plan, the long 23 range research plan. Approximately there were Just about eight 24 entries and those eight were compared against one another. For 25 example, thermohydraulic research was one. Seismic research O Heritage Reporting Corporation (202) 628-4888
[ L ( 16 l i was another one. 3 2 MR. SIESS: And what you have done now is different i 3 from that? 4 MR. ROSZTOCZY: i 5 MR. EBERSOLE: What we are planning to do now, we are 3 6 going further, and I will show you a slide which will give you 7 an indication of how far we are goinD. 8 In the terms of the ground rules, what we set forth t 9 for ourselves in Number 1, the prioritization process should be 1 10 simple. We recognize that to do it the first time and to have u 11 the proper documentation, will be a sizable effort. We have to j 12 do this to start out with. But then the process should be a I { I 13 relatively simple one, so it would be easy to update and we j i i f i 14 could continue to keep it in an updated form. 15 We also recognize the need that prioritization must 16 be baaed on knowledge, people are sitting in a room and taking L 17 quotes but they don' t have the knowledge of the programs. If 18 they don' t have the information in front of them, then those I 4 l 19 quot es won' t be as useful and as meaningful as it should be. 20 So probably the most ?mportant part of the prioritization is to f 21 gut it together, the important information presented in summary f 1 2 1 22 form, such a form that those working on the prioritization can 23 easily follow it and understand it and then the prioritirttion 1 l 24 will be more useful. J 25 In order to do this, the information has tao be 4 l Heritage Reporting Corporation j ] (202) 628-4888 J 1 t t
t () 17 1 documented and it is also important that there is an extensive 2 discussion on the pros and cons of some of these items and the 3 final documented information is kind of a consensus of opinion A that has taken into consideration the various views. r 5 The criteria for the prioritization, w2 are deriving j 6 this from the agency goals, from the strategy plan. 7 MR. WYLIE: Zoltan, could you take about 30 seconds l t ( 8 and remind us, just give us a summary of what the strategic i I 9 plan says are the agency goals? pick of f the major ones? I' m i 10 sure you have them right in mind, liko I do. 11 MR. ROSZTOCZY: Could I have the strategy plan from f 12 my briefcase? The strategy plan has basically I think five I () 13 areas which indicate the most important goals. l 14 The strategic plan says for us to go also to the i 15 agency in terms of various areas. The first one is operating i 16 factors. The second one is power plant operational readiness. 17 The third one is future reactor licensing. The fourth is j 18 nuclear material safety and safeguards. And the fifth area is I 19 ba sie nanagement. 20 In each of these areas there are a number of goals 21 put forth in the plant. For example, in the area of the future 22 reactor licensing, this includes standardization and regulatory 23 reform. This includes severe accident requirements for future 24 applications. It includes licensing extension decommissioning. 25 This include the review of the advanced light water reactors, Heritage Reporting Corporation l (202) 628-4888 l l 1
. - _. ~ _. -, 0 1 a i !,C:) ta 1 extended light water reactors and it includes the design review j 2 of the designs. I 3 MR. WYLIE: What I was trying to do is figure out 4 1 4 which ones of those are not related to risk red' Action. And a d I J r j 5 criticism of early prioritization scheme was that the prior one l J 6 was strictly tied to risk reduction issues. i 7 MR. ROSZTOCZY: Yes. In a general sense, you might [ 8 say that each of them had something to do with risk reduction, i 1 j 9 but it is not in a form that is easy to evaluate. The effort i I l j 10 that they had that prioritization was conducted, they tried to l f l 11 quantify the risk reduction, the accomplishment that even 1 i 12 research activity is going to accomplish. In some areas, to ( 13 have a quantification of that sort makes some sense. There are I i 14 !arge uncertainties that make some sense. In other areas, for ( 15 example when the question is to enacting a new regulation, then j 1 16 to try to estimate Just exactly how much risk reduction that is f i I t ] 17 going to provide is a very diffier't task and maybe not worth l l la the type of effort that it would require to try to quantify 1 19 that. We are trying to get around that type of agency course f I l 20 without necessarily count ing in numbers what we will change. l 1 21 MR. WYL!Et Thank you. L j 22 MR. ROSZTOCZY: We also emphasize that in addition to f j ,l F3 trying to document the aDency goals it is important to see what ( ) 24 will be a considerat ion of a specific research program of those j j 25 agency goals, what will be the resources needed. Some goals I i Heritage Reporting Corporation i (202) 628-4888 i i ? i r i
l I l C:) le i i can be accomplished with small resources and others are very 2 resource extensive. And we also ask the question, we also i ~ 3 indicate with the contractor that the whole thing has to be set 4 up in such a way that we ask the question whether it is [ 5 appropriate for NRC to unductake a given researcn activity, l 6 whether there are other entities that would be doing the work, I ] 7 other government agencies like DOE or the industry. I 4 8 In terms of the responsibility of who is doing the j 9 prioritization, the overall responsibility rests with the a i l 10 director of RES. He will be doing the final prioritization, j 11 This will be based on information that will be gathered [ i 12 together, assembled for him in such a manner that it is easy to { O l 13 work with and the recommendation on the prioritiration will be j 'l 14 made to him by the director of regulatory applications i { 15 division. i r r 16 In terms of the results, the ground rules were that i l J 17 the research activities would have to be prioritized. As you 18 go down the organization, the research program itself is ( ) 19 subdivided into five program categories, and then the five l 1 q 20 program categories split into 19 programs and in the 19 I 21 programs we had 72 activities, and then we contract out these t i 22 activities and we have approximately 300 contracts supporting i 23 the 72 activities. J s 24 To give you some examples, in terms of a program, ] 25 reactor primary system integrity is one of the programs. That [ 1 Heritage Reporting Corporation (202) 628-4888
f i ( 20 1 would be one of the programs out of the 19 is reactor primary i 2 system integrity. And then the activities under that program ) t ] 3 would be items like pressure vessel safety, piping integrity, [ t l 4 inspection procedures and techniques. l J i 5 And then under external events we have activities 6 like earth science, or seismology, component response to l l f j 7 earthquakes, orientational seismic analysis and seismic design l 1 8 margin matters. t l I 9 MR. MICHELSON: Question. What level are you trying i i 10 to cite your priorities? j s i 11 MR. ROSZTOCZY: We are going to use the activities ( 12 level. This will be the level that we are working with and j 13 each activity individually is going to be compared against the i l 14 criteria to see how does that activity contribute to the I 15 agency's goals. l 16 MR. MICHELSON: Some of these activities though may 17 involve a number of contracts, of course, some of which perhaps I 3 18 aren' t all that important and others of which might be quite I ( i ) 19 important if looked at individually. And yet your priority is l I 20 set at activity levels. How do you reconctie that? You know, ) 21 if an activity is not a high priority then ! Duess all the 22 cont ract s under that activity are not high priority. ) 23 MR. ROSZTOCZY We believe that a very impota \\nt part l 24 of prioritization is to display and present the typ* 1 l 25 information that you are asking for and that is probably more I Heritage Reporttng Corporation (202) 628-4888 1 i
i i ([) 22 1 i v.s. 't a nt than putting simply a priority ranking on it. So in ) 2 the process, if an activity will have a priority ranking, it 3 will be one of the most important, and on every program there q ) i 4 will be a one page summary which is going to address the type 5 of thing that you mentioned. A narrative summary that compares l 1 6 it to the various criteria and within that program if there are i j 7 elements that are very important then it will be pointed out in l L l j 8 the narrative, and if there are some elements which appears to l 1 l 9 not be, that will go in the narrative also. l 1 j j 10 MR. MICHELSON: How can you be sure though that i J 11 important contracts aren' t dropped because maybe the activity J i F j 12 as a whole has a lower priority? 13 MR. SIESS: Carl is assuming that you are going to do 1 l l 14 something with your priorities, and he is sort of assuming that t J 15 something with a low priority is going to be dropped and a high } 4 j 16 priority is going to get done. Is that a valid assumption? Is 17 he going to do something with these? 3 1 1 j 18 MR. ROSZTOCZY: Not exactly along those lines. We i 19 look et number one, we are not looking for a ranking that each i ? j 20 of these activities would be individually ranked. We were i 1 I 21 simp.y looking for grouping and we are considering grouping l ) t j 22 into three groups. So there will be one group, the top group, i 23 which is kind of lifted out from the rest saying that these are j 3 t j 24 some of the very important, most important activities that we 2 j 15 are working on and these should get proper care and proper i i 1 l 1 Heritage Reporting Corporation I (202) 628-4888 1 i
l I i 1 allocations and so on. Then there would be the middle group E which is the more normal group and then there would be the 1 i i 3 bottom group. The bottom group would be the one that you turn l l 4 to if a budget cut is forced on you, for example. \\ I 5 The difference being that if there is a budget cut, l i 6 then we wouldn' t go to one item from this group, then we would i i 7 look at the whole group, the whole group would be kind of a a potential area and then you would look at the summaries and the 9 summaries would tell you that in a given activity there are [ 10 certain things which are important and there are certain things 11 which are somewhat less important and you might end up copying 12 part of the program as opposed to studying the program. () 13 MR. MICHELSON: You just use it as a guide as to 14 where to look for individual contracts to cut? f 15 MR. ROSZTOCZY: That's right. l 16 MR. SIESS: Why wouldn't that be true in the top 17 group? Just because it is an essent ial act ivity doesn' t mean [ 18 that all the research being done in there is equally important. 19 MR. ROS/TOCZY: That is correct. That is correct. t 20 And that would not be overlooked. But at the same time, you i 21 have to recognize that there is a relatively large step from I 22 here to here and if you would try to individually 4 valuate all 23 of these, it is much harder to see the overall plature. So i f f i E4 you have to start somewhere and we felt that this was the most i l l 25 appropriate place to start, and this is what we gave to the Heritage Reporting Corporation (202) 628-4888 i l i ) J
I l { l i I O e3 1 contractor. The contractor has been working with it. They 2 have some views on it. And you will hear their views. [ r 3 MR. SIESS: I agree there is a very targe step from A there to there because above that line you are not doing l 5 anything. Below that line, you are spending money and you are 6 getting answers. And that is a big step. But the whole meat 7 of the research program, the whole cost of the research program O except for overhead, is in that bottom base of the pyramid. I l 9 MR. ROSZTOCZY: Not 100 percent, It also happens 10 that this line here is the line for our budget. These 72 [ i f 11 entries are line items in the budget. They don' t show up in 12 the budget individually any more. So many of the budget l O f 13 considerations are decided on this level. There is usually l i 14 some narrative on the budget also but they are decided on this { 15 level and counts are frequently made on this level. 16 MR. SIESS: Now you just brought up the same thing l 17 that worried Mr. Michelson. You are going to throw out at the i 1G activity level. Your budget is paper. And I realize the [ l 19 budget is very important. But there is no money spent at that { l \\ i 20 level. The money is spent down at the bottom end. The results l l 21 are obtained down in the bottom end. The research is done 22 there. Good or bad it is done down there. I don' t care how l l 23 much evaluation you do of the questions. Once you have good 24 questions, the research is done down at the bottom. 25 MR. S!ESS: But the research comes from the activity O Heritage Reporting Corporation (202) 628-4888
\\ a f 24 i level. 2 MR. ROSZTOCZY: Dr. Mattson would like to make a 3 comment. 4 M R. MATTSON: You cannot talk this subject in 5 genera 11tlos. You have to deal with it in specifics. And we 4 t i t 4 6 are going to give you a specific example. We put sotne things l l 7 in a lower priority for the program, but has within it some 1 f 8 good activities and you will see in the narrative sum: nary that j j i 9 there is enough information to pick them up and make decisions ] f f 10 about it. l 1 11 MR. MICHELSON: Unfortunately, I have seen a case J r s ) 12 where en activity was dropped. It wasn' t a lot of money but it [ 13 wasn' t appreciated that buried within that acttvity was an l l 14 important F7N that got dropped because you drop at activity j j t } 15 levels in the budget process. [ l } 16 MR. MATTSON: What he's trying to tell you is the new { ] ) j 17 method takes cognizance of that happening in the past and tries l 18 to protect against it and there is a specific device in the f 1 19 method we have developed to deal with it, both ways. The low f 20 priority activity that has some good FIN nurebers in it end the J 21 high priority activity that has some bad FIN numbers in it. 22 Both should be found. l i 23 MR, MICHELSON: That's great. If that's what you're l ) 24 doing, that answers the question. Thank you. j l 25 MR. MOELLER: You have a line item veto. l i I I Heritage Reporting Corporatton j (202) 628-4888 \\
1
- 0 l
3 1 MR. MATTSON: You have a line item veto but still i 2 keep a couple projects within it if they are important for the 3 mission of the agency. ) A M R. WARD: Zoltan, slightly a differemt question. In l i l 5 the past, I think a good bit of the whole RAD effort of the i i 1 6 agency was conducted through technical assistance contracts. j J i 7 And is that going to be integrated here some way or is 1 O technical work under those sorts of arrangements going to be I 9 minimized or confined more to real technical assistance work? l j l 10 MR. ROSZTOCZY: Sometimes some of the research 11 activity within the office of research is called technical 4 i f i 12 assistance. In addition to this, NRR has some technical [ 13 assistance of their own to perform their functions. Technical t 14 assistance is done in research on the part of the l } 15 prioritization, so that will be a factor. The NRR will not be l 16 factored in in the sense that the NRR activities will not be j 17 prioritized. ) 18 However, this prioritization will be reviewed by the ) 4 r 19 user offices, including the NRR, and in terms of any interface j 20 with NRR, any potential interface, when they are doing certain ] 21 things and maybe because of that, the importan e of something f 22 or the urgency of something that they will be doing is f ( 23 affected, that will be factored in. 1 i t 24 MR. REMICK: Zoltan, I assume that most of those 19 j I l 25 programs are nuclear power plant safety programs. Can you give ( l Heritage Reporting Corporation t (202) 628-4888 l t )
4 5 i ([) l 1 tne a rough ideo of what percentage of them are not, or give me 4 i j 2 a couple examples? 'i l 3 MR. ROSZTOCZY They are pretty much, all of them I r 4 are, there are items in it which ! am not sure exactly what f 5 type of definition you are looking for but one of them is L .1 6 regulatory improvements which would be development of 1 1 7 regulations basically. Management of generic issues is one. l 8 Waste management, there is a category which includes waste 3 9 management. High level and separately low level waste j j i 10 management. t ) ] 11 MR. REMICK: They are? 1 12 MR. ROSZTOCZY Yes. Everything is there. This is j () 13 basically our budget. The data is broken down. i a ] 14 MR. MATTSON: Zoltan, there is another answer to Mr. i i 15 Moe11er's question that helps with an earlier question. You 6 16 maid what about the tech. systems work, will it be reflected. i i l 17 If you start with the strategic plan and say what are the i l 18 strategic questions that the agency is addressing, then match l 19 up to those strategie quest ions the exist ing research program, l I j 20 in theory you will *1nd some holes in strategic questions with i ) 21 no research. perhaps those questions are being addressed by l i ] 22 NRR technical assistance, for example. perhaps they are being j 23 addressed by nothing. And perhaps that is all right. But i ) 24 following this process, starting from the strategic plan and } 25 going through st rat egie quest ions into the research program, l Heritage Reporting Corporation I ] (202) 628-4888
I i I i 27 1 you are able to identify the holes and then see whether they i 2 are filled by NRR tech. assistance or something else or nothing 3 at all. i 4 MR. ROSZTOCZY: The result will be presented in a ( 5 suitable form and that is ! think very important to us, that is 6 a part which will then be circulated and commented on by the 7 other offices and will be available for other comments like 8 ACRS comments. Based on these grounds rules we set out a 9 certain approach, the NRC is directing the activities, NRC/RES. 10 The contractor has been hired to support us in developing the 11 methodology of the process, to document the criteria and the 12 procedures, help us to collect and assemble the information, 13 develop a to.'m of how the information can be displayed in a 14 summary form that is easy to understand and easy to read, 15 conduct a trial prioritization and then assist us in the final 16 documentation of it. 17 Once all of this is done, then we will have a 18 document in our hand that we can kind of call as an 19 intermediate petoritization, and then we intend to circulate 10 that and discuss it within the agency, finalize it and af t er it 21 has been finalized and all the information displayed, then it would go to our office director, Dr. Batcher, and he would do 23 the final prioritiration. 24 MR. SIESS: Zoltan, ! guess what I am hearing is that 25 when you get down to the nitty gritty in the budget of deelding Heritage Re port i ng Corporation (292) 628-4888
3-t M ,? .O ea 1 what you are going to do and what you are not going to do with 2 the money you have, that is still goitig to be decided by either l 3 Eric Bechter or somebody sitting around the table. But theyl ^ 4 are going to have, to aid them in making that decision, all 5 sorts of facts and figures that come out of this process. Is 6 that a statement of how you are going to use this stuff? 7 He will have information in summary form assembled i 8 for him that he can uee for those decisions. Furthermore, this ) j 9 information in front of him will be inforwation that has been 10 thoroughly reviewed and agreed on by the entire' agency. l 11 MR. SIESS: But'this methodology is not in itself l 12 going to come out with a list like we saw a few years ago, the { 13 so-called hit list? It will enable somebody to develop a hit l l 14 list, but this will not grind out, if. I stuck it in a computer, 15 an ordered list of projects? 16 MR. ROSZTOCZY: Not an ordered. list but it would come I 17 up with these categories and you will see an example'of'what l inasummaryformsoitwoulddi-get you in a i 18 does it look like 19 certain direction but no way weuld it.. limit you from looking 27 others. You could look at other s, and sorecyinformation is r F1 available for the others.the same way. 3. s. 22 MR. MICHELSON: It certainly invites you though to l 23 drop certain categories.quite quickly, even though they may be 24 important to contract. J 25 MR. ROSZTOCZY: That is an important question and I O Heritage Reperting Corporation 1 (202)s 628-4888 i s s w
) 29 s 0 1 think we are kind of wrestling with that one. In the first 2 round you probably will not see a lot of that but in following 3 ones mayoe you will see a lot more of those programs that we 4 are not doing at the present time, programs which we believe 5 have definite importance but for one reason or another, they 6 are not being pursued right now. Eventually we would like to 7 have those included also, so one could also say what should I B add to the program as opposed to what I might cut from the 9 program. 10 MR. SIESS: Jesse 11 MR. EBERSOLE: Yes. Zoltan, it is fascinating to me 12 to see this massive ef fort 11y focus on one rather simple 13 topic. We are trying to keep the core wet on nuclear power 14 plants. Keep the core wet and we have it made. Now, is there 15 buried in all these complex programs, procedures which were not 16 recognized prior to TMI II which I will call studies of cascade 17 evolutions and interceptors that you must put in place to stop 18 critical cascade aides? In 1974 it was known and proclaimed we 19 needed some sort of inventory measurement in reactor vessels. 20 It never came to pass until we had accidents. Is there 21 something here that would preclude these subtle evolutions that i 22 lead us into trouble in the context of what I will call in 23 quotes -- l 24 MR. SIESS: Are you talking about research, Jesse? 25 MR. EBERSOLE: I'm trying to get to what is research. l O l Heritage Reporting Corporation (202) 628-4888 \\
i 1 MR. SIESS: I think we will have to wait 2 M R. EBERSOLE: I don' t know where research breaks 3 away from fundamental physical investigations in heat t 4 transferred fluid physics, et cetera, and when it gets into 5 adequacy of design. 6 MR. SIESS: If you cannot answer that, somebody else 7 can. 4 8 M R. ROSZTOCZY: Let me just very briefly say that the 9 process forces you to ask the question what.is needed to 10 accomplish the agency goals. So we are starting on the agency i 11 goals. It kind of directs you in this way. But one more step 12 is needed there. Somebody has to propose sorne of.those other 13 items before they really get into the prioritization process. 14 MR. SIESS: Let's wait and see what comes out of 1 15 this. I 16 MR. ROSZTOCZY: Mr. Chairman, that completes tny 17 presentation. i 18 MR. SIESS: You are almost on schedule. And even 19 though the break was put in at 10:00 O' clock, I think people 20 like to refill their coffee a little earlier than that, so 21 let's take a few minutes right now and then we won' t have 22 another break until lunch. l 23 (Whereupon, a brief recess was taken.) 24 i 25 i 4 l Heritage Reporting Corporation l (ee2) sea-4saa I
(:) 31 i ] 1 DR. SIESS: The meeting will reconvene. Let me make 2 a couple of comments, Zoltan would very much like to get-3 reaction from the subcommittee. I have told him that the L 4 subcommittee does not act in the sense of writing reports or 5 letters anymore. And he's not quite ready to bring this sort 6 of thing before the full committee, they want to go a little 7 beyond this. But he will be interested on our comments on, I l 8 guess first the need to prioritize these things and I don' t 9 think there is much question about that, we' ve addressed that 10 in the past. 11 Second, whether this approach has merit and something 12 that should be carried to a conclusion. It's not exactly a ) 13 cheap process. And they' ve got to be sure it's worthwhile in 14 order to go through with it. 15 There is no way we can strictly concentrate on the i 16 methodology here, because we are always going to be comparing L 17 it with our own views. But we have been told that we won' t l 18 really understand the methodology until everybody has had a j 19 chance to talk, that there is more to it than is in the report, 20 which I think some of us have read. l 21 So, we will do the best we can. I don' t want to 22 discourage questions that we need to ask or understand what 23 they' re talking about. I' d like to hold down as much as we i I 24 can, things in terms of specific projects except as they are 1 25 essential to understand in the process. And if Ne have any l Heritage Reporting Corporation (202) 628-4888 i
C) 32 1 time at the end maybe we can discuss this and see if there's 2 any consensus in the subcommittee as to the merits and whether 3 they should continue it. 4 Yes. i j 5 DR. SHEWMON: We will get to some examples later on 6 DR. SIESS: I got that in question. 7 DR. SHEWMON: I see nods to the affirmative. j 8 DR. SIESS: The three panel members represented three f I j 9 areas that they both individually and collectively reached 1 10 decisions, and they' re going to take us through some examples, 3 i 11 So those are exactly -- I don' t want to restrict you, but I' m 12 trying to say, let's see if we can get through the whole thing f 13 at least once over with understanding, what we need to do to q i 14 understand it. i i i 15 Larry, we' ll let you go ahead now. I I i 16 MR. YBARRONDO: Thank you. I' m very pleased to be [ 17 here. There i s, in listening to the questions on the side f j 18 there, it is clear that there is a great deal of interest in 19 this topic and we believe it to be very important. l 1 20 We welcome your critique and feedback and we' ll { 21 solicit that as we go along. There will be several other j i 22 speakers after me. Roger Blond is going to describe the trial l i i I 23 application of this methodology and Roger Mattson is going to 3 l 24 describe the results and then Jim pittman will have concluding 25 remarks. But I' m going to describe the methodology as such O Heritage Reporting Corporation (292) 628-4888
) 33 1 that we have evolved. 2 (Showing slides.) 3 MR. YBARRONDO: The importance and relevance of the 4 NRC Research Program, communication is a theme on this slide. 5 We picked this up in the reviews that we did talking with other 6 people, reviewing the previous efforts, reviewing the strategic 7 plan, reviewing the National Academy of Science report, this 8 theme of the importance and relevance of the research program 9 needing to be communicated better continually came up. 10 We took these concepts and views that we saw from 11 reviewing the background material and developed this schematic 12 of how we believe it flows. The strategic plan is on the top 13 here and out of it come agency goals. They are articulated in 14 the strategic plan, in the back section on page 22 of the 15 strategic plan by NRC there are some -- there's paragraphs that 16 talk about the research philosophy. In the National Academy of 17 Science report on page three there's a summary of the 18 philosophy and principles that they believe a good research 19 program should promulgate. 20 Out of these goals flows safety assurance questions. 21 And then there is a question of research prioritization. We 22 pick the activity level. In that chart that Zoltan Rosztoczy 23 showed you, that's A-10 in the previous presentat ion, so slide 24 A-10 is the one here where there was 72 activities and we 25 selected some eight of those activities for this trial Heritage Reporting Corporation (202) 628-4888 -.,-t
O 34 1 prioritization. 2 The results of this will be used for planning, 3 allocation of resources, in particular to facilitate 4 communications. I' m going to show this slide once or twice 5 more as we track through this methodology for you. 6 The importance and relevance of the research program 7 then is based on what we have seen in the strategy plan and the 8 agency goals. As indicated. we reviewed, we used that 9 strategic plan. As a matter of fact we dissected it in some 10 depth. 11 There is a first report that we t urr..'d in and I' m not 12 sure whether you had a copy of that or not, but the material i 13 was pre-decisional and we were sensitive about using it too 14 freely, but it is in the first report. 15 As I mentioned in the first slide, we reviewed some 16 documents. Here are some samples of them, in policy and the 17 planning on the five year plan. There was some work shops held 18 in Baltii.are by people in the SES where they talked about the 19 research program and the philosophy, the priorities of that 20 program. [ 21 We looked at the research philosophy, again in the 22 strategic plant the National Academy of Scienceal our own views 23 also. We discussed these plans with the senior NRC staff, of 24 people at the EE0 level, the regulatory, NMSS and research. 25 Our intent was to develop as complete and holistic a Heritage Reporting Corporation ] (202) 628-4888 d
a ( 35 1 view of not only what went on before but why people held those 2 views and what had happened previously. 3 We did notice in the strategic plan that the 4 priorities were not evident. And so, the results that are L 5 going to the application and the results that are going to be 6 shown to you are based on our interpretation of what the 7 priorities would be. In a full development of this methodoloDy 8 there would be a prioritization in this strategic plan ahead of 9 time. 10 Yes, s i r. Was there a question? 11 DR. SIESS: You can' t believe you went that long 12 without a question. (G_) 13 MR. YBARRONDO: I' m going to skip slide B-4, it is 14 the strategic plan analysis. And again, there are quite a 15 number of sheets of paper which show a dissection of that 16 strategic plan into the goals and activities that flow under 17 them, and that's going to be important. 18 I do have a slide which you can' t see, and by the way 19 there are going to be several you can' t see well. It will be a 20 blur of print back there. 21 This is one of the five program areas, and we looked 22 at the overall goal in it. And in the strategic plan these 23 items are well in pros form. And we broke it out into this 24 block form so that we could follow and understand more clearly 25 what we were reading, so we could really focus our attention on Heritage Reporting Corporation (202) 628-4888
l 36 1 the key words that were in the plan. 2 MR. MICHELSON: Does that slide appear in the report? 3 MR. YBARRONDO: In the first report. 4 MR. MICHELSON: Which report? 5 DR. SIESS: Well, that slide is in the handout. 6 MR. MICHELSON: I know it's in the handout, but I l 7 wanted to look at the other ones and I just wondered what 8 report it came out of. I 9 MR. YBARRONDO: That was in the Task I report that we [ 10 turned in. 11 DR. SIESS: We don't have Task-I. 12 Larry, I didn' t quite understand, the three 13 categories, accident prevention, accident mitigation and 14 generic safety issues, that previous slide. 15 MR. YBARRONDO: Yes. Those were categorized. t 16 DR. SIESS: Why generic safety issue under one or the 17 other? f 18 MR. PITTMAN: Well, whoever put the strategic plan I 19 together, that's how it came out. 20 DR. SIESS: Okay. 21 MR. YBARRONDO: We didn' t try to change items like 22 that as we found them. 23 DR. SIESS: I thought you said you broke those out of l i 24 the strategic plan, but they are in the strategic plan in those j 25 three categories. l i Heritage Reporting Corporation j (202) 628-4888
L i 37 1 MR. YBARRONDO: Yes. So we' re at this level.right 2 now. And where I want to be connects is how we went from that 3 strategic plan to involving what are the safety assurance I 4 programs or questions rather. We derived those questions.then 5 from the strategic plan. We believe it would have been better i 6 if the plan had prioritized those areas, but in the absence of 7 that we did that ourselves based on our own experience and l 8 knowledge. We think if this being done, then someone would I 9 organize those differently. 10 MR. WARD: Larry, what do you mean by safety 11 assurance questions? 12 MR. YBARRONDO: Those are the questions -- hold that 13 for Just a moment. 14 MR. WARD: Sure. l 15 MR. /BARRONDO: I' m going to get to it in about a 16 slide or so and it will be clear. In fact it is questions that 17 we would derive or put into several categories. 10 These questions that we derived, and I'm going to 19 refer you to them in appendix A of the report that you have. 20 The questions I' m speaking about now are in appendix A. They l i 21 were derived from the strategic plan. l i 22 DR. SIESS: That's where I' m confused, because your j 23 heading there says "Research goals are. translated into safety 24 assurance questionst" and it seems to me that's the reverse of I 25 the process. That safety assurance questions should be l 1 Heritage Reporting Corporation (292) 628-4888 i t _J
() 38 1 translated into research goals. How do the answers come before 2 the questions? 3 MR. YBARRONDO: The use of the word "Goals" as I look 4 at it, it is goals. And when you look into the strategic plan 5 they also talk about strategies for implementing the goals. I 6 believe we should have put goals and strateDies that we see in 7 the strategic plan are translated into these questions. But 8 the questions are essentially the areas in which information is 9 needed. 10 What I would emphasize is that the process is 11 dynamic, that is, you have to cycle back. If you think to the 12 diagram that I showed you a moment ago, let me put it back up 13 i7r you. 14 DR. SIESS: The point is, you arrived at a list of 15 questions without any regard to what research was already being 16 done. Your questions are derived from the safety assurance 17 goals. 18 MR. YBARRONDO: That is correct. l 19 DR. SIESS: Wall, that's why this heading is so 20 misleading. You' ve got research being translated into 21 questions, and the questions didn' t come from research, they 22 came from the plan. 23 MR. MATTSON: Let me see if I can say something 24 observing this dialogue from the side that might help. 25 fg The NRC strategic plan is designed to address certain V Heritage Reporting Corporation (202) 628-4888 i
39 1 questions which describe the mission of the agency. L 2 Unfortunately, those plans weren' t articulated in the strategic 3 plan. Research is also designed to address questions. Does it 4 address the right questions? Is it complete or incomplete in ^ 5 addressing the question? 6 We first had to have the questions. The questions 7 are implied by the strategic plan. So we went to the strategic 8 plan, dissected it, studied it and said, "What are the 9 questions that it's addressing?" i j 10 Now, we went a little further and said, "Well, 11 there's some quest ions it doesn' t address that are properly 12 within the authority of NRC." So we added a few. You won' t 13 find that they all come directly from the strategic plan. 14 There's some judgment in making these questions. 15 The questions are in appendix A. They' re designed to i 16 be complete. Every t ime you look at them you' ll find a little 17 nuance here or there that you want to add to them, and that's 18 why this is a growing process we' re involved in here. That's r 1 19 one of the reasons we' ve come down here. If you see questions l 20 that you think we haven' t covered, that's something that ought 21 to be brought out. Other people are being asked, "Are there 22 other questions we have luft out?" 23 The intent is to be broad in asking these safety 24 assurance questions which describe the overall role of NRC. 25 And then against which you can compare the existing research { a Heritage Reporting Corporation (202) 628-4888 i
-sU 40 i program and see whether it's complete, incomplete or 2 cumulative. 3 MR. WARD: Could I ask a question just to see if I 4 understand now. I was confused by your title there, too. I 5 guess what you' re saying, it seems to me that title might say 6 agency goals are translated in the safety assurance questions. 7 Is that closer to what you really mean? 8 MR. YBARRONDO: Yes. 9 MR. WARD: I mean, at that point maybe you haven' t 10 decided whether research is required or not. I mean, in order ) 11 to reach certain agency goals there are some gaps in knowledge 12 to reach certain agency goals and some of those can be ) 13 addressed by research and you can formulate those in specific 14 questions, I guess. 15 Are those the safety assurance questions? 16 MR. YBARRONDO: Yes. You' re going to find that there 17 are some questions for which there are no programs, and some 18 programs where thern may not be a question. One of the areas, 19 at least in the area of the strategic plan that we looked at 20 had to do with accident management. And we didn' t find t 21 accident management specifically addressed in this strategic 22 plan, although there are programs dealing with accident 23 management. 24 So we followed this process tremendously profitable 25 in terms of testing the completeness in the scope of a research Heritage Reporting Corporation (202) 628-4888
41 1 program. What is the information that is needed? Why is it 2 needed? And then tracking that against the programs. 3 In a sense then, what we were -- Just back tracking. 4 MR. WARD: Would you have been better off or worse 5 off, dramatic in either direction, if you hadn' t had the 6 strategic agency plan to start with? 7 MR. YBARRONDO: I believe we would have been much 8 worse off. I believe that is the reason that the previous 9 efforts had so much difficulty. This process, in our view, has 10 to be integrated from the top to the bottom. You think of that 11 triangle that was shown to you by Zoltan, starting at the top 12 of the strategic plan and ultimately culminating at 300 FIN 13 plans down below. That is to be an integrated process from a 14 management and a technical point of view, where things come 15 apart. Did I answer your question? 16 MR. WARD: Yes. I hope you would say that. 17 DR. SIESS: You' re putting a lot of faith in the 18 people that wrote the strategic plan. 19 MR. WARD: In past years a long range research plan 20 had to be almost a surrogate for an agency plan, and it was 21 putting too much of a burden on it. 22 MR. YBARRONDO: Well, with the questions then, as we 23 derived then, we were then faced with the issue of, "How do we 24 discriminate these questions one from the other? What 25 categories can we put them in?" We believe they' re going to Heritage Reporting Corporation (202) 628-4888
) l i () 42 j 1 change, as we discussed. 2 We choose three categories.' We had very substantial 3 debate on these categories and the definitions. We started out i i 4 at one point, I think we had five of these, we wanted to be i j 5 able to discriminate, measure a little bit better. ? ] 6 When you get more categories you get more data. It's i 7 more difficult to process it. l 8 We finally settled on what you see here. We are in l 1 9 the process with this trial prioritization of testing these 10 definitions. We believe that they will yet change as this 11 process goes on. l 7 J l 12 Frankly, we' re reasonably happy with the first one I l 13 and the last one. We have some -- we need some feedback and + r i 14 critique on that middle one there. On the definition, for 15 example, of a bench work. There's a concept in this that deals 16 with timeliness and effectiveness, and it wasn' t clear to us l 17 even at this point as to how to effectively communicate that. 18 We' ll do that but that's an area where if you have some l j 19 questions or comments now or later we would be happy to get i 20 those. i 21 DR. SIESS: I' m just curious, throughout the whole I j 22 thing you always had three categories. How did you dweide that i 23 there should be three categories and not two or four? i l 24 MR. YBARRONDO: That was based on Judgment and our 25 experience. And also, we had a blackboard and we put all the 4 l i j Heritage Reporting Corporation (292) 628-4888 j
N)J f 43 1 criteria up and we took some trial samples and we decided that, 2 gee, if we had five categories in one place and three in 3 another, that we would lose some of the consistency between the 4 criteria. We wanted a complete set of criteria, but we wanted i 5 them to be consistent, so that we could compare them more 6 readily. 7 There's nothing magic about the way we settled here 8 other than to keep it simple and to keep it focused and to be 9 able to measure. 10 DR. SIESS: You started with two, say high and low, 11 and then you find some fuzzy ones so you want something in the 12 middle. ) 13 MR. YBARRONDO: Yes. There's a science, the art of 14 asking questions and measuring people's responses which means, J 15 I' m neutral, I really like it or I don' t like it at all. I 16 DR. SIESS: Now we' ve got irrelevant on there. 17 MR. YBARRONDO: Well, we wanted to be diplomatic. 18 DR. SIESS: Well, at this stage, you know, you' re 19 developing the quustions out of the plan and you probably 20 shouldn' t come up with any irrelevant. 21 MR. YBARRONDO: You' ll see when we get to how we pick 22 some of the programs. 23 D R. SIESS: GettinD the programs this would be i 24
- relevant, 25 MR. YBARRONDO:
There are some very important i Heritage Reporting Corporation (202) 628-4888
? () 44 1 programs. There are some that are, where it's difficult to [ 2 say, and I'm sure there are some where there would be a 3 consensus that they may not be needed. i 4 DR. SIESS: Forrest. i 5 MR. REMICK: Larry, I guess I have more problems with 6 the first one than I do the second and for this reason, when I h 7 think of things that might be done having an immediate impact-8 on health, I don' t see that unless there's a plant out there, I 9 looking at nuclear powerplants, a plant out there that is I ? 10 affecting the health right now. If that's the case, then we 11 could do something to immediately impact the health. Otherwise 12 it seems to me that the type of things that we might do would i O' 13 eventually impact on the health. i 14 So I guess I have more trouble with -- you said you l 15 were satisfied with the first one and not so happy with the 16 second one. I guess I' m the other way around. If the word i 17 "Health" wasn' t there, if it was "Risk," then I could agree. i 18 Risk and safety. I 19 Give me en example of something that might be done j 20 that would have an immediate impact on health? 21 MR. YBARRONDO: Roger. 22 MR. MATTSON: I put my hand on a sheet of paper that 23 has all of the questions in all eight areas tha; we looked at 24 in the trial method. I'm looking at what I'm going to hand out 25 and there are one, two, three, four -~ four of them that are O Heritage Reporting Corporation (282) 628-4888
a 1 1 1 I 1 i all black on this funny little chart, which means that t hey' re 2 vital to a vital area. You' l l see some examples. 3 DR. SIESS: Wait a minute. Are you making i 4 distinctions between health and safety? d t i 5 MR. REMICK: Yes, I am. l 6 MR. MATTSON: Let me Just give you one. One would be j 7 the work one containment. t 8 M R. REMICK: But changing that isn' t going to affect i 9 anybody's health. It might affect their safety. f 10 DR. SIESS: Are you making the distinction between i 11 health and safety or are you just putting it in there as health } 3 i i j 12 and safety and requiring it? } (f 13 MR. BLOND: It's Just the health and safety j 14 requirement. d j 15 D R. SIESS: L rislate it as safety, that's what they ( 1 t 16 mean. 17 MR. REMICK: I guess I don' t like their choice of I 1 i j 18 language. Health throws me off. l s 19 MR. YBARRONDO: Examples of these questions are in { 20 appendix A of the Task !! report that you have and these f a l l 21 definitions which I just gave me of what we call the safety 1 22 assurance questions are also there. I 23 I might add that on these questions that you see back i 8 l 24 there one could get more sophisticated and weight those f 25 questions individually or by categories. And that type of 1 i i ? Heritage Reporting Corporation { (292) 628-4888
() 46 1 decision would have to be made at the strategic plan level. 2 DR. SIESS: You said examples on appendix A. 3 MR. YBARRONDO: pardon me. Those are the questions 4 which we used. I stand corrected. 5 Now, having the questions then we needed critical 6 criteria to evaluate the research activities. This is one of 7 the slides, it 's B-8 in your handout which you -- fortunately, 8 I wanted to show you the complete slide or I am showing you the 4 9 complete slide but you can' t read it well. Let me speak to 10 what is on there. 11 We just finished how we categories the safety l 12 assurance significance in the three areas. We then derived 13 three categories to measure the activities. And the categories 14 are usefulness, appropriateness and the resource req
- aired.
15 DR. SIESS: Now we' re talking about research. t 16 MR. YBARRONDO: pardon. 1/ DR. SIESS: Now we' re talking about research. i 18 M R. YBARRONDO: We' re talking about research. 19 MR. MICHELSON: Could I ask just one question. I' m 1 20 still puzzled about the difference between vital and important, i 21 one being immediate and other being eventual, seems to be the I ] 22 key words. 4 I 23 I'm puzzled why improving seismic capability is d 1 24 immediatel whereas, thermal hydraulics is only eventual. I 25 thought thermal hydraulics today is just as important as 4 Heritage Reporting Corporation (202) 628-4888 i
() 47 1 seismic today, neither of which might be a problem. But if I 2 had to make a guess, I thought thermal hydraulics well ahead of 3 seismic in terms of eventual need. I' m Just trying to 4 understand. 5 MR. YBARRONDO: You must be a speed reader. 6 MR. MICHELSON: No, I was Just reading what you told 7 me to read. 8 DR. SIESS: We' l l have dozens of questions like that. 9 I look at your list and I want to know why one is not the same 10 as three and so forth. 11 hP. MICHELSON: I'm trying to understand the basic 12 categor'.zation process. O 13 MR. MATTSON: That's a proper question for the 14 process, that's exactly the kind of subject matter you bring to 15
- he process to be reconciled.
se MR. YBARRONDO: By the way, we believe why -- that is 17 exactly why. The debate ensues and then a clearly definition 18 with results is why discussions on something like this have to 19 be held with other groups, user groups. 20 DR. SIESS: In making these lists that you made up 21 that are in appendix A, how many people had to agree, all three 22 of you or just two out of three? 23 MR. YDARRONDO: That was pretty -- I would say 24 unanimous agreement on those. We had some debates over the 25 questions. Heritage Reporting Corporation (202) 628-4888
() 48 1 DR. SIESS: You see, thare's a big dif ference between 2 what three people agree on and what nine will agree on. f 3 MR. YBARRONDO: I agree. Especially the time 4 involved. Yes, I do agree. ) 5 There were also disagreements amongst us as to which 6 categories the questions should go in. 7 Usefulness has a connotation of timeliness, the 8 probability that will be, in a generic sense, that it would be 9 applied and the probability of the success of that application. 4 10 We put that snto three categories as you see, highly 11 useful, useful and somewhat useful. Then we left out the 12 appropriateness of the research for NRC in terms of the -- that 13 is appropriate -- Joint funding is appropriate or it is 14 somewhat -- questionable rather. I 15 Then relevance, and undar relevance we have 16 vacillated there between putting hard dollars in, which you see 17 now. That is the cost to date and what's going on currently i 18 and the project ahead versus low cost, medium cost, high cost. 19 What you see before you now has the dollar numbers in i 20 there. We felt that would be more useful for managers, for 7 21 example, people who would be prioritizing and would want to see 22 what's gone on before? What does my future look like in this? 23 And what's my current rate of spending on this program? 24 DR. SIESS: Larry, the probability of success in l 25 getting an answer to a question depunds on two thinDs. One is i I Heritage Reporting Corporation (202) 628-4888 i
4i () 49 1 the nature of the question, how difficult it is. And second, 2 on the ability of the researcher. 3 MR. YBARRONDO: That is a factor. 4 DR. SIESS: Were both of these taken into account or l 5 is there a way of taking both of those into account? 6 MR. YBARRONDO: The last one in particular is I would say, we did not look -- we 7 Judgmental. We did not 8 looked at that. That 's a really good question. And it comes 9 down to digging into the 72 activities where someone observed 10 earlier the individual contracts and people -- you did -- where 11 the individual people doing the work are visible at the 300 1 12 activity level. Whereas they are integrated when you get to 13 the 72 activity level where we did this effort. I 14 So, although we are aware of and we went through this 15 National Academy report where the act ivit ies for ' 86 and ' 87 16 are all catalogued in there, we made some observations on the 17 number of activities. I believe it's fair to say we did not 18 try to Judge the goodness or badness of some of the 19 organizations which are shown in there. i 20 DR. SIESS: How much did you focus on how good the 21 questions are' There's some questions you can ask that nobody l 4 22 can answer. And there's some questions you can ask that i 23 anybody can answer. You know, asking the right questions is 24 very, very important. 25 MR. YBARRONDO: We agree. l Heritage Reporting Corporation i J (202) 628-4888 i i
i i f 1 DR. S!ESS: And did you focus - you asked the L t l 2 questions, and your list is one that you folks developedt l 1 l l 3 right? 1 \\ 4 MR. YBARRONDO: Yes. j l l l 5 Roger, are you going to comment? l 6 MR. MATTSON: There were discussions within the panel l l I i 7 about the capability of the people conducting the research. l 8 Not only the people managing it from a research -- from the } l 9 research organization, but also the capability of the people in 10 the field conducting the research. 11 And in the one example that we' re going to hand 12 around today of an activity sheet, it's an example where we are ( 13 praising of the people doing the work. When they' re all done, i I l 14 there will be examples of places where we say, "We don' t think l i l 15 these people understand the questions." l 16 DR. SIESS: But that's not really as critical of 17 their ability to answer the question. That may be more 18 critical of the person that asked the question. I I l 19 MR. MATTSON: But we also looked and discussed, will [ I 1 l 20 this project likely answer the question, in our Judgment. The f 21 answer is as good as we are. The broader the process, the more 22 people involved, probably the better the answer becomes. But i I 23 the harder it is to get a consensus on the answer. l I 24 DR. SIESS: Okay. l t 25 MR. YBARRONDO: In summary, this criteria are, simple I (:) Heritage Report ing Corporation l (292) 628-4488 1 l
(^) "2 1 as they seem, very difficult to derive and to be satisfied 2 with. And it's an exercise continually trying to seek a 3 balance between enough detail to measure and discriminate 4 versus too much detail which bogs down the process. 5 We go next to the questions and the criteria. We 6 believe we have evolved a comprehensive and consistent process 7 to gather the information. The process is described in the 8 report that you have, starting on page six. Let me take you 9 through the assessment steps and then the product of the step. 10 We conducted the interviews, and Roger Blond will 11 describe in some detail how we did that. The product of that 12 step is a tape, that is, we taped the interviews. And we will ( 13 share with you how we did that and why we did that. 14 I can tell you that in the beginning I was hostile, o 15 is the right word, toward taping the interviews. Having 16 examined all of the tapes and many times gone through them, I 17 am a proponent of the taping process. It was really very 18 beneficial. 19 DR. SIESS: What does VHSC stand for? 20 MR. YBARRONDO: That's the video cassette compact 21 form. I had to ask that question also. l 22 DR. SIESS: Let me ask you one more question, the 23 question 1 asked of Zoltan. The panel developed the questions, f l 24 so you saw no need to laterview users in NRR. All your 25 interviews were with research people. Heritage Reporting Corporation (202) 628-4888 I
l [ L 52 1 MR. YBARRONDO: At this point, yes. I i 2 DR.-SIESS: At this point. 1 3 MR. YBARRONDO: We did ask questions though of. the ) [ 4 people who were doing the research as to who their user is. l t 5 DR. SIESS: Who they thought their user was. t 6 MR. YBARRONDO: Who they thought their user was or 7 was is the degree of their interaction with that user? What c 1 i j 8 type of interaction is it, is it a telephono call, a meeting, I { 9 is it writtent how does it take place? j l i i 10 DR. SIESS: You relied on the strategic plan to give [ l l 11 the questions. You didn' t go to the regulatory staff and ask l i i 12 them what they thought the questions were. l 13 MR. YBARRONDO: We asked them what they were doing i 'i l 14 and why they were doing it. I { 15 DR. SIESS: At research. l l I 16 MR. YBARRONDO: Research. l l' l l 17 DR. SIESS: They' re answerers, they're not askers. i i j 18 MR. YBARRONDO: Exactly. l 1 19 MR. ROSZTOCZY: Let me interrent you and indicate two l '4 20 things. One of them is, as far as the quections are concerned, 1 I ) 21 they did ask the questions, they give it back to us. We are j j i 22 reviewing it now and reviewing it both in the research office l 1 j. 23 'and in the user offices. So before they are finalized go j i ) i 24 through them both. j j 1 1 25 DR. SIESS: Fine. t i 1 J Heritage Reporting Corporation l 1 taea) sea-4ssa i ] 1
() 53 1 MR. ROSZTOCZY: In terms of the interviews, if the 2 process would be done something along these lines, then the 3 idea was that when they did not look at the basic information, 4 they will tell you the people involved running that task. Once 5 they are finished with that step, that's for all the items, 6 then we will go to the user offices and discuss it with the 7 user offices, so that we go to the user office, for example, 8 and then we pull out the four or five activities which feed i 9 that and we will discuss all four or five of those with them at 10 that time. But this first step that has not been done yet, 11 that will be done in the future step. l 12 DR. SIESS: Thank you. [ () 13 MR. MICHELSON: Well, this Task II report that we I i l 14 have copies of, is that a draft or a final report? i 15 MR. ROSZTOCZY: It's a draft. i 16 MR. MICHELSON: I didn' t appreciate that it was just i 17 a draft. l j 18 MR. BLOND: Let me Just interject one other thing, 19 Roger Blond. We did go around to the ED0s on a staff, NRC 1 20 staff members and senior management for the organization with 21 the questions and como up with their opinions of how they l j! 'i 22 should be organtred and what questions should or should not be. i 1 23 DR. SIESS: Very good. ] 24 MR. YBARRONDO: That was ahead of time. It wasn' t i 25 part of the interview process, the formal interview process i Heritage Reporting Corporation j (202) 628-4888 I I
54 1 here that I' m speaking of. I 2 We repeat again, the Judges, the three of us 3 formalated the list of these questions. The definitions of the 1 4 attributes. And then acting as a panel we interviewed people 1 l 5 and we have assessment sheets which are in the Task II report 6 for us acting individually, that is, each individual judge 7 filled out an assessment sheet for each of the activities. And 8 then we took our individual sheets and acting as a panel we put 9 those together. 10 DR. MOELLER: Was there generally concurrence in your l 11 individual independent assessments? r i l 12 MR. YBARRONDO: Generally there was, although there l l 13 were some cases where we disagreed on the areas. And it was l i L l 14 interesting for me to emphasize the importance of picking 15 Judges who have the diversity of background and skills, but are 16 generally knowledgeable in an area. One person would say 17 something in that area, for example, was vitall and somebody I i l 18 else said it was relevant. i l 19 What I found -- what I observed participating in that I 20 process is that some -- if you can' t convince a knowledgeable 21 peer, but somebody that's not an expert in an area in which you 22 are an expert, there's probably a problem. 23 Why do you have the ability in bringing that kind of l 24 searching questions and why do you have that view. Out into 25 the open and getting the reasons articulated and then recorded O Heritage Reporting Corporation (202) 628-4888 l l
P (1) i finally in a composite sheet which are in the oack of the 2 report. 3 The panel composition, let me talk a moment to that. 4 MR. MATTSON: Larry, could I interrupt Just a second. 5 MR. YBARRONDO: Yes. 6 MR. MATTSON: It's important in these tapes to 7 recognize one aspect of them, two parts of them. One is that 8 the people who are being taped clearly see that their opinions 9 are going to be heard. 10 The second aspect of it is that they actually are 11 heard. When you get into these debates about, well, what is 12 the program when you' re not an expert. I mean, it's ( 13 generalists that have to make these judgments. Well, you' ve 14 got the tape to go back and look. You don' t depend on your 15 notes. You don' t depend on somebody else's transcription, you 16 can go back to the place in the tape where you asked the person 17 the question about their program. And it is factual 18 information about the content of the program. 19 Remember what Zoltan's slide said, in the past there 20 was difficulty in being absolute complete factual and 21 representative of the program you were dealing with. 22 In this case with those tapes you don' t have that 23 problem. You can always go back and double check. 24 DR. MOELLER: How long was it a typical interview, an j 25 hour, two hours, how long? l j i Heritage Reporting Corporation (202) 628-4888 I l i i
t,q - i 56 1 MR. YBARRONDO: One hour. Roger Blond will speak to 4 2 that, about how we conducted the interviews and that was 3 purposeful. 4 MR. EBERSOLE: Could you apply this to the, I think, 5 proposed slow down or shyt down of the thermal hydraulics 6 program? I 7 MR. MATTSON: Sure. 8 MR. EBERSOLE: Would that be a worthwhile exercise i 9 because it's in being right now. 10 DR. SIESS: That's the object ive. 11 MR. YBARRONDO: I'd like.to speak for a moment then 12 to the composition of the panel. [ l 13 DR. SIESS: Are you through with the interviews? 14 MR. YBARRONDO: Yes, sir. 15 DR. SIESS: I Just wanted to make one comment. 16 Looking at the list of questions you Asked, it seems to me you 17 had an excellent basis for assessing the qualit y o* the 18 research managers you talked to. i 19 MR. YBARRONDO: The last part, who was that ? l 20 DR. SIESS: Assessing the quality of the research ^ i 21 managers you talked to. i 22 MR. WARD: Yoit' re going to put -- you' re going t o f 23 make it difficult for them to co.more interviewL. l 24 DR. SIESS: I know. But the kind cl questions they 2 i 25 asked, do they understand what they' re doing ar:d how it's going i 4 Heritage Reporting Corporation r (202) 628-4888 i
i 57 i i to be used? There are some very interesting questions there. 2 MR. pITTMAN: I think Roger will touch on that. ~ 3 MR. YBARRONDO: Those questions, the composition of 4 them and the thrust of them comes from the experience that's 5 represented on the panel, and therefore we believe, as NRC does 6 this, it's important to have people with basically completely l 7 different backgrounds, whether you have three judges or four or i 8 five, the diversity of background and exportance is important j i i ] 9 to being able, as you said, ask the right question. And 10 someone said earlier, how do you know if someone is pulling the 11 wool over your eyes. Well, you know it by using people in the 12 panels who are experienced and have been there before. 13 DR. MOELLER: Were the interviewees asked not to talk l 14 to other potential upcoming interviewees? 15 MR. YBARRONDO: We asked them not to prepare for the 16 interview. We wanted their candid and straightforward 17 opinions. One of the men that we used as part of this team is j 18 Glenn Humpress from Southwest Research and he has a background l ) 1 19 in the design of questions and the structuring of interviews, i i } 20 so that you get people's candid opinions. l 21 DR. MOELLER: So if you interviewed Doctor A today, 1 a i 22 he or she was not suppose to go prep and get ready the next 23 interviewee. l 24 MR. YBARRONDO: We didn' t specifically state that, j 25 but we asked that they not prepare. Implicit in 'het was, this l Heritage Reporting Corporation (202) 628-4888 i
(:) 1 isn' t the formal test, we want your opinions and we want you to 2 describe your program for us. 3 So we tried to keep the interviews as relaxed as we 4 could so that there would be a free flowing on inner change. 5 DR. SIESS: It looks like the same kind of questions 6 ACRS subcommittees ask when they' re talking about research ( 7 program. 8 M R. YDARRONDO: The panel represents an experience in i 9 both commercial and government strategic planning and i 10 prioritization efforts. And we' re not presenting this to 11 emphasize our credentials but rather to emphasize that in 12 future panels it is important to pick people with a diversity l I 13 of skills and experiences. 14 You see these here. There's one other in this F j 15 category and that's the Department of Defense. 16 DR. SIESS: What's an internal bias? j 1 17 MR. YBARRONDO: Oh, right here at the bottom. l ) 18 Internal bias, in our view, is when you pick people who are ] 19 close to the problem and they are active participants in these i i 20 specific activities. It is in that sense that we use it. l 21 In summary, we believe that this research 22 priorit izat ion methodology provi Jes a method of ef fect ively 23 communicating the importance and the relevance of the research 1 j 24 program. 25 We believe this process that we' re describing for you 4 I i r Heritage Reporting Corporation (202) 628-4888 1 i
CE) 59 1 and we will describe the application of, and based on the NRC's 2 strategic plan and the agency goals, that is an interactive i 3 process of taking those goals, and we stand corrected on how we 4 presented that earlier. How those get translated in the safety 5 assurance questions. 6 The questions then are divided into critical 7 categories that you see there. 8 DR. SIES3: I have a little problem with the word 9 "communicating" there. The first step is to develop a retearch 10 program that deals with the importance of developments. And 11 after you develop it, then communicating. 12 M R. YBARRONDO: I agree with that. We entered into ) 13 this process ongoing. It's underway. ( 14 DR. SIESS: But still you could use it to change the [ 15 research program, not just take the existinD one and 16 communicate it. 17 MR. YBARRONDO: We believe, by the way, that this 18 process is designed to be flexible, to expand and contract, so 19 that if the program changes or it's redirected the process can 20 respond. We kept it comprehensive and consistent. We used the 21 senior people, various disciplines and skills to judge the l 22 activity. And we believe that this process is easy to use. f 23 We use the word "repeatable" here in a limited sense 24 at this point. We' ve only looked at eight activities so far, 25 and there were three judges. When we filled out our individual I f 9 Heritage Reporting Corporation (202) 628-4888
I e 60 1 sheets we tracked very well. We had some areas of 2 disagreement, but by and large the process was repeatable. 3 It is retrievable in both the tape and written form. ~ 4 And we believe that those forms and the interview process 5 facilitate or will facilitate communication. 6 DR. SIESS: As an adjective you could use, 7 think, 7 and that is the process is transparent. 8 MR. YBARRONDO: Say that again, sir? 9 DR. SIESS: The process is transparent. 10 MR. YBARRONDO: Yes, that was used at the National 11 Academy and Sciences report, also. I agree with that. 12 Roger Blond is going to describe next how we' ll take 13 this process that I have described generically and apply it to 14 the eight activities. l 15 DR. SIESS: I would suggest that if we have any { 16 general questions for Larry we save them for the end. l 17 MR. MICHELSON: General questions now, what was Just d 18 presented? L l 19 DR. SIESS: Yes. Ano we' ll give Roger a detail, j 20 other Roger a detail. We' ll accumulat e some general quest ions. 21 Thank you, Larry. 1 I 22 MR. YBARRONDO: Thank you. i 5 23 DR. SIESS: We' re out of order on here. Mattson is ( 24 going to come last. I 25 MR. ROSZTOCZY: That's correct. 1 i i Heritage Reporting Corporation (202) 628-4888
j 1 MR. BLOND: I get the fun Job today. I get to 2 describe the process that we used to conduct the trial. We' re 3 trying to go through and prove that the methodology was I i 4 effective, would meet all the objectives that were established 5 for usl to then try to see how we can convert that trial or the 6 process that we developed into a working process. Something 7 that we could have some confidence in that could be then 8 tr&nslated and used across all the research programs. 9 I will not be talking about the results of our trial i 10 processi Roger Mattson is going to go through and try to 11 indicate what we ended up with, specifically, on the programs i 12 that we did look at. I think that's a whole dif ferent thrust, () 13 different issue in terms of how we conducted the trial versus 14 what we came out with for the final product. 15 The whole issue here is that final product, and we' re 16 all trying to end up with the results of the application of the 17 method, the final priorities. 18 When we started the analysis, when we went through 19 the project our first exercise, as Larry has described, was to 20 go through and develop a methodl and then we had to determine t 21 whether that method was going to be, able to be performed, 22 whether we felt that it had a chance of success. So we 23 established some purposes for our trial. i t 24 We wanted to determine whether the criteria that were I 25 established, the safety assurance measure, the usefulness l Heritage Reporting Ccrporation (202) 628-4888
() 62 1 measure, the appropriateness for NRC to be doing this work and 2 the resources that are required, whether those criteria would 3 adequately discriminate between the activities. So you had to I 4 see whether in fact it made any difference at all to what we l 5 would be looking at. 6 Then we had to determine whether the process that we 7 were going to use was easy to conduct. One of the criteria 8 that we went into the process with was, we wanted something 9 that wouldn' t take a lot of resources, that would not be i 10 intensive in that sense. I 11 So we wanted to see whether the procedures that we 4 12 developed were adequate for the job, whether the approach that ) 13 we had taken was consistent across all the programst that was 14 very important. We didn' t want to have -- to fall into traps f J 15 that we had seen in the other past prioritizations, we wanted j 16 to know whether we could como up with a consistent approach. l 1 17 And again, we looked at the resource intensity of the l ) 18 process, both for the NRC staff and for the panelist, for i 19 ourselves who were going to actually conduct the exercise. ) i 20 Then, through the trial process we wanted to develop [ 21 the presentation of the results. We didn' t really know, as we I { 22 got into the methodology, what we wanted these results to look i i d 23 like. We needed to have some actual results before we could a 24 determine how we could communicate these results. That was a 1 25 key aspect of the trial, to develop good presentation of the (:) ^ Heritage Reporting Corporation 1 (202) 628-4888
1 63 1 results. 2 What we really wanted was to get back to the original 3 objective, planning, resource allocation and communication. 4 That's what we really wanted to come down to. 5 Then finally, we wanted ta see if we could determine 6 if the process was repeatable. If we absolutely needed an 7 outside group to conduct the evaluation, What constituted an 8 acceptable mix of talents and skills on that panel. Was it 9 necessary to interview one person or did we need to interview 10 more than one person per activity, so that we could get the 11 merging of opinion specifically on that activity. 12 So what we did was construct a trial process, a 13 procedure, where we thought we would try and get most of these 14 purposes or the purposes of the trial answered. 15 We wanted to exercise the trial against activities 16 which we thought would be meaningful and have some I 17 significance. So we selected activities in conjunction with 18 the NRC staf f to really try to forcu the process. To make sure L 19 that it would look and be able to discriminate something that i 20 was meaningful. 21 So we looked at significant programs. programs with 22 large budgets, relatively speaking, on the order of about sac 23 million a piece. 24 DR. SIESS: That's total budget? i 25 MR. BLOND: That's total budget within that whole } i Heritage Reporting Corporation (202) 628-4888 r
r ) 64 s 1 prograrn area. 2 DR. SIESS: Not per year. 3 MR. BLOND: No, that's their yearly budget. 4 DR. SIESS: It is yearly. 5 MR. BLOND: That was the yearly budget, yes, within i 6 those -- within the program. Remember the chart that Zoltan 7 put up which had the nice pyrarnid. The prograrn act ivity, I 8 think there were 19 program activitles and 72 activitles. ( i i 9 DR. SIESS: I'm trying to -- in your resources 1 ] 10 category you have a bigger dollar, as that a total -- that's i 11 called total budget againt and that's per year? 12 MR. BLOND: We have actually modified that process as 13 we' ve moved along, as we have come up with the results. 14 DR. SIESS: I'm just trying to find out what total i 15 means. I 16 MR. BLOND: In this context it was Just in the 17 selection of which activities we would look at. And we wanted i j 18 to choose active programs, things which were currently fairly l 19 significant within the research program. j f 20 DR. SHEWMON: Given a couple of these problems is, we I ~ 21 come down to seismic design rnargins resources 2 million and you i 22 say it has to be 20 million. l \\ q 23 MR. BLOND: That's what I' m trying to explain. What 24 we' re t alking about is that the seistnic and fire protection i 25 prog rard area, there are 19 such program areas, and that i O Heritage Reporting Corporation l (202) 628-4888 i i
65 4 1 constitutes four activities. These are the activities in the 2 72 activities within there. 3 MR. MICHELSON: Well, it's almost impossible the way 4 fire protection even shows up in that since it's only a few 5 hundred thousand and 20 million, and seismic must be the rest s j 6 of it, I guess. f 7 MR. BLOND: We had nothing to do with the selection r 8 DR. SIESS: That happens to be a branch name or 9 division. I 1 10 MR. BLOND: Yes. 1 f 11 DR. SIESS: It's still part of the title, 12 MR. BLOND: So each one of these program areas ( ) 13 constituted a program of approximately $20 million worth of 14 research. 15 DR. SIESS: per year. l 4 16 MR. BLOND: per year. Then we wanted to look at, at } 17 least, two diverse areas. Actually, what we tried to choose J j 18 was programs within two different divisions within research, so i 19 that we could utiltre different staff within those divisions 20 and not eat up a lot of people within all one division j 21 research. 22 And then we chose eight activities. We decided that I l 23 four within each of the program categories was a good number. i 4 24 It was something that we did have a fairly rigorous exercise 3 25 against and have a variety of activities to look at. ] j Heritage Reporting Corporation (202) 628-4888 l
(:) 55 1 Som6 of these activities were relatively expensive 2 and some were relative inexpensive across the board here. So 3 that was our selection process of which activities we would 4 actually go for. 5 MR. EDERSOLE: Tell me why I'm bothered about that 6 reactor containment area in the context of seeing it look a P 7 little bit like the old large LOCA investigative effort over 8 the last 20 years. 9 DR. SIESS: Jesse, wait until he tells you where it 10 comes out. Right now these are Just the ones they looked at, i 11 he hasn' t given it a rating yet. 12 MR. EDERSOLE: Just tell me why it looks like it's ( 13 going to be worth a lot. i 14 MR. BLOND: That's not my job at all. What my Job at i 15 this point is, it's Just to indicate -- we wanted to look at 16 this program area, at this research program. 17 MR. EBERSOLE: So you' re going to criticize it. [ 18 MR. DLOND: We' re going to evaluate it. Not 19 necessarily criticize it. 20 MR. EDERSOLE:
- Well, I' m talking about it in the l
21 present context. [ 22 DR. SIESS: Roger Mattson is going to give us the 23 results. And if you read the report you' ll find out where it 24 comes out. He' ll get to that. Right now this is the l I 25 procedure. i l l i Heritage Reporting Corporation l (202) 628-4888 I t
l O '7 1 MR. BLOND: page nine of the report lists actually l 2 how we conducted the interview in terms of which people we 3 talked to on eacn of these activities, and when we did it. It 4 spanned about a two or three week period where we tried to get 5 or maybe a two week period, I guess, to talk to the 6 individuals. And we' ll go into some detail now about the way i i 7 that process worked. 8 MR. MICHELSON: Did you talk to each of them on all 9 of the questions, even if it wasn' t their area? 10 MR. BLOND: Yes. I' ll go into a little bit of an 11 explanation of how the process worked. i 12 MR. MICHELSON: But I want to know if they 13 interviewed -- I thought they were interviewing people in their l 14 area and not another area. 1 15 MR. BLOND: That's correct. We interviewed people in 16 their area. We would ask all of the questions. We had a 4 l 17 question interview form that's in the report, it's appendix B, 1 J 18 I believe. 19 DR. SIESS: Everybody was asked all the questions, 20 but only about the projects in their area. i 21 MR. BLOND: Correct. 22 DR. SIESS: Well, I think there was one question that 23 says there's something in other areas that you think ought to I 24 be asked. 25 MR. MICHELSON: A lot of these questions are pointed O i l Heritage Report it.g Corporation l i (202) 628-4888 i l 6
(:) SS 1 to specific areas to begin with, like seismic. And are you I 2 going to ask people in thermal hydraulics about meismic. 3 DR. SIESS: No. 4 MR. BLOND: There's an interview set of questions 5 which are in appendix B, I believe. 6 MR. MICHELSON: Yes, appendix B. 7 DR. SIESS: They are very large type. l 8 DR. SIESS: And they ask one question, it says, 9 "Outside of your own area responsibility do you see any need 10 for research which is not currently being performed." And [ 11 except for that question all of the others are -- the N f 12 question i s -- () 13 MR. BLOND: It 's not pointed to any specific 14 activity, but it's pointed to try to elicit the information i 15 that we can judge how well those activities comport with the 16 various criteria that we had established. j 17 DR. SIESS: But the other questions all related to 18 that particular area of responsibility? 19 MR. BLOND: Correct. It was focused on. 20 MR. WARD: Well, I'ta sorry it isn' t clear to me, it i 21 looks N-3, N-4 and so forth. 22 DR. SIESS: The N quest ions were all related to 23 outside their areal and then all the other questions with the l 24 different letters in front of them, in general, related to i 25 their area. i Heritage Reporting Corporation (202) 628-4888
CE) SS 1 MR. WARD: I apologize. 2 MR. BLOND: I' ll explain the process a little bit, we 3 can go through some of those questions. 4 In terms of the pre-interview part of the interview 5 process the environment was something that we wanted to try to 6 establish to be a little pleasant. Not something that was 7 contentious or a problem between the interviewer and the 8 interviewee. 9 So we wanted to get up a -- be in a room with 10 windows, have some reasonable feeling that it wasn' t stark and 11 coldt and that became a very important consideration. And I' ll 12 describe one case where we couldn' t get the room that we wanted () 13 and the interview was a problem because of that. 14 MR. WARD: You didn' t have any lawyers or designers. 15 MR. BLOND: No. Strictly people that smile. 16 DR. SIESS: Excuse tie, Roger. I don' t know how many 17 members have read the report where all of this is described and 18 how interested everybody is on the mechanics of the interview. 19 We can save some time -- 20 MR. BLOND: That will be fine. 21 DR. SIESS: Yes. Let's assume that they have read it 22 and they know the mechanics of it. Because we' re not competent 23 to judge anyway. 24 MR. BLOND: That's fine. 25
- think one key aspect of the way that we would go Heritage Reporting Corporation (202) 628-4888
lO 1 through it, before we start the interview and turn the camera 2 on we would try to put the interviewee at as much ease as 3 possible. We would explain the process in detail. He would 4 actually go through the questions with him, so that he would be 5 prepared and wouldn' t have anything slap him in the face and 6 not know how to respcnd. 7 He was free to proceed at his own pace. We tried to 8 establish the interview questions to last about an hour. But l 9 some of the interviewees I think we went for an hour and a 10 half, possibly. Some maybe 30 or 45 minutes on that order. 11 And it depended very much on the pace that was established with 12 the interviewer and how he conducted itl and the information 13 that was being received. Some people had more and some people 14 had a lot less information that they wanted to put out. 15 The way we would start the interview though was we 16 asked questions about new areas. We wanted to find out 17 information to get the person away from his specific program 18 and ask them, what is it from his experience and knowledge now 19 that NRC should be doing that it might not be doing. And most 20 people were kind of -- didn' t know how to respond, actually, to i 21 the question. Some of the responses, Roger will go into a / 22 little bit of detail on that, were very interested. I think i 23 that's another area which we will evolve and look at as we go 24 along. 25 The questions themselves we evolved as we moved Heritage Reporting Corporation (202) 628-4888 )
O 71 1 through the trial. We didn' t stay stagnant with them because 2 we found some areas where there was no answers that we were 3 getting to the questions so we condensed or compressed them and 4 some areas where wo needed more. 5 In talking, within the last few days, there's one 6 area in which we want to add to the question list and that's 7 more information about the FINS, the actual contracts which are 8 let and associated with the activities. We touched on that 9 area, but not in as much detail as we probably should, 10 MR. MICHELSON: But isn' t that the critical decision 11 when it comes to what you cut and what you keep? 12 M R. BLOND: That became evident as we went through () 13 the evaluation process. And again, Roger will get into that. 14 As we're going through the interview process it wasn' t quit e as 15 evident. We were put on different marching orders, now we see 16 that we really need to get a little bit more into the specific 17 contracts. 18 DR. MOELLER: As you indicated earlier, professional 19 guidance in conducting the interviews and you just said you 20 began with asking them about their thoughts on new areas, was 21 that recommended to you? 22 MR. BLOND: Yes. Absolutely. 23 DR. MOELLER: Did the interviewers hold some dry runs 24 and rehearsals to learn? 25 MR. BLOND: Yes. We withdrew it ourselves, played (:) l Heritage Reporting Corporation (202) 628-4888
l J l 1 the games as if we were the researchers. 1 l 2 DR. SIESS: Did you change the questions at all as I 3 you went through? j I i j 4 MR. BLOND: Yes. a L { 5 DR. SIESS: You did modify them? ) ( l 6 MR. BLOND: Yes. 7 DR. S!ESS: The list we have in the report is your 8 final list of questions? i i j 9 MR. BLOND: It was the final list of questions that l i 10 we ended up with. I think after the first or second interview ) i 1 11 we made some fairly significant changes. l 1 h 12 DR. S!ESS: These are the ones you would recommend j () f 13 for future? t 1 14 MR. BLOND: There's a couple now that we think we l 15 would like to add in addition. 1 + 1 16 DR. SIESS: They' re very non-adversarial. t i 17 MR. DLOND: That was an objective. 1 18 DR. SIESS: So there was no really basis for an i 1 f ) 19 interviewee becoming defensive. l 20 MR. BLOND: Correct. And I think that's what we i i 21 found. And as soon as we got into it, there was information i i ) 22 gathering. They should have had that information on the tip of ) l 23 their tongue. If they were doing their job properly that ) i i ? 24 informat ion should be available to them. f I l i 25 DR. MOELLER: Did you check like how long the fellow j j i i I i Heritage Reporting Corporation l (202) 628-4888 I p I
t 1 1 or lady had been in the job? They might say, "I' m just new 2 here and I' m sorry. " t 3 MR. BLOND: That was one of the reasons that we 4 wanted to interview at least two people on each activity. We j 5 wanted a manager, a branch chief or section leader and a 6 project manager or the project manager themselves. As it turns l 7 out the branch chief or section leader is probably sufficient, d 8 at least in some areas. You might go to a second i f you don' t 9 think you have sufficient information. 4 10 D R. SIESS: But none of those people were new on the + ] 11 job, were they? 12 MR. BLOND: No, none of them. J 0 j U 13 DR. SIESS: They all look pretty familiar to me. The l J 14 list is on page nine. l 4 i I 15 MR. BLOND: Again, just very briefly, we didn' t think i 16 preparation was going to be necessary except if there was some i j 17 information on risk relevance, if there was -- and we wanted 1 { 18 fairly current cost information, what the budget activity was. I i j 19 Although the quality of that information has changed t 20 dramatically since we have done this, so the actual cost f i l 21 numbers we' re not really convinced of what we' ve got at this l 22 point. f i e J 23 One of the interesting things gets to the wrap-up, 1 t 24 the final questions that we asked. We asked whether or not i I J 25 t here had been any thought or any analysis done on the cost I i ) 4 1 1 Heritage Reporting Corporation (292) 628-4888 i ) i
! () 74 1 benefit of this research. As I' m sure you' re aware the CRGR l 2 requires cost benefit analysis for any new change that comes 3 i n. We were wondering whether anything had oesn done in that l 4 area in the research activity. ] ) 5 DR. SIESS: You mean the cost benefit on the use of 6 the research. 7 MR. BLOND: The use of the research. t 8 DR. SIESS: Not on the research itself. l j 9 MR. BLOND: That's correct. And there was no l l 10 knowledge of whether any of that had ever been done before. j 11 And the last thing was how much information had been i I 12 blamed in terms of the strateDie plan which is what we would i ) 13 establish as our top goal. l 1 14 In general, the strategic plan is a new document, l 15 it's something which is not available yet, it's still free ( f a 16 decision. And I think the version that we had, Zoltan now has ( i l 17 a newer version than the one that we used for our initial [ ] 18 analysis. 19 So we're unclear what kind of a response we would get i I 20 for that. In general, again, there was very knowledge about l 21 that process at this point within research. l i 22 DR. S!ESS: The strategic plan is set at a level i I 23 pretty much above branch chiefs. ! could visualize developing [ 24 a subset of strategic plan for research. There's some detail a ( l 25 little further down the line. I' m not surprised they' re not i 1 (:) t Heritage Reporting Corporation (202) 628-4888 t 4 )
i 1 () 75 5 l 1 familiar with the strategic plan. 2 MR. MICHELSON: Could I ask a question on the l 1 j 3 questions that you asked, since we seem to have some discussion l j 4 going anyway. When you asked the question, which ! think is an l j 5 important one, whether they thought the research was vital l i 6 important or relevant, did you follow that up by an explanation l .i l 7 of what vital important or relevant was before they answered [ 8 the question, similar to what's in the report, those kinds of [ l 9 definitions? { l 10 MR. BLOND: Yes. We go into that in some detail i 11 before we started the interview, so they could be thinking f 12 about it. 13 MR. MICHELSON: They weren' t just here's the i i i \\ 7 14 question, what's the answer. 1 i f 15 MR. BLOND: No, no. ( j 16 MR. MICHELSON: There was quite a bit of chit chat 17 about you' re doing. Thank you. j J 18 MR. BLOND: In terms of the documentation of the 19 process we have -- this is that C-4 tape, the VHSC-4, we have [ 20 these tapes as archives which we can go bacit and review. l 21 DR. SIESS: How long is on that tape. 3 22 MR. BLOND: It's one hour, f 23 DR. S!ESS: !s it subject to Freedom of Information 24 Act? i l 25 MR. PLOND: No. This is my own thing. Especially at i () i Heritage Reporting Corporation (202) 628-4888 i i
r i f f () 76 1 the trial process we committed that the tapes were something a 1 2 that we would use i nt erna l l y, and it's something that's an 1 j 3 internal process. 4 J 4 DR. MOELLER: One interviewee appeared on two 5 different days. l 1 l 6 MR. BLOND:
- Yes, i
7 DR. MOELLER: Why was that? 8 MR. BLOND: They had to leave. It was just a lack of l 9 time to complete it, so they had to take it up again. We tried 4 16 to be as flexible as possible to make sure that we had arranged i 11 to their schedule, whenever they had free parts to conduct the 4 t 12 interview, 13 We would take written notes on the interviews as they 14 were going along also. Then in terms of our documentation the i 15 panel members, the three of us as individuals would go through A i 16 and develop a position on the criteria. And then we would try l ) 17 to reach a consensus in a group, and that's what the scores are 1 18 that you' ll see. f 19 DR. SIESS: Roger, who actually conducted the i 1 20 interviews? 1 21 M R. BLOND: Yes. 22 DR. SIESS: Who? ) 23 MR. BLOND: Oh, I conducted the interviews. Roger [ J l 24 Mattson conducted the interviews and Dennis Myer. And we would j j (:) 25 have at least two of us or try to have at least two of us Heritage Reporting Corporation ( ) (202) 628-4888 1 l a
i 1 together during the interviews. So that, again, we would feel 2 somewhat comfortable about interpreting the information. r 3 MR. MATTSON: We had a rule that every member of the 4 4 panel had to observe every interview. We talked about whether l l 5 we ought to change because it's time consuming. But for the 6 trial all three of us observed at least every interview. [ 7 DR. SIESS: And what did you conclude for the future? 8 MR. MATTSON: You' re talking about two out of three i I 9 and I don' t think we have settled it. 1, I 10 MR. BLOND: We never had the chance to repeat the i 11 trial. That's a matter of resources. i s 12 MR. MICHELSON: Was the idea to repeat this process ( 13 once a Fiscal Year or something as a budgetary preparation or ] 14 is this going to be thought of as every two or three years? { 15 MR. BLOND: That's something that the research 1 16 program has got to come to grips with. What we're hoping is 17 that it a dynamic process. r 18 MR. MICHELSON: I was just trying to figure out how [ 1 i j 19 they -- [ l 20 DR. SIESS: ! think that may be the kind of input 21 they would like from us. Unless they get support it won' t be t 22 done at all. f d3 MR. ROSZTOCZY: Yes, Yes. We would like to have your j 24 opinion on that, but I think in general the thinking would be i j f 25 t hat in the beginning when we put together the entire area, all I i e i t Heritage Reporting Corporation { (202) 628-4888 1 l
s- --_G-a r-4- Es () 78 f 1 72 of these activities and establish this, it would then become 2 some kind of an active line. Once you are Just updating it in 3 follow-up years. And one would only work on those areas where l 4 something is changing. The agency cost changes in an area, and j j 5 these persons would be the ones conducted -- provided they come 6 out which are affected by it. 7 MR. MICHELSON: Conduct interviews only in those 8 areas. i e 9 MR. ROSZTOCZY: Only those areas. If there are new i 4 10 areas starting up, thun one would conduct occasion interview. i 11 MR. MICHELSON: Is this a particularly expensive l i 12 process to go through once you know the methodology, and you' re 13 just now talking about the cost of updating the interviews each 14 year, two years or whatever. 15 MR. ROSZTOCZY: Our ground rule was to try to keep it l 1 l 16 inexpensive. I think probably Roger can tell us how I 1 l i i 17 inexpensive it is. l 1 18 DR. SIESS: Defore you do that, what fraction of the i 19 t ot al program was covered in this? j 1 20 MR. BLOND: Eight out of 72. i ) ) 21 DR. SIESS: Eight out of 72. [ l l 22 DR. SIESS: In dollars what fraction? l 23 MR. BLOND: It was a significant -- ) i 24 MR. ROSZTOCZY: A large portion of dollars, it was I I I ) 25 close to 40 percent -- 40 to 50 percent. l I i 1 I Heritage Reporting Corporation l (262) 628-4888 1
l 1 i i r j 1 DR. S!ESS: Thank you. j l i 2 (Continued on next page.) i t I i 3 i l 1 1 4 l l 5 I I } l d 6 l l i I i 7 1 i 8 l ] 9 J l i 10 ( l k 11 1 ] 12 i 13 I i 14 l l l 15 ) l j 16 i l 17 l 18 l 19 J i i 80 l 1 i 21 ( I l 22 ) l i 23 1 1 i 2 24 j 1 3 25 O Heritage Reporting Corporatton (202) 628-4888
i O* i 1 MR. EDERSOLE: Did the interviewer or interviewers 2 realize that they should keep in mind that your basic goal was 3 to discuss the mobility of fission products in the context of 4 whether you were preventing outbursts or prevention of the heat l i 5 removal process? Because the whole thing goes back to that. l t 6 And so when we talk abcut seismic and component pooling and i l 7 this and that and so forth, do the interviewers remember that [ ] 8 is all they have to talk about in the final context? I J j 9 MR. MATTSON: We did not use those exact words, but } ~ J 10 when we talked about the safety assurance question, we talked I 11 about the risk element to the safety assurance question. f I 12 Ultimately you are keeping fission products away from people [ 13 and that goes into safety assurance. i ) 14 MR. EBERSOLE: And even though you are talking about ) 15 some distant thing like component cooling, reliability or I 16 whatever, that still predominates. And they all know that. f i 17 They get so lost in their ewn sciences, f 18 MR. S!ESS: Someone has to divide it up to get it l 1 r I 19 down to people. i l 20 M R. BLOND: That in why we wanted to establish a [ i 21 elear distinction in categories where things are vital and have j i 22 a true safety significance t 23 MR. EDERSOLE: For instance, it doesn' t mat ter if the ( 24 building f alls down if we don' t harm the clay. 25 MR. SIESS: It takes research to prove that 1 j i Heritage Reporting Corporation [ (202) 628-4888 i
e i 4 f i () 81 i 1 MR. EBER50LE: In terms of the problems we 2 encountered, scheduling the interviews was a prob 1'sm and we e l 3 tried to be as flexible as possible. The interview room was, l 4 to be i n this dark room was a significant problem. It was l i j 5 amazing. We wwre i n a room with no carpeting and bare walls 6 and the people were intimidated. There was a real factor that i i 7 we had no control over. And we recognized it as we were [ l t j 8 conducting the interview, but we couldn' t really do much about i i j 9 it. l j 10 The sound of the camera, we were just using my home 11 video camera and we eventually got a special microphone that 12 could be used for that so that we could pick up the sound a ) 13 little bit better. 1 i j 14 As we conducted the interviews, we soon lost all j i ) 15 relationship with the camera. We had no idea what was on there t i t } 16 any more, after about a minute or so. And so everyone fell i i 17 into normal discussion and normal language. And 1 think that I f 18 as you looked, as we went through and reviewed the tapes, when [ j 1 i 19 you see this back on the television set and you are in j t 1 20 colloquial discussion, and to track someone from the interview, { 21 and that is one thing which we want to establish as a kind of a i 22 ground rule. These are going to be viewed, and watch what you 4 i j 23 are saying and the way that you say i t. i i i 24 Overall we felt that there were very minor prob 1SMs f 25 and that they were very easily corrected. And it was not a big l l l Heritage Reporting Corporation (202) 628-488S 1 6 --,-----,-r-~%wr-.-
,wyv--,. -,-yy,y,
--,,m ,--p 3-,,y,+- ._-p.,.,--ry,-,wmw,,--,,,-,,,wr-ye-y--w-- yvw.,-g --+ ,,ye-o,c.,--p 9 .-,-er,- -+---+--,-e*Nm-va+----m--- g
t r l l 1 problem associated with the trial at all. [ t 2 In terms of conclusions, we found the trial easy to I i 3 perform. We didn' t think it was onerous. We didn' t think it I 4 was really that difficult. We feel that the exercise is f 5 repeatable, that we would end up with three different panelists i 6 again just focusing on the mult i-disciplinary areas, might come ) 7 up with very similar types of conclusions. And we hope that is 1 L ] O true. We really haven' t had the opportunity to test that out, t 1 I J 9 though. We see that the process is retelevable. There is good I l 10 information, a good way to present it, and we can go back and i I 11 verify what we have. So there is a quality assurance check on l 12 the process. The multi-disciplinary panel is very important. f () 13 We saw as we went through it it is very easy to 14 communicate on the criteria. When you talk about safety I ] 15 assurance or enhancinD safety, that means something to 16 everyone. And I think precisely in the ways it has been 17 brought up. l l 18 As we talked about usefulness, the connotation was i 19 there and it became very easy to discuss. We found as you will l r 20 see that we think the process discriminated very well against ( l 21 the criteria, so the programs become more Meaningful. And we j 22 felt that the tapes were a very useful part of the process. It 3 23 was something that we could go back t o. It did increase the I j 24 intensity and added focus to the interview. I t, wasn' t just ( [ i' 25 another interview doing prioritiration one more time for these r 1 ( I j Heritage Reporting Corporation i j (202) 628-4888 l i
1 0 1 guys, it was something that was a little bit different and 2 special. And that added a sense of significance tc the i 1 3 program, to make them think a little bit more. i 1 4 That concludes ty part of the discussion. Again, ! ( i ] 5 think we will get into some interesting pieces when we look at l l [ 6 the results, and that is what Roger Mattson will get into. 7 MR. S!ESS: Okay. Roger. Roger Number !!. If it j 8 will help, tell us what portions of the report we might be i 9 looking at. This is the last appendix, Roger? 10 MR. MATTSON: The last time ! did one of these Roger i i 11 Roger routines it was Roger Boyd, and he was always Roger the l i l 12 Elder and I was Roger the Younger. I think there's been a 4 1 13 change in that. ) 14 I am goinD to present some summary conclusion type j 15 things on this fitat line and then quickly move beyond that to j j 16 some more specific information, because it is clear from the i 17 discussion that you want to see the details, and you have some a i 18 example details, and it will give us a chance to see how it 1 19 really works. 20 Dy way of conclusion, what we mean in saying that it ) J 21 is applicable, this process that we have stilized, is that it 22 is a dectaton analysis so it provides a structure, but we have i 23 used a form of decision analysis that is not too complex. This ] 24 is a complex program, a comples business already. If you put i 25 in a comples decision analysis technique, you soon lose I Heritage Reporting Corporation (292) 628-4888 i
.r' [~D d4 \\_) 1 yourself and you don' t know whether you a*e dealing with an 2 abstraction or reality. 3 It is applicable because it gives us a way to bring 4 diverse points of view to the table and then try to reach 5 consensus. So it encourages diversity and then deals with 6 diversity. 7 It is also applicable bor'ause it deals wa ';h a program 8 that is not just safety.
- Jesse, 1*
- hear you talk about fission 9
products, but it also has to deal with the environ $ent and 10 safeguards and waste maneDement and things besides reactors 11 MR. EBERSOLE: I have another questioniwhich I 12 deferred until you got into this, but let me do it now. (~h (_) 13 Yesterday we were talking about safety in the context of 14 preserving a resource, a national resource, that is, it is the 15 nuclear option. But is this part of the safety picture, rather 16 than Just persor.41 radiat ion sa fety? I think we said it was 17 and that should be a factor in the consideration of all these 18 activities. 19 MR. MATTSON: It has not been in what we heve dere so 20 far. That is a comment we are considering. We' ll think about 21 that. It is scrutable. It is simple, repeatanle. The results 22 ae documented and recorded. We haven' t used the word 23 transparent, but I think that is a word close to what we are 24 trying to achieve. You can come in as an outsider, whether you 25 have a degree in one of the specialt oss or not, and unde >* stand O U Heritage Reporting Corporation (202) 628-4888 1
() 85 1 this process. It is not -- you don' t have to know f ault trees 2 and event trees, you don' t have to know some conflicts decision 3 period. You are going to see in a minute it is a little bit 4 like picking a campground. 5 MR. SIESS: How do you know it is repeatable? 6 MR. MATTSON: pardon me? { 7 MR. SIESS: How<do you know it is repeatable? 8 MR. MATTSON: As Roger Just said, we don' t yet. This j i 9 is a trial and we haven' t repeated. We have aimed at h i 10 repeatability by using a consistent set of questions, by having 11 discipline in using those questions and by having a consistent 12 set of panel members. That may not be enough. 13 MR. SIESS: Repeatable ho me mean that another set of 14 panel members would come up with the same answer. 15 MR. MATTSON: Or other managers of that program would 16 do as well in describing that program for people that i 17 understand whether it is well designed to answer the questions 18' that the agency has in front of it. That is a variable also 19 MR. EBERSOLE: How did you overcome the bias of a l [ 20 program manager in the direction of his own area of work? l 21 MR. MATTSON: We talked about it, saying what did you l 22 observe by way of bias and doesn' t that person have their head k 23 in the and seems like that really isn' t relevant, and we had j ) 24 discussions about that. And some knowledge of the people. The 25 panel members do benefit by knowledge of the program. We chose ) I 1 Heritage Report ing Corporation (202) 628-4888 4 r 4 ---._.x-- ..-_,.,,,m,._,-....,-.,-
9 (~T 86 U 1 ourselves because of our diversity of experiences relative to 2 that program. But we had knowledge of it. I don' t think you 3 could go to three NAS members that had not been in nuclear 4 technology or dealt with NRC and put them on the panel and get 5 the same results as you would from this panel. But our goal is 6 to turn this over to NRC so that we don' t do it
- f. ext time, they 7
do it next time. It's their senior managers that do it next 8 time. We are developinD a methodology, trying it out, 9 educating them and then turning it over to them. So by 10 definition in the future the senior managers would have that 11 knowledge 12 MR. SIESS: How much did your rankings depend on the 13 interviews 14 MR. MATTSON: Oh, a lot. 15 MR. SIESS: It did? 16 MR. MATTSON: Yes. We sometimes overturned what the 17 interviewee said, but we couldn' t, we would not have had good 18 ranking without interviews. The other alternative would have 19 been to read mountains of paper and we did read a lot of paper 20 in preparing ourselves for the interviews, but we didn' t read 21 mountains and mountains of research results 1 22 M R. EDERSOLE: What did you do with the perennial 23 problem that you come up to the point where it is almost 24 inevitable, that much more remains to be done? 25 MR. MATTSON: Not too often. We came up with people l Heritage Reporting Corporation (202) 628-4888
l i 87 l 1 who said this area is essentially done, here is an area that is 2 Just starting, and we asked questions like yes, well, you uaid 3 it was essentially done two years ago too and it isn' t. What 4 is going on? But there were people who said they were nearing 5 completion. We are going to talk about one in a moment where 6 the seismic network in the United States, being phased out of 7 NRC, going to the USGS, and the man didn' t fight that that was 8 the wrong thing to do. Said that's the right thing to do. 9 Okay, usefulness and effectiveness. We think it is 10 useful and effective because it is structured and formal. The li process is a complex array of information and it produces 12 results that are internally logical and overall consistent with 13 your own common sense. If you Det done, let me show you a 14 sheet that has some of these things ranked. And if it comes 15 out that environmental research is the most important thing 16 we' re doing to keep fission products away from people, 17 obviously there is something wrong with the structure of the 18 thing. It isn' t producing what feels right when you are done. 19 And it must feel right. Whatever it is, when it is done it has 20 to still feel right. So to us this one still feels right so it 21 is effective. 22 MR. MICHELSON: I am trying to understand now the 23 detailed process with the overall process. We started out this 24 morning with talking about the NRC's otrategic plan and then we 25 converted that strategic plan into a number of key research Heritage Reporting Corporation (202) 628-4888
T 88 1 questions and then we got down into converting them into views 2 of a number of individuals. Now, when you say this process is 3 repeatable and that sort of thing, do you mean the process of 4 interviewing the individuals or do you mean the process of each 5 year I might start out with a strategic plan and go through all 6 these steps? Which part is the repeatable part or is it all 7 repeatable? 8 MR. MATTSON: When we use the word repeatable we mean 9 that you could use different people and come up with 10 essentially the same conclusions. 11 MR. MICHELSON: Starting with the strategic plan or 12 starting with the questions given and your trying to get the () 13 views of individuals? 14 MR. MATTSON: Given the strategic plan, and given 15 what we have derived from it, then using the safeguards of the 16 process we developed for interviewing and reaching Judgments as. 17 a panel and documenting and prioritizing, that process we think 18 provides for repeatability. 19 MR. MICHELSON: But you are not claiming you can go 20 back each year to the strategic plan using your process and 21 work through it? 22 MR. MATTSON: No, I think if you started over every 23 year in deriving the critical questions out of the strategic 24 plan, you would do it differsntly every time. 25 MR. MICHELSON: That is a highly judgmental area. Heritage Reporting Corporation (202) 628-4888
1 MR. MATTSON: And that is why it is important for 2 you and others to read tha questions that Roger ar.d Larry and I' 3 derived from that strategic plan and added from our own 4 experience to that list and then have put in an appendix here. 5 MR. MICHELSON: That takes care of my question. 6 MR. SIESS: When are we going to do that? 7 MR. MATTSON: We hope that the research people and 8 the NRR people and the NMSS people are going to do that as we 9 go along. They are going to say they, there is a very important i 10 question to my licensing activity that I am looking for an 11 answer and it clearly is in the strategic plan and you guys 12 don' t even mention it. ( 13 MR. SIESS: You said you -- we can' t do it today. 14 MR. ROSZTOCZY: That is certainly not for the 15 subcommittee and we would certainly appreciate any comments 16 along those lines, if you feel that you need a separate 17 interview to discuss those questions individually to see the 18 priorities, we would be happy to provide it. 19 MR. SIESS: That is not the priorities. I think Roger 20 was talking and I was addressing the list of questions that 21 came out of the strategic plan. Appendix A. 22 MR. ROSZTOCZY: The list of questicas thet came out 23 of the strategic plan, and then after they put out the 24 questions they arranged those questions into three groups. l 25 MR. SIESS: Appendix A. Heritage Reporting Corporation (202) 628-4888
4 90 1 MR. POSZTOCZY: Yes, that is correct. So if you want 2 to discuss Appendix A in more detail, we would be happy to 3 provide the presentation for you. 4 MR. SIESS: I think we could spend a day on those, if 5 I had this many people at the subcommittee meeting. I' m not 6 quite sure whether it's worth a day but I'm sure we could spend 7 a day. 8 MR. MATTSON: What you are really doing when you 9 spend that day is you are prioritizing the strategic plan. 10 MR. SIESS: A lot of it is interpreting the 11 questions. 12 MR. WARD: Chet, it seems to me that might be 13 worthwhile. It is unclear whether we need anything further 14 from the staff on it, though. l 15 MR. SIESS: I' m trying to look at them and see i 16 whether we could do it without having them explain what the 17 questions they have are. That's all right. Let's go ahead 18 with this, because we' re not going to get these guys back 19
- again, i
i 20 MR. MATTSON: The bottom line of this slide is that l i 21 we have something that is scrutable, useful and offective for 22 three purpose -- planning, resource allocation and 23 communication. You' ve heard that before. Just remember we mean 24 those three functions to be both inside the agency, within 25 research, between resaarch and NRR, between the staff and the O Heritage Reporting Corporation l (202) 628-4888 l
( 1 committee, between the staff and the Commission. We have tried 2 to think of all those places that this thing has to communicate 3 and be useful and in our Judgment it works that way. 4 MR. WYLIE: Can I ask a question in this area? Where 5 in this program plan does it provide for the input from NRR7 6 MR. MATTSON: Essentially the next step. 7 MR. WYLIE: I know, but it is not spelled out here 8 that I can find. 9 MR. ROSZTOCZY: There are a number of areas where 10 there is input to that. The first one was when they were 11 deriving the questions from the strategic plan they went around 12 to talk to the different offices including NRR. 13 MR. WYLIE: Thas has been done? 14 MR. ROSZTOCZY: This has already been done once. The 15 second part will be that the information that we have now, the 16 type of information was in the report, and we had sent this out 17 for comment, asking other offices at NRR to comment on those 18 questions. i 19 MR. WYLIE: Has that been done? 20 MR. ROSZTOCZY: No, that has not yet been done. That 21 is next. And then the third part is after all the research 22 areas have been prioritized, then we will go back to NRR and 23 discuss with the appropriate people those areas which relate to 24 them and ask their opinion on the summary sheets whether those 25 are factual and whether they are properly expressing the 4 Heritage Reporting Corporation (202) 628-4888
O 92 4 1 expected views and the expected usefulness of the -- 2 MR. WYLIE: Will that process be identified in the 3 plan that comes out? 4 MR. ROSZTOCZY: Yes, it should be. 5 MR. SIESS: Is there going to be some kind of 6 document that describes the whole process? 7 MR. MATTSON: Yes. 8 MR. SIESS: Over and beyond what was run in the 9 trial, all the things that Zoltan has talked about? 10 MR. MATTSON: There has to be. 11 MR. WARD: Zoltan, how does the research committee 12 interact in this or what do you plan? ) 13 MR. ROSZTOCZY: I do not know yet exactly how Dr. 14 Bechman is planning to do this. I am sure that this 15 documentary has been made available to the committee, they will 16 be looking at it and they are going to comment on it. The part 17 I do not know is whether he wants to include that advisory 18 committee before he issues the prioritization or whether it 19 will be given to them when it is issued and then we will hear 20 the comments. Being a living document, it does no* make a 21 tremendous difference. When it is issued there will be public 22 opinion on it, there will be the opinion of various groups and 23 in the next update certainly we will take those into account. 24 MR. MATTSON: The next slide, there are three kinds 25 of charts that I am going to show you now about some typical Heritage Reporting Corporation (202) 628-4888 r i
Cl) S3 1 results. The there charts are first a summary of an activity. 2 Remember, we looked at eight activities out of 72 activities, 3 so I am going to show you one of eight. And then I am going to 4 show you activities versus safety questions. Eight activities 5 ranked against the safety questions in Appendix A. 6 MR. SIESS: And we don' t have these? 7 MR. MATTSON: Not yet. I am about to hand them to 8 you. Then I am gong to talk to you about activities ranked 9 against criteria. First, if you would help me with a summary 10 of the activity. 11 Now this is a one page summary that has condensed 12 into a small space all of the information that was developed by () 13 the panel in deciding upon the ranking of this area and that 14 information is already provided in this Task 2 Report, so this 15 one page summarizes what is printed at pages 14 to 44 of the 16 Task 2 Report. Just consolidates that into one page and puts 17 it, instead of a bulletized form it puts it in a narrative r 18 form. Now, this is a very important sheet of paper. This is a 19 piece of paper where the panel communicates two directions. It 20 communicates upwards in the organizatiot, what it's heard and 21 what its Judgments are about a particular activity, one 22 activity out of 72. It is also the place where the panel i 23 communicates laterally te NRR. to the ACRS, to others, users, 24 if you will, about what the panel thinks and what it has 25 learned about a particular activity. And it is also the way i Heritage Reporting Corporation ] (202) 628-4888 l J
94 1 the panel communicates downwards to the people from whom it 2 heard this information. The section leader, the branch chief, 3 who have described their program. And it tells them what was 4 heard and what was done with the information. It doesn' t try 5 to write a thesis, doesn' t try to write reams. It tries to 6 write it in simple, understandable language that people within 7 the agency and outside the agency can cope with. 8 There is one other feature I want to point out on it 9 because it will bear on the next piece of paper I am going to 10 show you. Look in the middle of the paragraph entitled Safety 11 Assurance. In that paragraph there is a sentence that reads: l 12 "The current program is designed to supplement the existing i (~h \\d 13 data to answer additional questions concerning the i 1 14 seismological risk to nuclear facilities." l 15 Well, is that a question that is within the strategic 16 plan? The answer is yes. If you look at Page 21 of fask 2 l 17 Report, you will find that that is Question R-1. That is a l 18 question we derived form the strategic plan and then we ranked i 19 or categorized as relevant to safety, not vital, not important, 20 but relevant, and that is the question which this research 21 program was designed to answer. 22 You are going to see in a minute that it also 23 addresses some other questions from the strategic plan. We l 24 think that is the one that it was primarily aimed e,t. l l 25 Okay. You ouDht to be able to go in here now and in a Heritage Reporting Corporation (202) 628-4888
( 1 narrative way find what we are going to say in a specific 2 criteria type way in a moment about safety assurance, 3 usefulness, appropriateness and resources, the four attributes 4 that w2 are characterizing each of these activities accordinD 5 to. Every activity is waiting for its safety assurance, its 6 usefulness, its appropriateness, and its resources. 7 MR. SIESS: This tells me that this is neither vital 8 nor important. It is questionable as to whether NRC should do 9 the funding and there $20 million left to spend. 10 MR. MATTSON: That is what it tells you. And some of 11 the reasons why. 12 MR. MICHELSON: And yet if I go to your B-4 question ) 13 it say what techniques should be developed to improve seismic 14 capability. Now that is a vital question on an irrelevant 15 problem almost. That is the thing that blows my mind. Your B-4 16 question. It's on page 15. There is a vital question. What 17 technique shall be used to assess and if necessary improve the 18 seismic capability of safety related equipment. That is a 19 vital question. 20 M R. SIESS: Yes, but that has nothing to do with 21 this. 22 MR. MATTSON: There is another research activity 'n 23 the seismic area that addresses that question. But it is not 24 this activity. ] l 25 (Continued, next page) i Heritage Reporting Corporation (202) 628-4888 --- A
I 1 MR. MICHELSON: Why is it vital though to know those 2 techniques when we' ve already decided that the' problem of i 3 seismic capability is almost irrelevant. j l 4 DR. SIESS: That's not what this says. 5 MR. MICHELSON: Isn' t that what R-1 says. I 6 DR. SIESS: This has to do with seismic detection 1 t 7 networks. f 1 i i 8 MR. MICHELSON: Well, apparent ly we' re not, is that t 9 what that has to do with it. 10 DR. SIESS: Just earth science hazard. l 11 MR. MICHELSON: I didn' t get that from the question + s i 12 DR. SIESS: It's earth sciences. I ) 13 DR. SHEWMON: R-1 has to do with seismic hazard. 14 DR. SIESS: And that's what this has to do with. f 15 DR. SHEWMON: Then you' re using hazard in a rather 16 special irregular way. You' re not talking about what's the 17 health and safety of the public risk, you' re saying geograms l 18 probability of earthquake. l l 19 MR. MICHELSON: That's what I didn' t understand. ( 20 MR. MATTSON: 'In terms of characterizing the hazard i ]- 21 this said that relevant to safety, and we know about as much i 22 about it as we' re going to knowl that's essentially what this I 23 page says. ~ i 4 24 DR. SIESS: This is geology geophysics, it's not even 1 25 size model. j Heritage Reporting Corporation (202) 628-4888 j l
) 1 MR. MIOMELSON: Only the seismic experts understand 2 iti I didn' t. 3 DR. SHEWMON: Hazard usually has a meaning outside of 4 geology or seismology. Now, Chet knows so well he doesn' t need 5 to worry about the rest of the way. 6 DR. SIESS: We' ve always used that, in full committee 7 meanings. 8 D R. SHEWMON: Hazard still has a meaning to other 9 people. 10 DR. SIESS: Oh, yes. 11 DR. MARK: Have you not identified a need to change 12 some of the words here. () 13 DR. SIESS: That's embedded in all of research 14 progrem. It's a little hard to get rid of the seismic hazard 15 term, it's in there. 16 DR. MARK: True, but it's an in-house term. 17 MR. MATTSON: You' ra now involved in the prioritizing 18 process. Welcome aboard. You are now using the language and 19 you' re deciding that you like the language, and if you like 20 this program and whether it addresses a vital issue or relevant 21 issue and what its value ist you Just entered it. 22 DR. SIESS: I think it's excellent. 23 MR. MATTSON: The next type of information is to 24 summarize the activities versus the safety questions.
- And, 25 S a i... if you would help me here, here's another one to hand Heritage Reporting Corporation (202) 628-4888
(:) S8 1 around. 2 This gets a little more complicated. This is the one a 3 that I mentioned we would.take the aight areas that we looked 1 + d 4 at. Remember we were going to look at 72. And we would cross 5 those with the research questions, appropriately corrected the j 6 questions of the agency for which research might be 7 appropriate. i ) 8 So, running across the time you see earth sciences 9 through code, models, validation and analysis. And running l 10 down the first column you see the 16 vital questions, the 16 l 11 important questions and the seven relevant questions. l 4 12 Now, let's take the example that I handed out a j j () 13 minute ago which is earth sciences which is the first column or i 14 second column. And you' ll see down in R-1 that the work is l 15 important for resolution of that issue. But you look up the 16 column and you' ll see that it's also relevant to some other 4 i 17 questions that come from the strategic plant not vital but 5 i 18 relevant. l ) t 19 DR. SIESS: It's important to resolution of a 20 non-important question. i n 21 MR. MATTSON: That's right. l 22 Now, you can -- I guess I should point you this way. 23 This kind of presentation is flexible and somewhat arbitrary. I 24 UR. SIESS: The higher they are -- 25 M R. MATTSON: The question of how to present this l (:) i \\ f 1 j Heritage Report ing Corporation i i (202) 628-4888 i
99 is diffic.lt for us. 1 information -- just a minute, Chet, 2 There's a page in the report, if you refer to it you can see 3 another way to present some of this information. 4 D R. SIESS: 12. 5 MR. MATTSON: 12? Yes. I' m going to show you 6 another sheet in a minute that's still a third way. 7 DR. MOELLER: Well, on page 12 you must have used 8 colors or something in the final report because looking at page 9 12 didn' t do anything. 10 MR. MATTSON: This is hard to see, and that's why we 11 moved away from page 12 and moved to this sheet and then a 12 subsequent one that I will show you. Remember I said it's a 13 little bit like picking a campground. Remember the sheets 14 where you' re going to travel to Montana for the first time and 15 you never heard of any of these places but you want running 16 water, you want to fish, and you want hook-ups for your trailer 17 house or whatever, and they got the little shaded areast that's 18 what it is. It's not very sophisticated, but it is decision 19 analysis. And it's a way tc bring a lot of live ideas to your 20 mind quickly. 21 You can tell by how many hits and what shading they 22 are, whether you' re in an important area or a weak area. And 23 then once you find out that you' re in a weak area and it's your 24 area and you want to defend it, you go try to defend it. And 25 if it's an area that you always thought should be killed, you Heritage Reporting Corporation (202) 628-4888 t
O 100 1
- say, "Hey, look how weak it i s, it's all a few hits and they' re 2
all open." It drives you quickly to the priority. 3 DR. SHEWMON: One of the things that I' ve been 4 waiting on for an hour or two, I guess, is to hear about human 5 factors. And if I was going to find it anyplace it would 6 probably be under operating reactors accident prevention, and I 7 don' t find it. Is that because the NRC's long range plan 8 doesn' t worry about operators making mistakes explicitly or 9 what? 10 MR. MATTSON: Somebody help me. If I look at the 11 questions, I think you' re going to find it. 12 MR. ROSZTOCZY: I-5. ( 13 MR. MATTSON: I-5, okay. 14 DR. SIESS: I-5 on page 18, human factors. 15 MR. MATTSON: To understand the questions, this is 16 really a shorthand notation in the lefthand colurnn. 17 DR. SIESS: Well, you' re looking at this sheet. If 18 you look back at page 18 it's spelled out. 19 MR. MATTSON: Now, paul, you said it was very 20 important to you, would you argue that it should be vital? 21 DR. SHEWMON: Well, I would argue that it should be 22 considered, and it doesn' t sueri to make the cut. Now it 23 doesn' t make your cut because it wasn' t one of the 72 items 24 that you looked atl is that right? 25 MR. MATTSON: So far, yes. The human factors prograrn Heritage Reporting Corporation (202) 628-4888
l 101 I () 1 has not yet been reviewed, that's all that means. It doesn' t i 2 mean whether it's important or not. 3 DR. SIESS: paul, there needs to be this much more j i 4 out here. It doesn' t enter on any of these lines. l 5 MR. WARD: See, they just had eight here and there l 6 are really 72 of these things. [ 7 MR. MICHELSON: It's a really big matrix. 8 MR. WARD: I think it's a good way to display it, 9 though. 1 10 MR. MICHELSON: Better than the other way. 11 MR. MATTSON: Did you say you thought it was a good 12 way to display it? ( ) 13 MR. WARD: I think it is, yes. l 14 DR. SIESS: A good way. 15 MR. MATTSON: A good way. I 16 Again, this can be changed. You know, Just trying to 17 find ways to indicate it. 18 Let me give you a third sheet. And this is a summary [ f 19 of activities versus criteria. l 20 DR. SIESS: The type gets smaller and smaller. i 21 MR. MICHELSON: Before we go to that, Roger, would 22 you clear up one more problem I have. On your page 12, of i 23 course, that method of demonstrating allowed somewhat useful l 24 items to even end up in the vital category, and you lose that { 25 flavor in this other chart. i Heritage Reporting Corporation i (292) 628-4888 i i
O 102 1 MR. MATTSON: No, hang on a minute now. Whether it's 2 vital or not is determin2d by whether it addresses a vital NRC 3 question. 4 MR. MICHELSON: Right. 5 MR. MATTSON: The program may be completely useless 6 addressing a vital program and it would show up as a vital 7 program. You would decide to kill it because it was a useless 8 way to go at that vital program. 9 MR. MICHELSON: It was Just used in categorizing the 10 basic questions then. 11 MR. MATTSON: Yes. 12 MR. MICHELSON: And so the paga 12 demonstration has ( 13 nothing to do with what you handed out. Okay. 14 MR. MATTSON: page 12 has more to do with the page 15 you have in front of you. I brought it up at that point to 16 show you that there are several ways to display this. l 17 MR. MICHELSON: Then I thought you meant that was 18 another way of communicating the same information. l 1 19 MR. MATTSON: I' m sorry, I didn' t mean it. The same 20 information I believe was contained in the page you now have in 21 front of you, except you' ll not ice on page 12 the resource 22 information was handled by one of these little windowsl i 23 whereas, on the page you hava Just been handed out we Jecided 24 high, medium and low was not a very good way to communicate i 1 25 information about resources, numbers were. So we went back to i Heritage Reporting Corporation (202) 628-4888 1
103 1 the numbers in the resource category. 2 DR. SIESS: One quick question. Resources involve 3 two termsl cost and required skills. There's a perfect 4 correlation between those two. 5 MR. MATTSON: Resource is cost and skill. 6 DR. SIESS: Yes. You couldn' t get skills at a low 7 cost. a 8 MR. MATTSON: We had in our -- and you' ll see it back l 9 in the appendix -- in our ranking we had included whether you 10 needed specialized skills to deal with a particular area. As 11 one of the things we considered under the category of 12 resources. ) 13 The thought there is that -- let me give you an 14 example. If there's an area where you assembled a body of 15 people in the laboratories to research that specialty, and 16 you' re losing your enthusiasm that that's an area all that i 17 important for safety, and you decide to kill it, then you ought 18 to have in your mind the fact that if you ever want to go back, 19 it's going to be very costly to go back, because those are 20 specialties that take a long time to train and assemble. If 21 you' re within striking distance of some useful information, j 22 Maybe you would continue to spend in that area until you got to a 23 that new point and then kill it. That's a resource 24 implication. 25 We' ve had some dif ficulty knowing how to use that in Heritage Heporting Corporation (202) 628-4888 l
O 104 s_/ 1 the summary information. 2 DR. SIESS: page three you defined high cost as $1 3 uillion and a requirement for hard to obtain skills. 4 MR. MATTSON: Yes. S DR. SIESS: And moderate cost is 300 to a million 6 with readily available skills. And the low one is less than 7 300 presumably with no skills. Now, that says to me, if I 8 can' t do a million dollar project with moderate skills or a 9 $300,000 project with high skills. 10 M R. MATTSON: Roger is going to try it. 11 MR. BLOND: The numbers that we put on our initial 12 evaluation were associated with the contracts, with the FINS. () 13 When we got to the activity we decided that was inappropriate. 14 So the presentation now as you see it on the figure that Roger 15 just handed out has the cost to date, the cost that has been 16 spent to date, and the cost that is expected to be spent to 17 complete the program and the resources that are being used. 1 18 DR. SIESS: And you just dropped the scales out of it I 19 entirely. 20 MR. MATTSON: Yes. On the summary sheet. 21 DR. SIESS: Go ahead. I don' t want to delay. 22 MR. MATTSON: Let me just try to close. You would ] 23 have found that useful information? l 24 DR. SIESS: I don' t know. I just didn' t understand i 25 your definition. J ] Heritage Reporting Corporation (202) 628-4888
105 1 MR. WARD: I don' t see how it correlates as you said. 2 I mean, it might be kind of a flag or something there. 3 DR. SIESS: Well, if you flag, high cost is over a 4 million dollars and requiring specialized skills. 5 MR. WARD: I don' t think that makes any sense. t 6 M R. MATTSON: That's why you can' t deal with them 7 both simultaneously. We realized that. In our handwritten or 8 more detailed information it's got both considerations. And 9 then when we got around to presenting it for decision-Makers 10 and advisors like you we came up with this sheet and by then t 11 it's lost. And now somebody is saying, maybe that's important 1 i 12 information it ought to have been includec. 13 MR. WARD: So you' re aware, the sort of problem you I 14 were talking about earlier, if you discontinued a program you 15 might lose this specialized set of skills. 16 MR. MATTSON: Yes. 17 MR. WARD: You might illustrate that by Just a flag i l 18 or something. s 19 MR. MATTSON: An asterisk. ) 20 MR. RARD: Yes. i 1 21 MR. EBERSOLE: Chet. ) l 22 DR. S!ESS: Yes, Jesse. l 23 MR. EDE RS0!.E s I am bothered by an antique question l 1 I 24 that goes back 20 years. Question B-1 says, "How should the 25 NRC measure whether a utility maintains," and that's Just (2) l Heritage Reporting Corporation (202) 628-4888
1 maintains, they didn' t say designs, "it's nuclear powerplants 2 in an adequate state of operational readiness." 3 Then on B-14 you say, it's related to that, "How 4 should NRC measure whether the quality of construction 5 operations are adequate to assure compliance with regulatory 6 requirements," and didn' t say anything about design. J 7 Now, I think we all understand that simple detailed 8 compliance with regulatory requirements with all the 9 ambiguities of interpretation doesn' t necessarily mean you' ve 10 got to save a plant. Because I know for years, you' ve got to 11 do a little more here and there then the regulations can ever 12 cover. 13 And I understand right now at TVA this is a vicious 14 question. Right now there's a flack going on. And I recall 15 the uanager saying, ne' re going to build the most perfect and 16 safest plant in the world. And I would ask him, "How do you do .\\ 17 that?" "We' re going to comply with every regulation, but not a t l 18 cent more." 19 DR. SIESS: Got an answer? l r 20 MR. ROSZTOCZY: That's a very good comment. And I 21 think that's a sure thing of the strategy plan at it stands. I 22 This is not emphasized to the extent as it should be. I think F j 23 they were pulling off what was there, it's not there. i 24 MR. MATTSON: It's an oversight by us, too, we knew c5 better. Design should be in there. j CE) Heritage ReportinD Corporation j (202) 628-4888 l l
...~ - = () 107 1 DR. SIESS: I think there will be some other ? 1 2 questions like that if we can find the time and place to 3 address these questions. i 4 MR. MATTSON: They' re telling me over there I' m 1 i 5 5 taking more than my fair share. I just tell you that you can t I e I 6 pick your campground by this sheet very quickly,'because if t 7 you' re in a program that's got a number of dark bullets you' re i 'r 8 going to get funded and if you' ve got a program that's got a 4 I j 9 number of open bullets you' re in trouble. And it quickly l 4 I i 10 focuses decision-makers and prioritizers, the people who spend 1 i 11 dollars on what's important. i l { 12 And if you decide to go ahead with funding of an area s 13 that doesn' t get a dark circle, then you' ve got some reason for 2 ? l 14 it and you' ve discussed it and people have agreed to it. And L i i 15 that's priorit ization. 9 1 l 16 MR. MICHELSON: It doesn't really tell you about the 17 FINS within that program. It doesn' t have to sort each and t i 18 everyone of them out. i j 19 MR. MATTSON: I promised you ! would give you some i t 20 information on that, I forgot. You ought to be able to go into j i 1 21 the summary of activity, although they' re not by FIN number. l l j 22 If there is something important in a bad area we would I 23 highlight it. And if there's something bad in an important i 24 area we would highlight. And ws intend to do that with some of l l 25 these. We have done it. I (:) i i 1 l Heritage Report ing Corporation ) (292) 628-4844 i 4
t i a () 108 l 1 MR. MICHELSON: I assume in a task this size there is i i 2 another section called fire protectioni is that correct? l 2 3 MR. MATTSON: No,.there's no fire protection research 4 program. 5 MR. MICHELSON: Well, the title says that there is or i 6 something. t 7 DR. SIESS: That's beside the point. The title is l 8 research -- h 9 MR. MICHELSON: I know. It's about time they { d t n 10 straighten out their act a little bit. f j 11 MR. pITTMAN That's where the fire use to be. 1 12 MR. MICHELSON: I know how it got there, I'm Just I I l () f 13 saying it's time to straighten out the act. 1 i 4 14 DR. SIESS: This is a little confusing, this is a ] 1 t 15 valid point though. Is there anything in the strategic plan 4 i 16 that calls out fire protection? Is it in your list of { l J 17 questions somewhere? l f 18 MR. ROSZTOCZY: I don' t believe so, and I think 1, 1 i j 19 they'll have to check on that. 1 I 20 DR. S!ESS: Because, you know, what I have difficulty l j 21 finding from this is the research that's needed that is not } j 22 being done. 23 MR. ROSZTOCZY: That's correct. i 24 DR. SIESS: Neither one of these charts shows it 1 I 25 because the Commission hasn' t said it's needed. i i Heritage Reporting Corporation j (282) 624-4884 { i l
=. l a t . () f 189 ? 1 MR. ROSZTOCZY: We are not at that step yet. We are / i 2 trying to prioritize all the unwilling programs and then the L 3 next step will be to ask which other areas that we might need { 1 i 4 to do research. l 5 MR. MICHELSON: Where does the money come from to do i 6 enough homework to make such decisions? You' ve got to do a j 1 e i 7 little homework sometimes to decide if a new area is a problem j I i i l B or not. i 9 DR. SIESS: That's why they' re doing the scoping. I i 10 MR. MICHELSON: Does that come from the scoping money i I ) 11 then? 1 1 12 MR. ROSZTOCZY The research division directors in a 13 sense are being asked every year, when we go through on the 14 budget cycle, whether they have some other areas where they h 15 feel we should go on. So they do have various ideas about
- i 16 those.
They have recommended some of them. i 17 DR. SIESS: Zoltan, there's a scoping on fire, is 18 that a research project? l l 19 MR. ROSZTOCZY: Yes, that's correct. And that's i 1 20 close to completion. l l i j 21 DR. StESS: That won' t show up here because there's i I } 22 not a question on it. l 1 i ) 23 MR. ROSZTOCZY It will show up. It will show up 4 j 24 there. The only thing is, I'm not sure if it shows up in an 1 l i 25 individual item or report together with something else. (:) I Heritage Reporting Corporation (202) 628-4848
i 110 1 DR. SIESS: For it to show up here there's got to be 2 a question. J MR. ROSZTOCZY: No. At the entry, every research 4 program is an entry on the other scale, and it shows up. It 5 might not relate to any of those questions. l 6 DR. SIESS: I don' t see a line here that says -- 1 7 MR. ROSZTOCZY: No. You have to look at the ones on 8 the top of that sheet. There are eight of them there now. One 9 of those will be the fire research when we get to it. And then 10 when you look at the fire research and try to line it up in the l 11 questions, you will find that there is no questions. 12 DR. S!ESS: Dut since fire was in the category you () 13 picked for the first four columns, why isn' t it in here? 14 MR. PITTMAN: We didn' t look at all the activities. 15 DR. SIESS: Oh, you didn' t look at all the i 16 activities. These are selected activities. So if you looked I 1 17 at fire, there has been a column here with nothing in it l 18 because there was no questions. i 19 MR. ROSZTOCZY: That's right. That would indicate 20 that something is wrong with the strategy. [ 21 MR. MATTSON: In order for me -- i i 3 22 MR. ROSZTOCZY: You don' t need the program. 1 23 DR. S!ESS: Go ahead. I 24 MR. MATTSON: In order for me to finish I' ve got to 1 i j 25 go back a slide to D-5. One of the things that it says on 1 () J l Heritage Reporting Corporation f (202) 628-4888 3
111 1 there, the middle bullet, is that this process will identify 2 activities that might be underway for which there are no safety 3 questions. We haven' t found any yet. Of the eight activities 4 we found safety quest ions that: they applied to. There may be 5 some act ivit ies we' ll eventually find, but not yet. 6 It could identify, according to the third bullet, j 7 safety questions for which there are no activities.
- Well, 8
that's sort of, what additional things ought we to be doing in 9 research then that we' re not doing? And you recall, that's the 10 quest ion we asked people to firtet put them at ease during the ) 11 interview process. We got six answers out of eight interviews 12 or eight activities that we interviewed, we got six answers. 13 Let me just quickly lint the six. The first one is, 14 more attention to severe accident s in boiling water reactors 15 without arguing their merits. Th e real world of accident i 16 management, was item number two. Item number three was, l i 17 further verification of accident behavior within the design 18 basis. It came from a research g u)' now working in severe t 19 accidents that used to work in the design basis and he's 20 saying, "You know, we never did answer all the questions back 4 21 there," i 22 Fire protection both within and beyond the scope of 23 appendix Rt there was a suggestion that there was research 24 needed in that area. pre-stress concrete containment seismic i t 25 response. And liquef act ion potentia l Just being the SSE. j Heritage Reporting Corporation (202) 628-4888 I
I O 112 1 MR. EBERSOLE: What aboat such a matter as, how 2 shall we determine whether a given function shall be made ,sp s 3 automatic or manual? 4 MR. MATTSON: You havun' t been int ers iewed yet; 5 Jesse. Go write that down. 6 MW. MICHELSON: This.As within the scope of the eiQht 7 activities, also, you' re asking people. 8 MR. MATTSON: These came from the eight -- the people 9 in charge of the eight activities that we used in the trial. 10 DR. SIESS: They can add items outside of there. / 11 MR. MICHELSON: They coultf. ~ 12 MR. MATTSON: In fact. *overal people said, "Gee, I J g,) 13 don' t think that way, I only think.in my own area, I' m not 14 comfortable with that q.sestion," and we moved on. 15 One more item, it's not on D-5 and it's an oversi ght, 16 and that's something said at page 11 of the report, the eteer 17 insights. Somebody a3ked earlier, have wou got insights to the 18 people who are running the program. You do, but t?iat's not the i 19 purpose and so we' re not really doing much abottt 'that. But you I 20 do get some other insights. You.get insights about form, s. 21 You get insignts about whether policy is reachie'.n trite the i 22 staff. Whether the 4tretegic plae has ever been read by a + 23 branch chief, for example. You get insights about whether 24 people know who their users are generally or they don' t know 25 who their insers are generally. Just a riavor. i Heritage Reporting Larporation (202) 628 *BL6 {
O i j 1 We interviewed a man who said he felt like a whipped 2 dog. He had his program cut to such ribbons by people beyond 'o 3 his control. And that's an important piece of feedback. We e 4 don' t quite know what to do with that yet. We have liuted a 5 few of them in sort of generic terms on page 11 and we' 11 6 probably,)ust leave them on Mr. Beckor's doorstep for what l 7 they' re worth when we' re done. They don' t enter the i i t 8 prioritization process. 9 DR. SHEWMON: Roger, one last point then, amongst 10 your things on page 11 there is some question as to how much 11 independent analysis is required, capability is requiredl \\ 12 required for what? 11 MR. MATTSON: To f ul fill NRC's mission. In talking 14 to some people they think it's a lot, and talking to others 15 they think it's none, and there doesn' t seem to be any policy [ t { 16 on that in the strategic portion. i 17 DR. MOELLER: You mentioned very briefly that you ) 38 considered reading volumes of paper versus the interview i l i 19 approach and so forth. Did you have professional guidance in i 20 deciding all the possible ways that this problem could be i 4 - 21 addressed? In terms of your -- you decided on these oral TV ." 22 I taped interviews, but what was the range of possibilities and ~ 23 why is this the best? P4 MR. MATTSON: The advice to us was that you got good 25 summaries from the people who were in charge of the program, i Heritage Reporting Corporation l,' (202) 628-4888 4
(} 114 1 but you had to protect against vibes, so you use knowledgeable 2 people on the panel. You let them do some reading. 3 Larry said we didn' t tell the people to prepare but, 4 you know, the memo that went to them there was a sentence that 5 said, if you' ve got any information that you think you might 6 want to use to describe your program, especially in the area of 1 j 7 cross or risk significance, bring it with you. So that was 8 another valve that people who have done these kinds of 9 interviews and gathered this kind of information said we should j 10 use, safety valve, to make sure that we got the information we 11 needed to know that were factually dealing with the program, 12 and yet not be overly affected by the bias of the man who was l () 13 in charge of the program. 14 M R. pITTMAN: I would like very quickly to summartre 15 by describing what lies in front of us to complete this 3 i 1 16 program. i 17 Our future activities concerns both ourselves as 18 employees of the Nuclear Regulatory Commission and our 19 contractor. Our first step is to complete the review of the I 20 trial effort that we have described here this morning. What we i 21 have done. The insights we have gained. That we can take it l 22 out to the other divisions within researchl that we can take it { l 23 out within the other offices of the agencyl and that we can get l l 24 their feedback, their comments, their criticisms of what we l ) 25 have done, as much as we' re asking you for that same l Heritage Reporting Corporation t (202) 628-4888
l l () 115 j 1 information. 2 Then from that information that we have gathered, our 3 contractor can then take and revise the methodology or process, 4 4 if you will, in which then we can go out and take a look at the 5 balance. And we may have to go back and look at some of these 6 eight that we have already visited, but we can do the balance i ) 7 then of the program within research. 8 After we have done that, gathered all the data and 9 assessed it, then we' re going to take it back to the user 10 offices again, and to the REG division and ask them to review 11 the process that we have done. The results that we have 12 obtained. Their comments on iti are we off base, are we on 13 base. If we' re of f base, why are we' re of f base. 14 Then we will revise our data sheets, our ratings, our I l 15 rankings, both by ourselves and by our contractori and then it 16 will go to the director of research for his prioritizatien. J 17 I want to emphasize strongly that last bullet, that l r 18 NRC will be doing the prioritizing of the results. The l 19 eontractor is going to provide us with the data. They' re going l l j 20 to gather the data and they' re going to massage the data. But 21 ultimately in the end we are responsible for the results that 22 come out of the prioritization process. t 23 MR. WYLIE: Let me ask a question. The gathered 24 assessed data will follow research activities -- 25 DR. SIESS: This only covers eight areas. i Heritage Reporting Corporation (202) 628-4888
1 1 () 116 1 MR. pITTMAN: We have only covered eight in the 2 trial process. 3 DR. SIESS: Excuse me, your next two slides seem to 4 repeat the material that we' ve had before. 5 MR. pITTMAN: That's right. f 6 DR. SIESS: I wonder if anybody, including you, would [ i 7 object if we skipped these and use the last 15 minutes to try 8 and get some feedback from the subcommittee. 9 MR. pITTMAN: No. I would just like to say one i 10 thing, I would just like to conclude, Chet, just by saying, I 11 think we've got it down straight that we have used from the top I 12 down approach to get down to the level of the activities in 13 which we' re working. I hope we' ve got across the message that 14 we' ve got a process here that's workable, that's usable and one 15 that will communicate, not only within the agency, btc outside 16 of the agency of what we' re doing. 17 Thank you. l 18 MR. REMICK: Chet, could I make just one quick 19 comment on the one slide. I thought Dave Ward had a very l 20 important point, because you' re talking about communicat ing, i i I j 21 you say research goals are translated into safety assurance 1 22 questions. I think it's agency goals you' re referring to. I 23 That sure helps me knowing what you' re t alking about. q 24 DR. SIESS: The last slide that I asked him to skip j 25 in great big letters it says, "Facilitate communteation," and I h l i Heritage Reporting Corporation (202) 628-4888 l
1 think personally that's very true. If this committee were J 2 reviewing the research program for priorities as we did for a 3 period of several years looking at budgets, the kind of 4 informatirn that I see on some of these summary sheets that 5 Roger handed out, single pages as well as the ones with little 6 circles on it, I think have been extremely helpful. And not 7 orily in the subcommittee's review, but when it came to full O committee in the report we used to write. 9 How much is this costing? Is it less than one 10 percent of your research budget? 11 MR. ROSZTOCZY: Much less. The total effort about 75 12 K. () 13 DR. SIESS:
- Well, I' ll start off the comment by i
14 saying I think it's a very worthwhile ef fort. I think that we 15 probably want to look at the questions in relation to the l 16 strategic plan, because that's an area in which we can advise 17 the Commission as to whether there are things omitted from the 18 strategic plan as interpreted by the panel or whether we would 1 I 13 interpret them differently. Our interaction with the staff ) 20 might be at a different level. 21 I personally would encourage this effort completion, 22 the steps that they went to. 23 Carson, do you have any comments? l 24 DR. MARK: Nothing to add. i i 25 MR. MICHELSON: I think I would endorse what you CE) l Heritage Reporting Corporation (202) 628-4888
118 1 said, Chet. This seems to be nothing more than a logical 2 process, and I guess a well advised manager would follow 3 through it if he had the time and capabilities of doing all 4 this. It's well structured. It flows out quite logically. So 5 I would say it's a much better way to go about it then just 6 havinD a manager trying to deal with it at one time. 7 DR. SIESS: I don' t think anybody can sit down and 8 come up with this sort of thing without some sort of a process 9 behind it. And this is very useful. 10 Dade. 11 D R. MARK: Can I put some words in Carl's mouth. You 12 do want to have at least one sentence somewhere on the subject 13 of fire. 14 (Laughter) 15 DR. SIESS: That is a very important item, because if 16 they can' t derive fire protection from the strategic plan and 17 we feel strongly about it, if we do, then we should tell the 18 Commissioners, you know, you left something out of your 19 strategic plan which we didn' t really look at. 20 MR. WYLIE: Let me ask a question in that regard. El There's a scoping study goinD on in fire protection. There's 22 one going on in equipment qualification. There's something on 23 human factors. It seems to me that this is a good way to 24 approach, to decide whether or not you' re going to do those at 25 all or not. Now, is that going to be factored into this Heritage Reporting Corporation (202) 628-4888
) I 1 program? 2 MR. ROSZTOCZY: This is suppose to provide you for 3 the information to make that kind of decision, so it will be d 4 given to all of those who are going to use it. Obviously, the l i' l 5 reactor is going to use it, other people in the organization l ) 6 including the Commissioner. ( l .l 7 MR. WYLIE: The intent to factor in for decision 8 making is whether you can start a program or not. 4 9 MR. ROSZTOCZY: That's right. 10 MR. MICHELSON: But the problem is, nothing is said 7 11 in the strategic plan about fire protection. 12 MR. WYLIE: But those were questions raised outside i 13 of this plan. i } f 1* DR. SIESS: If the staff was constrained not to do ) 15 anything that wasn' t mentioned in the strategic plan, then 16 presumably there would never be anything done on fire research. 17 I' m not sure that's a constraint. 4 l 18 Dade. J 19 DR. MOELLER: I would simply repeat a couple items 20 that we said. One is, how comprehensive is this strategic 21 plan, you know, it's all based on that unless it's thorough and i 22 complete and addresses all the issues, then you may be missing 23 something. 24 I felt the oral interview approach is good. I still l 25 needed to know what other possible approaches would be f i Heritage Reporting Corporation (202) 628-4888 i i
O 120 1 available. I see though the advantages of this approach that 2 you' ve already said such as facilitat ing communication. r 3 Another thing that it helps on is, it makes quite a 4 few people feel important who otherwise might believe they have 5 been neglected, because you invited them in and you showed them j 6 you wanted to hear what they had to say. a 7 I like the part that it permits you to save good sub-B units of a bad program as well as to throw out bad sub-units of t 9 a good programl that appeals to me. i 10 I think I would repeat Dave Ward's question of Just 11 how does the new Advisory Committee on Safety Research fit in. i 12 And lastly, one item I didn' t get to ask that perhaps l 13 I don' t understand it, but I found that I could not in my own 14 m i r.d support what I understood was a $1 million line of ( 15 demarcation between an expensive and inexpensive program, 16 because from what I looked at, when you have $20 million 17 programs and you' re saying anything above i million is i i 18 expensive and anything below is not. I need something that l l ] 19 covers the full range. I need -- I felt I needed -- 20 DR. SIESS: Three level. 1 21 DR. MOELLER: Yes. i l 22 DR. SIESS: A million 300 to over a million, 300 to a i 1 23 million. I i 24 DR. MOELLER: Right. But one million, if there are 1 25 programs that cost 10, 15 and 20 million, to say one million is Heritage Reporting Corporation l (202) 628-4888
1 (^) 2e2 1 the cut off. 2 DR. SIESS: You' re right. 3 MR. ROSZTOCZY: Dr. Moeller, we have dropped that 4 approach. We have changed it. The new sheet, what you have 5 seen, instead shows three numbers. Total cost of the program, 6 the cost 7 DR. MOELLER: Right. For a year. 8 MR. ROSZTOCZY: -- and that's what we have provided. I 9 In terms of the possible alternatives, I can tell you, another 10 way to do it is to ask the divisions who are doing the programs 11 to prepare those sheets, Simply ask the divisions to put 12 together something and prepare those sheets, which can be done ] 13 somewhat faster, the question accomplishes everything. 1 14 DR. SIESS: I f you keep talking you' re not going to 15 get einswers. I'm glad you said that because the report is 16 still quite confusing whether they' re talking about per year or I l 17 per five years or per total. And this makes sense. i 18 paul. I i 19 DR. SHEWMON: I guess my concerns, I think in general 20 it's an interesting exercise. I wonder how much paper is going J 21 to be generated by whom in the process, and that is a question 22 of whether one should get rid of these three judges, can you 23 pick up three other judges who will do as well and who is going ) 24 to do it. 25 My lesser questions perhaps that weren' t clear to me Heritage Reporting Corporation (202) 628-4888 i
O 122 1 was, i guess the usual way of getting questions into the system 2 is still there. If NRR has a concern, they come in, the 3 research people. 4 And finally, what they call research and what I call 5 research may be different and maybe their definition is better 6 for their system because if you' ve got a problem you can 7 define, then defining the problem is research. And whether you 8 can have a research outfit that can ask meaningful questions 9 and give you results that will help it is a question Dave Ward 10 and I argue periodically about human factors in that area. I l 11 agree it's important. I' m not sure whether you can Dive 12 somebody money and have them make the operators perform better, 13 but if you define the question better, maybe that's the 14 research that needs to be done rather than -- okay. 15 DR. SIESS: Dave. 16 MR. WARD: Well, I guess overall I think it's a good 17 program. I mean, it's logical, you know, almost frighteningly 18 logical. Because of that ! - you know, there might be a 19 danger of getting too locked into the process in coming to 20 believe that it really reveals truth. But I think there's 21 protection against thatt and that the research director is 22 going to have to make the final decisions on priorities and 23 exercise his judgment there. That seems to me a good idea. 24 I' d like to see the committee, our committee, take a l 25 more active role or something -- I think the translation from O Heritage Reporting Corporation (202) 628-4888 ________---_____________--------------I
l 1 the strategic plan to the questions or whatever we call them is i 2 very, very important. It seems to me that this sort of process 3 couldn' t even have been conceived unless the strategic plan had 4 been developed. And I think that was very -- passed attempts 5 of the long range plan for research, I think really suffered 6 from not knowing what the research program was suppose to 7 serve. I mean, there wasn' t any agency long range strategy, so 4 8 this helpt. 9 I think one of the problems in us reviewing the 10 strategic plan -- ! mean, the questions derived from the l 11 strategic plan, the research questions, is that we really 12 haven' t -- we provided some input early on to the strategic ) 13 plan, but I don' t think the committee has taken a very serious 14 look or evaluation of that, so we might run into that. 1 15 As far as the process with the interviews, tapes and 16 that sort of thing, I guess I agree with Dadr. there might be i 17 another way to do it, but I'm kind of glad to see that this ) 18 kind of progressive approach with getting some guidance from 19 experts on interviews I think -- I'm really glad to see that's f I 20 been used, because I think it can be useful, i 21 D R. SIESS: Let me toss a question, I don' t know if l 22 there's anybody here that can answer it or maybe it's just 23 rhetorical. If the people that wrote the strategic plan had 1 24 known it was going to be used for this purpose do you think i 25 they might have done it somewhat differently? ( i Heritage Reporting Corporation (202) 628-4888 i i
~ + i () 124 1 MR. ROSZTOCZY: The strategic plan is being revised. j 2 A new process was established to include every manager in the 3 entire agency, in the division -- three sets of meetings. One 4 quarter of the agency managers meeting at one time going 5 through the major areas. c 6 DR. SIESS: That's not the question. The question 7 is, would it have been easier to get the questions out of the i 8 strategic plan if the people that wrote the strategic plan had 9 realized it was going to be used for that 10 MR. ROSZTOCZY: They didn' t know it. l j 11 DR. SIESS: Well, that leads to the suggestion the j 12 people who write the strategic plan should know clearly what ( 13 has been done with it. How its been used. 1 14 MR. WARD: I guess I have a little trouble with that l l .l 15 answer, they didn' t know. What did they think it was going to i t 16 be used for. 17 DR. SIESS: Well, not necessarily for research. 18 MR. WARD: Sure it's going to be used for research. 19 DR. SIESS: But they phrase things differently. i 4 i 20 MR. WARD: I mean, maybe not for this particular l 21 process or anything. But surely a purpose of the strategic 22 plan was to provide a basis for research. l 23 DR. SIESS: I suspect that if they knew it was going f I 24 to be used to develop a list of 28 questions ranked in order, i 25 they might have written things a little differently. l C^) a o Heritage Reporting Corporation ) (202) 628-4888 i J
125 1 MR. WARD: possibly. I think this might be useful 2 feedback to the strategic plan, I guess that's the idea. 3 DR. SIESS: I also think that the level -- agency 4 levels below the top should take the strategic plan and develop 5 an office strategic plan from that. pull out those things that 6 are applicable to those office. 7 Charlie. 8 MR. WYLIE: Well, I endorse what has been said. I 9 think it's a good logical approach to decision-makingl I think 10 it's a good program. 11 DR. SIESS: Forrest. 12 MR. REMICK: I definitely think it's a worthwhile 13 program. Just that the only product was the summary of 14 activities that communicates so well. From my standpoint that 15 would be a tremendous plus of the thing. There are some things 16 I could nitpick, the sheet that has advantages and 17 disadvantages, I wasn' t quite sure advantages of what or 18 disadvantages of what. I could nitpick that just because 19 something is going to be turned over to USGS ! don' t see that 20 as a disadvantage, I see that as more of an observation, 21 necessarily. But those are minor things. 22 One comment, we talked about human factors in I-5, 23 believe, but I would argue that it's also in B-10, the one 24 about severe accident training and so forthi a lot of human 25 factors in that. So I don' t think human factors has been Heritage Reporting Corporation (202) 628-4888
l 4 1 l l (:) 225 1 completely overlooked. 2 Thare's just no question in my mind, I think it's a 3 worthwhile exercise. The real test will be when we come down 1 4 to whose ox is going to be gored. It's very much -- some of 5 the discussion earlier I thought was very much what DOD must i 6 face when they try to get rid of a military base and each 7 congressman says, "I'm in favor of it unless it's in my i j 8 district." And that's what it's going to come down to here, it i 9 will be ACRS members pushing for their favorite research topic, l 1 l 10 and there are office directors and division heads disagree when i j i 11 it comes down to the final result. 12 DR. SIESS: Jesse. 13 MR. EBERSOLE: Well, Chet, you know, I' m an l l j 14 infiltrator into your committee here, I' ve got a meeting at 1 15 o' clock and I think that got in part. I recognize this might i i I l 16 be due to my reputation as being, I think, a complicator, so i i i 17 I' ll try to avoid that and be global in my observations here. l f 18 I broke this report up into four pieces and the last i 19 three ! sort of lumped in the packages of earth sciences and i 20 core melt progression and hydrogen generation, and the last one l 21 was core models validation analysis. 3 i ] 22 The first two, of course, have to do with l l 23 earthquakes, and then the disastrous aspects of having a core j 24 melt all in the context of trying to deal with after the fact I 25 disaster. (:) l Heritage Reporting Corporation l l (202) 628-4888 I i I i
_~ d T () 127 1 DR. SIESS: You realize this is only one-tenth of the 2 total purpose. I S l J 3 MR. EBERSOLE: Right. But I' m wondering, is there 1 4 any -- I don' t know what goes beyond this. I' m bothered by the t 5 fact that I see the best part of this in the context of just i 6 codes and validation and analysis. I don' t know what research 7 is. I j 8 DR. SIESS: They only looked at eight out of the 72 l 9 areas, they work it in. i 10 MR. EBERSOLE: So it's going to be a big package 4 j 11 before we get done. l 12 DR. SIESS: It will be - ( 13 MR. EBERSOLE: I think it's a great ef fort. Don' t j 14 get me wrong, I think this is great. } 15 DR. SIESS: This diagram will go out over there. } 16 MR. EBERSOLE I think this is a well worthwhile [ l 17 endeavor to get some sort of perspective in this business. ) f 18 DR. SIESS: Let me make a final comment. The thing 19 that impressed me when ! read this was the frequent use of two i 4 20 words, "Questions and uses," which I have found absent in much ] 21 of what went on before. ) i I 22 So, I would like to poke this thing, which is from a l 23 letter that was written to the Honorable Alan K. Simpson in 1 \\ j 24 1981 in response to a question from Congress about the uses ) i 25 which the NRC's research program had been put. And I think in Heritage Reporting Corporation J j (202) 623-4888 0
I 128 1 1981 the budget was significantly larger than it is now. 2 And the quote says, "If good research is done to [ f 3 a"swer good questions, the results usually will be used. 4 Unusable results are more likely to be the result of bad L i 5 questions then a bad results." l 6 DR. MARK May I inquire of one thing. paul brought l a 7 up the fact that the present list of questions will be combed ) B and gone over by people like the users, the NRR and other i i 9 branches. There used to be a time when research had difficulty l i 3 10 in selling a program unless it had a user endorsemert. Has i 11 that been improved, that if it's a good program is it led by 4 12 this excellent bunch of people, it wouldn' t have to take the () 13 enthusiasm of NMSS to get it on the books? j 14 MR. SpEIS: The program has the support of the user i J l l 15 office, there's no question about his being given, you know, I 16 the highest consideration. Larry raised this question today } 17 about the NRR users, if they have participated in this process, 18 which they have been consulted or whatever. I want to bring to 19 your attention that there is nome type of insight in NRC right i 20 now, you know. RES -- the offsce of RES has most of the user l 21 organizations right now. We have generic issues there, we also ( 22 have the implementat ion of severe accidents. l i l 23 So there is a kind of a program. You know, you have i i 24 the office director in a kind of schizophrenic way, where one l 1 25 group is telling him this and one group is telling him that. Heritage Reporting Corporation ) j (202) 628-4888 f f i r
t ([) le9 1 So that's why I would like to encourage, see if we 2 can get going with something like that. It's going to make my l 3 Job easier and averybody else's Job. I f 4 If we can find a way to help us, to formally tell our l 5 management that, hey, this is something that makes sense, maybe 6 you should support it. I looked, for example, this is the l 7 first time that I have been exposed to this and I look at what i i 8 these people have done here, there is no question it is great. h 9 But if the user people have been involved I think some of these 4 10 would be different. How important it is for the research 1 11 people to participate with the user, because there is those 12 types, you know, that have a project and they don' t put it in i ( 13 some type of perspective. They want to count all the neutrons i 14 before they feel that a decision can be made. And as all of us j l 15 know, you know, making a decision with what you have is the i 16 most difficult thing, it's going to perpetuate things forever. j 17 I don' t want to prolong this, but i 1 18 DR. SIESS: Now, these things will be -- users will l 19 have a chance to comment on these things since they've argued 20 it. l ] 21 MR. SPEIS: That's correct. That's a very important l 1 j 22 step. s ] 23 DR. SIESS: This gives you a good basis now for what 1 l 24 we refer to as a meaningful dialogue between researchers and l 25 the users. And I think that's got to be an advantage. l Heritage Reporting Corporation (292) 628-4888
() 130 1 MR. SPEIS: What I' m saying, this will facilitate 2 the job of the director of RES because he has all the groups 3 right now. He has the resource side and he has the 1 4 implementation side, you know. And this will help him a lot. i 1 5 Otherwise, you know, he will have to -- I 6 MR. MICHELSONI perhaps the interview process should i i 7 involve the user office as well as the research office. 8 DR. S!ESS: That's what I asked. 9 MR. MICHELSON: I never heard -- I mean, this process J 10 will work perfectly well interviewing research as well as RES. i 1 i 11 DR. S!ESS: There's some advantage in carrying it i ) 12 through to this point before you go back to users and have a 13 discussion. i 14 On the user need issue, the panel here developed the i 15 needs along the strategic plan. Now it seems to me that if the 16 Commission has promulgated a strategic plan that indicates a i ) 17 need, it shouldn' t be necessary to get some office over there 18 to agree on it. 19 MR. SPE!S But there's st ill priorit ies and there r ] 20 are specific programs. 21 MR. MICHELSON: And the time schedules. l 22 MR. SpEIS: The time schedules, right. 23 MR. ROS2TOCZY: The goal is to be able to give to our 24 office director the shoot that's in your hand in such a manner l l 25 that everybody agrees on that this is a fair representation. i (2) I Heritage Reporting Corporation ) (202) 628-4888 1 )
l I w 131 1 DR. SIESS: But this is for a research project, i i 2 M R. ROSZTOCZY: But it has to -- that's right. The 3 questions come directly from -- i 4 DR. SIESS: If you go back to the strategic plan and c i j 5 see something in there that's an issue, a quest ion and there's l l 6 no research being done it, and you have to now get some office 1 j 7 to sponsor. i j 8 M R. ROSZTOCZY: We need a decision on the appropriate i i 9 level that ther a should be something done on it. 1 l 10 DR. MARK: That dialogue is what attempts have been f ] 11 missing. Research couldn' t go ahead unless NRR said, yes, it's f i 12 a good idea. And then they would introduce ideas which { 13 research said, that is nonsense. i 14 MR. SPEIS: You mean on a specific program? l 15 DR. MARK A few hundred thousand dollars here or ( ) I 16 there. l 17 MR. SPE!S: But there are other sources. L 18 DR. MARK: And you' re going to have a real headache l 1 1 13 when you get to management because that isn' t within RES. l i 20 DR. S!ESS: Forrest. 21 MR. REMICK Just an observation, I think long term ! l 2 i 22 would like to see this type of product for the technical i 1 \\ j 23 assistance efforts, also. j 24 DR. SIESS: It's somewhat smaller and shorter. By 25 the time they got it done the project would be over. 1 ) Heritage Reporting Corporation j .{ (292) 628-4888 .. m
! () 132 1 MR. REMICK: Sometimes, not all of them. Some 2 continue a long time. 3 DR. S2ESS: Gentlemen, we talked earlier about going 4 over the actual questions that they developed out of the 5 strategic plan, which ! think would involve having the [ 6 strategic plan in front of us at the time. Would you like to j 7 do that at some time in the future. Not today. [ d i ) 8 MR. WARD: I don' t know about as a subcommittee i 9 activityl maybe we should do that as individuals or something l 1 i 1 10 and let you correlate that. i 11 DR. S!ESS: Let's start off by asking the people, do i 12 they have a copy of this draft report. I don' t think the j 13 questions are going to change very much between the draft and [ T 14 the final, are they? l 15 Sam will be sure that you get copies cf the strategic 1 i 16 plan. And Sook at these questions and see if they make sense j i 17 to you as questions individually, collectively. Are they i 18 repetitious. Forrest made a comment about whether human i i I 1S factors was here or there. And put those thoughts down in [ l l ) 20 writ ing and we' ll collect them up and go on from there. I i ) 21 t h i r.k this is at a detail type level. t ) 4 22 Now, as far as getting something formal back to your J I 23 boss, whoever that is, whether it's the Commission or Eric, the l j 24 subcommit t ee can' t do that. 1 25 MR. SPE!S Is there anyway it can be filed through 1 i l Heritage Reporting Corporation (202) 628-4888 l I
(:) 233 1 the committee. 2 DR. SIESS: I assume ! can bring it up in the full l ) 3 committee week after next, since we have nine members of the l 4 committee here, that's more than a majority. We' ll bring it up 1 5 and see what the full committee, whether they'll write a letter 6 to Eric saying that the subcornmittee has reviewed this and that t j 7 the subcommittee felt very strongly, felt it was desirable and [ j 8 pass it on to him that way. The full committee will do it on l i 9 the basis of what we' ve heard. i i 10 You could bring it into the full committee and spend l j 1 i 11 a couple of hours, if we can find thz time, and then the full i 4 i 12 committee would act. But I' ll try it the other way. I don' t t ( 13 know why it shouldn' t work. We had a pretty good group here. 14 MR. MICHELSON: ! think it would be interesting for f 15 researchers to explain what they would do if they didn' t use J 16 t h i,3 process in the future, if the process would be used. 17 MR. SPE!S That's a good question, but you know the 10 answer to that. [ J i 19 MR. MICHELSON: I think I know the answer, but ! I 20 wouldn' t want to be presumptuous. l 21 DR. SIESS: They listen to the ACRS and they do what 3 22 we told them. i 23 MR. MICHELSON: Within their own structure what do f 24 they do to essent ially -- 3 25 DR. SIESS: Oh, they came into us in the past with a 1 j Heritage Reporting Corporation (262) 628-4888 1 1
i I l () 134 i hit list, you know, and how they got it I don' t know. It was 2 very difficult. t 3 MR. MICHELSON: Out of somebody's hip pocket i l 4 probably. 5 DR. S!ESS: No, they've come up with criteria. They ] j 6 give top priority to projects that's got some cost in them and 1 j 7 top priority list. We've had some criteria for doing it. It's 8 basically different than this, this is a more structured way of f 9 deriving at the thing and getting it down to where people know f i j 10 what you' re doing. l 11 DR. SHEWMON: Chet, talking of your hit list, ! 1 12 suspect one has been generated or is being generated again. l 13 Will we hear about that at all this year? L 14 DR. SIESS: Well, we did -- when we have Beckor in to i } } 15 talk about the budget off the record. Month before last. i j 16 December meeting, we heard it. I ): 17 DR. SHEWMON: That was before he knew what his budget ( l J l 18 was. l 19 DR. S!ESS: He had & target figure. It's in the 20 minutes for the December meeting, it came out as a separate i 21 item. Ask Tom. And he had priorities. 22 Gentlemen, I' ll havo to adjourn the meeting now i 23 because we' ve got another starting at 1 o' clock. I j 24 Thank you. i 25 MR. ROSZTOCZY: Thank you very much. Mr. Chairman, i 4 Heritage Reporting Corporation (202) 628-4888 1
i i i I 135 1 DR. SIESS: Thank you. i 2 (Whereupon, at 12:12 p.m. the tneeting was adjourned.) j 3 1 i 4 i 5 } f i 6 1 i r ] 7 l 4 l a t t l 9 ) i 10 f i 11 12 i 13 ( f l 14 [ l 15 l r } 16 p r i 17 r a l 18 r i 1 i j 19 i i 1 r 4 20 L 4 i 21 22 7 1 i
- )
23 1 l 24 l I 'l 25 q t l 1 i e ) Heritage Reporting Corporation (202) 628-4888 l )
l i i CERTIFICATE () 3 This is to certify that the attached proceedings before the l ( 4 United States Nuclear Regulatory Commission in the matter of: l ? l 5 Name SAFETY RESUARCH PROGRAM i l 6 7 Docket Number 8 Place Washington, D.C. i 9 Date: January 29, 1988 i 10 were held as herein appears, and that this is the original l 11 transcript thereof for the file of the United States Nuclear 12 Regulatory Commission taken stenographically by me and, t 13 thereafter reduced to typewriting by me or under the direction 14 of the court reporting company, and that the transcript is a 15 ) true and accurate record of the foregoing proceedings. /S/ N e m 16 L 17 (Signature typed)* Joan Rose 18 official Reporter 19 Heritage Reporting Corporation 20 21 l 22 23 24 25 I () Heritage keporting Corporation (202) 628-4888 l e
l O su==^av or AcTiviTv SEISMIC AND FIRE PROTECTION EARTH SCIENCES The Earth Sdences Research Activity provides the basic data required to understand I the frequency and severity of earthquakes in different areas of the country, it consists of geological / geophysical studies, efforts to revise Appendix A to 10 CFR 100, and i installation of the National Seismographic Network in cooperation with the US Geological Sodety to replace the current seismic networks. Several programs are in j place to advance understanding of the frequency, transmission, and effect of seismic
- events, Safety Assurance. The Earth Sdences programs are relevant to safety assurance, Significant work in this area has been going on for more than a decade. These programs have had major effects on nudear power plants. The current program is designed to supplement the existing data and to answer additional questions i
concerning the seismological risk to nuclear facilities (R 1). The risk significance of l these programs is highly uncertain, with the general consensus being that there are i significant safety margins in current methods of seismic design, Thus, these programs probably will not have a major safety impact on licensed facilities. Although the current i i body of seismic information probably is sufficient for characterizing the seismic risk at existing sites, these programs should provide a better basis for such characterizations if O additional seismic risk analyses are to be required in the future. The Earth Sciences Activity will advance the state of knowledge of seismic risk, but the extent and nature of NRC's future role in this area is uncertain, i Usefulness. These programs have been highly usefut to the NRC. They have been l very well coordinated with NRR through weekly staff dscussions and have been directly used in making regulatory dedsions. The programs have been well coordnated with USGS, the Corps of Engineers, and foreign govemment efforts. The program managers have succeeded in focusing their program on NRR needs and have been responsive to requests to address licensing problems. Acorocriateness. The objective of the Activity is to advance understanding of seismic risk, even though much is already known. The USGS is assuming Federal government responsibility for the new seismic networks, and NRC is in the process of phasing out of its supportive role in that area. Utilities and EPRI have seismic monitoring programs at individual sites, it is questionable whether NRC should continue to fund significant research in this area. However, NRC must keep abreast of improving kno 710dge developed by the sdentific communityin this area and adjust its requirements and practices as necessary to reflect contemporary knowiedge and residual uncertainties. 1 Resources. The Earth Sciences Activity has required a high expenditure of resources, 1 About $50 million have been spent to date, and another $20 million wi!! be spent by NRC in replacing the network and handir',g it over to USGS. The FY 1988 budget request is Q for more than $4 million, l
O 1 q 't i !! E ,fj 1 3 3, a i ]IK f ] [- ji I 384 I I1 jj 11: Y 11 1 I (1 -i; n'y i} 11 2 y i ly 11 i p I i i iI 1 lJ } o 'fI E !I !# !j i! i If.;f g3 } ' '. 1..-. '. M It' 0*;:. s * '. o a a a 15. 4:eade<;.
- .f i
e 4 J e >>4 >>aSSa> h g 0 ~ ~ v ge
- 1
~ g a E' g n 3 e l
- s l
- t-{l l
ii t tIo, e e e c e o e c t <
- eco fl i
l 1j 512ci 1 ., e c o e e e e o e e o i I f} b j 1-LJ G a.,_ g s eco g o e o e Ee o e e I = I 1 1 a a I 5 E f j4#d C 1 5 1 8 _a I I I. 53 I dl n=< a t g y ,i f ,$3 j O 3E. fj g = 9 u v> = Q Q C. C. C. .C .3 l
Ib FESEARCH ACTIVITIES et p { e s i .j T i., [i,'s ,, ]' 1 ,l i r g u.[ NRC RESEARCH QUESTIONS 3 ,l i i,< i i,i n. e i i v+ wouv s enveti..el ruce en e v: pe e renwe enweau.. .e. vs wonee poween e.rean n et enu ou sen -4 g v <i .... ees.e u oce us.... w 3 e es.o. e..e e .. e, e.... ei m e
- v. en.es... e.
o v e: own. nove.ei pie. ..ne. pin e ew.vuua n s on Q. ................,..,ei v.. .e v e..... u.e - v... ..iv.... g I ,v............................... t aumnie ownas nenu su eeanaw r v+own.........................< . 6ae.v.,e re.i eien ca nai. eewe v.is: awn.t. gs n a g eie.o osi.e.e eu ee.eu v.in sewe non.a..ren. e.. i.p en.w e v.ta.anerne one e re ssu u v .gur.evoo,......,................ .g. t. e... i. we
- v. e; u... u.c uvm.o e ma ees
...o .e v.is: c. o. e te iae inut es t e usuwe v.te; swouw t*eest e nes4un et emie tu S. .O< t:,4e eeuun se se se um,4e ence g,,............................... .e, e -e .....,..,e........... .s os espee evene e a. em u% e st, meen4 g.....,44eneu e w ei<ie r e 6 <s coe e e esas t 6s.t g ..we m e, e mee o.aoc. si E .,o.... .,n............. to: i. pen met m u.c..
- e. w e, sec, o
6.em.e s.e w.e m eve ng ..............e.................< g Se susew e. u.eu.e en e A e H-Swee we. u.e4emie oc en e, o w. o w e ei. k i e cu n =e, oc uve s o ie go e, ma 6 t e c.e is ne. p u.,m . w nee.e est s & t e. 6.e.n o e...wic v e.e m,+ e lie. .. in. m no ..eue .ese suo en u p. menu we.. e 4 Fis. si.s en m.o owe me erw 4% &te semo tun, nu.ainwame . _.....,. e... u a. .e,o..o, e. g ..s : e.e.c.u..oie n e.en. po.su.
- 4. e.ss e............
e s r 4 g g a 4: n.ame u ana.u,in w e,oe a4 seu oc.cev 6.umg e w 3 .a s wcwe gwe esece o e.eewe se ues ro ueu .....,..............,ee,..,,.......... . e een en. e....... ee e e..... e o. et c e een.n.. et c m.e tu eewe .no a.eut 34 9 i e -. - e.-. e_ o .. -. -een
c NRC RESEARCH PRIORI'lIZATION PRESENTED I TO THE ACRS SUBCOMMITTEE ON SAFETY RESEARCH PROGRAMS JANUARY 29,1988 (i t PARTICIPANTS: f Z. ROSZTOCZY, NRC/RES 1 J. PITTMAN, NRC/RES D. MEIER, SCIENTECH l R. MATTSON, SCIENTECH, INC. i L. YBARRONDO, SCIENTECH, INC. 3, R. BLOND, SAIC G. HUMPHRESS, SwRI L Yf ust' nesearch Pnonteation g A-1 [y g;_ =;_.__ g prepared by Scientech. Inc. w
l F' P N I l h i 4 / i mum Ii$ 1i i i 8 x 3 1 i E l 1 l i O o oc z 0 D R o o i i \\ O N O L E Q H co e Z O cc yl i 4 8 i 6 M .8a .S E 8 1 i c. E i ne b i' cc i 5i o ggy u _. ---...,a=-..,-----_.,,-._.._
I D REQUIREMENT 6 B q li f "PRIORITIZE RESEARCH PROJECTS ACCORDING TO SAFETY SIGNIFICANCE " - NRC STRATEGIC PLAN JULY 29,1987 l l 1 t I s O PROVIDE INPUT TO: I PLANNING I RESOURCE ALLOCATION 1 COMMUNICATION i A-3 i USP'e Research Priontization s prepared by Scientech. im w
BACKGROUND i I SEVERAL PREVIOUS RESEARCH PRIORITIZATION EFFORTS HAVE BEEN UNDERTAKEN I i
- 1. RISK BASED PRIORITIZATION
-INTERNAL STAFF EFFORT - USED ANALYTICAL HIERARCHY PROCEDURE (AHP) - QUERIED BRANCH CHIEFS AND DIVISION DIRECTORS - VERY QUANTITATIVE - HARD TO EXPLAIN, TEDIOUS TO IMPLEMENT r )
- 2. SIMPLIFIED RESEARCH PRIORITIZATION
- INTERNAL STAFF /BATTELLE COLUMBUS LABS - USED COMBINATION OF DECISION METHODS (AHP,"LIGHTYEAR") l - QUERIED DIVISION DIRECTORS I -INTERNAL BIAS - RESULTS VAL lDITY QUESTIONED A-4 1 USNf2C Research Prioritization i pepared by Scientech, k g T _ __ j
F O e i BACKGROUND (CONT'D) t n 4 I ,[
- 3. STAFF RESEARCH PRIORITIZATION
-INTERNAL STAFF - USED TABULAR QUANTITATIVE APPROACH ) - INTERNAL STAFF "EXPERT" TEAMS AS JUDGES i - STRONG INTERNAL BIAS j - RESULTS QUESTIONED [ - HARD TO EXPLAIN n lI i F; i l, l'. h i l l II j l A-5 [ 3 prepared by Scientech. Lr j USNTr' Research Priontization ~ l
l DIFFICULTIES (PREVIOUS EFFORTS) INFORMATION NEEDED FOR PRIORITIZATION WAS NOT i DOCUMENTED INFORMATION PRESENTED WAS NOT FACTUAL AND DID NOT REPRESENT A CONSENSUS s I CRITERIA SHOULD BE RELATED TO ALL AGENCY GOALS, RISK REDUCTION ALONE IS NOT ENOUGH r 0 - QUANTIFICATION OF RISK REDUCTION POTENTIAL IS UNCERTAIN ,\\ 1 PARTICIPANTS HAD LIMITED KNOWLEDGE OF SOME PROGRAMS I SOME OF THE ASSESSMENTS WERE PARTIAL l l! [ l b A-6 yusn nesearca erioritizatio" ! ir2sesa
- * **a'*
- 9
- 1 h
DIFFICULTIES (CONT'D) i PARTICIPANTS DID NOT HAVE ENOUGH TIME TO STUDY AND j DISCUSS PROGRAMS i i l CRITERIA AND RATINGS WERE INTEPRETED DIFFERENTLY BY k SOME PARTICIPANTS i; LACK OF UNIFORMITY AMONG PARTICIPATING GROUPS i i PRIORITIZATION OF RESEARCH AREAS ONLY I I .l l 1 ii l f; l! A-7 USNF3Q Research Priontization g g y__ prepared by Scientech, Inc. ~
p GROUND RULES OF NEW EFFORT 1 r i METHODS - SIMPLE TO USE PROCESS ~ ( - COLLECT ALL NEEDED INFORMATION O - DOCUMENT INFORMATION IN IMPARTIAL MANNER 0 l - EXTENSIVE REVIEW AND DISCUSSION - SYSTEMATIC DISPLAY OF INFORMATION ij j CRITERIA - AGENCY GOALS (STRATEGIC PLAN) - CONTRIBUTION TO ACHIEVING GOALS - RESOURCES NEEDED \\ f - APPROPRIATENESS OF NRC INVOLVEMENT 1 l c A-8 j lL USNGC Research Priontizaton g prepared by Scientech. Inc. i
h 11 GROUND RULES (CONT'D) l 9 l i j AUTHORITY - ASSEMBLE INFORMATION - DIRECT DRA, RES l - ACCUMULATE USER OFFICE & RES DIVISION COMMENTS-1: DIRECTOR, DRA, RES I l - PRIOR!TIZE - DIRECTOR, RES ( RESULTS - RESEARCH ACTIVITIES (72 TOTAL) WILL BE PRIORITIZED I - EVALUATION AGAINST CRITERIA WILL BE DOCUMENTED a - RESULTS WILL BE PRESENTED IN SCRUTABLE FORM k 4 - LIVING DOCUMENT, SUBJECT TO PERIODIC CHANGE i L i i i l A-9 UStar Research Priorcization [gg prepared by Scientech. Inc.
T 0 1 A c n I hce t ne o S y b de rape rp t 1 )sN IF S S. S ( H M ME M E S T C A AI A IT C R R C R R R I R A G O G V t A NE G
- t. O G O
T R f l I 1S O E R R C T ER P T R P A P A NO 5C 9-2 C 1 7 003 ~ y no itaz itnio rP hc r ae se R 1 PNS U i\\ Jl ; 1 i i I l i.l lt ? ll .I
r APPROACH NRC/RES WILL DIRECT ACTIVITIES 1 CONTRACTOR SUPPORT TO: l, ? - DEVELOP A METHODOLOGY (PROCESS) - DOCUMENT CRITERIA AND PROCEDURES - COLLECT AND ASSEMBLE INFORMATION - DEVELOP DISPLAY OF RESULTS - CONDUCT TRIAL PRIORITIZATION s. - ASSIST IN DOCUMENTATION i J ') lL USNGC Research Priordization i g-prepared by Scientech, Inc. 1
t B. DESCRIPTION OF RESEARCH PRIORITIZATION s i L n. tl by ei i li H L. YBARRONDO, SCIENTECH, INC.
- !q t
I 9 ll { B-1 USh Research Priordirr.: ion [$ prepared by Scentech,Inc.
- -. - - - - - ~ l, a IMPORTANCE AND RELEVANCE OF NRC RESEARCH l l PROGRAMS NEED TO BE BETTER COMMUNICATED i 11 STRATEGIC PLAN l I! ll 1 RESEARCH AGENCY PHILOSOPHY AND i GOALS l PRINCIPLES 1 m m I V SAFETY ASSURANCE QUESTIONS l r y RESEARCH ) RESEARCH ACTIVITY PR10RIT8ZATl0N y-I I U M JL i ,j 1 ) i i l l INPbT TO: l PLANNING j l j 5 ALLOCATION OF RESOURCES I D i ! us$ResearchPrbritiraten! r% hex
-~ u 4 IMPORTANCE AND RELEVANCE OF RESEARCH ! BASED ON NRC STRATEGIC PLAN AND AGENCY GOALS
- l
[ REVIEWED AND USED STRATEGIC PLAN REVIEWED NRC PLANNING DOCUMENTS - PPG (NUREG-0885) 4 ( - FIVE YEAR PLAN I - SES WORKSHOPS 14 1 - RESEARCH PHILOSOPHY DISCUSSED PLANS WITH SENIOR NRC STAFF 4 [ -EDO -NRR l -NMSS 1 i -RES t f Y B-3 [ USNQesearch Priordizatori 5 prepared by Scientech. Inc. i
.cn I hce tneic S y b de rape r S p I; ISYLANA NA LP C I G ETA R TS k no tazid rio rP hc raes eg O ? SU { It a Lh8 !r ~, '. r iL > b I 1 1
l 1 j 1 IMPORTANCE AND RELEVANCE OF NRC RESEARCH 1 t j j PROGRAMS NEED TO BE BETTER COMMUNICATED I j STRATEGIC PLAN l i 1 i ll') RESEARCH AGENCY PHILOSOPHY AND ] i GOALS PRINCIPLES l l M M l l 7 j SAFETY ASSURANCE 1 QUESTIONS I i! l t 7 RESEARCH ACTIVITY PRI R TIZAT ON 7 I M i! J L i. i l! INPUT TO: PLANNING 1 y ALLOCATION OF RESOURCES COMt'UNICATIONS 4 4 B-5 i prepared by Scientech. Inc. j usthesearchPrion!izafon g i 1
~'i L RESEARCH GOALS ARE TRANSLATED INTO j SAFETY ASSURANCE QUESTIONS
- l 9
l [ VITAL - DIRECT AND IMMEDIATE IMPACT ON HEALTH AND SAFETY l IMPORTANT - EVENTUAL IMPACT ON HEALTH AND SAFETY l RELEVANT - USEFUL lNSIGHT ON HEALTH AND SAFETY i !~
- SAFETY QUESTIONS CHANGE IN RESPONSE TO AGENCY STRATEG C PLANNING PROCESS B-6
[ usg Research Priorcization E prepared by Soentech Inc. gg
I. 7_ g .cn I h,ce tneic S y b de rape rp l S ) N TR O O I P T E S R E F U O Q A Y X I T D E N E F P A P ~~ S A ( n h no taz idro irP hc raese Rg O tsu E e i i t' l' ( O
} CRITICAL CRITERIA ARE USED TO EVALUATE l RESEARCH ACTIVITIES i
- r.".1."=L.%.,
- Hmewat is entwung uWy 3
i Re.oustes Indude i
- =~"
rp- -e I I ( \\ n l \\ ? -r r o -r l kaa is, nac: a.= wee. J f . co i. i i.
- =-
- C:~0."'~'""-
- 2 = --
- Somosuhd emeSe j
j l B-e t pr pared by scient ch. ine. i ustgesearen er racaton g, ___ j o I
COMPREHENSIVE AND CONSISTENT PROCESS i IS USED TO GATHER INFORMATION i, ASSESSMENT STEPS PRODUCT OF STEP i t y 1.......,.....
- q
.c..,, v., s c ..$i, s,.'. " ' ' * ~ " -5 vwsc y i. 1 r l i s,............... 1 ..... a o.- o... i ui i .......... o... ...~o-.. i t , r i i j ..., o. . --., 2,- i -o - - s~ .... o s,,. ...m.. .......,,,,.....m., s.,... ,,,,,,,,,,,,,,,,,,,,,,.c .i..<,-- ...,..,.....,...,., ~ ,.o e c I 1 1 I B-9 I
- 'i!taass-PP"'ed by scientoch. inc.
__-.u- +a _.A. l ?. INTERVIEW PROCEDURES AND QUESTIONS i i g (APPENDIX B OF REPORT) I i i f n i' 6 B-10 k USN@lesearch Prentiza!on [ prepared ty Scientech. Inc. ^ ^
r-i l SENIOR MULTIDiSCIPLINARY PANEL IS USED TO EVALUATE AND JUDGE ACTIVITIES b I RISK ASSESSt1ENT l REGULATI0tt l I bi \\ ANALYTICAL /EXPERit1 ENTAL I RESEARCH i i B-11 prepared by Scientech, Inc. Usr lescarch Priordeaton [ g, 1
EVALUATION PANEL HAS EXTENSIVE KNOWLEDGE ! AND EXPERIENCE IN COMMERCIAL AND GOVERNMENT l STRATEGIC PLANNING AND PRIORITIZATION EFFORTS COMMERCIAL -EG&G - SAIC i - SOUTHWEST RESEARCH 5 - SCIENTECH GOVERNMENT l - NRC RESEARCH j -DOE i - EG&G IDAHO j MULTIDISCIPLINARY - REGULATORY - RISK ASSESSMENT - ANALYTICAUEXPERIMENTAL - INDEPENDENT OF INTERNAL BIASES i B-12 usNgescarch Prioritiraton g g prepr. red by Scientech. Inc.
___.~_#__ _~ ~-.__. ~ _ _- _ _.-.- - t l RESEARCH PRIORITIZATION METHOD PROVIDES l A MEANS OF EFFECTIVELY COMMUNICATING THE IMPORTANCE AND RELEVANCE OF THE l RESEARCH PROGRAM PROCESS IS BASED ON NRC STRATEGIC PLAN AND AGENCY i ) GOALS ) RESEARCH GOALS ARE TRANSLATED INTO SAFETY ASSURANCE l QUESTIONS - CRITICAL CRITERIA ARE USED TO EVALUATE RESEARCH ACTIVITIES -SAFETY ASSURANCE j - USEFULNESS l j - APPROPRIATENESS -RESOURCES i B-13 Usr ^ 9eserictiProrczaton g prepared by Scientech. Inc. I
y _ --. t l l (CONTINUED) I l l e COMPREHENSIVE AND CONSISTENT PROCESS IS USED TO GATHER i j INFORMATION i i- - SENIOR MULTIDISCIPLINARY PANEL IS USED TO EVALUATE AND i JUDGE ACTIVITIES p PROCESS IS EASY TO USE, REPEATABLE, RETRIEVABLE, AND L FACILITATES COMMUNICATION i i, 4 i I B-14 { ustgescarch Prorstmon g 372g pped by Scentech,Inc. =__
n-- ,A _n u A, W O r, - - - -._ -w - ~ ~ ~ l C. TRIAL APPLICATION OF RESEARCH PRIORITIZATION r F i by ei s { l R. BLOND, SAIC r l-t. k c C-1 l US?nc Research Pmn!C3!On g grepared by Soentech, Inc.
-. - - _ _. + _.. - _ _. ___._.E _m_ _.___.,_h m au., 2_ e M ~ - ~ - l TRIAL RESEARCH PRIORITIZATION L PRESENTED i TO THE l ACRS SUBCOMMITTEE ON SAFETY RESEARCH PROGRAMS i E JANUARY 29,1988 s c l l I C-2 i t ustsc nescarcn ercrezzon g 1/2&M prepared by Scantech,Inc.
TRIAL DEVELOPED TO TEST PRIORITIZATION PROCESS AND PROVIDE USEFUL RESULTS PURPOSE OF TRIAL DETERMINE IF CRITERIA ADEQUATELY DISCRIMINATE BETWEEN ACTIVITIES DETERMINE IF PROCESS IS EASY TO CONDUCT e PROCEDURES ADEQUATE? O CONSISTENT APPROACH? e RESOURCE INTENSIVE? NRC STAFF PANEL l DEVELOP USEFUL PRESENTATION OF RESULTS e COMMUNICATES RESULTS PROVIDES RESULTS IN A FORM WHICH CAN BE EASILY USED PLANNING RESOURCE ALLOCATION COMMUNICATION DETERMINE IF PROCESS IS REPEATABLE 1 I IS OUTSIDE GROUP REQUIRED? IS IT SUFFICIENT TO INTERVIEW ONE PERSON / ACTIVITY s. g 1!28MMAC.PSYJ4P1592 C-3 j usv=c nesearen prontea:on g-pr. par.a w scioni.cn. a j
l ACTIVITIES WERE SELECTED TO PROVIDE l MEANINGFUL TEST OF PROCESS i SIGNIFICANT PROGRAMS l - LARGE BUDGETS (>$20 MILLION PROGRAMS) - AT LEASTTWO DIVERSE AREAS -8 ACTIVITIES SEISMIC AND FIRE PROTECTION l EARTH SCIENCES COMPONENT RESPONSE TO EARTHQUAKES l VALIDATION OF SEISMIC ANALYSIS j - SEISMIC DESIGN MARGIN 6 REACTOR CONTAINMENT I CORE MELT PROGRESSION AND HYDROGEN GENERATION l CORE / CONCRETE INTERACTION DIRECT CONTAINMENT HEATING CODES, MODELS, VALIDATION, AND ANALYSIS c-4 iuzauon g. .m.__ g.p..o w w. inc. u
1 DESIGN OF INTERVIEWS PRE-INTERVIEW - ENVIRONMENT SET UP i ORIENTATION ON-CAMERA ACTIONS PROLOGUE lNTERVIEW QUt: STIONS i TRY TO FOLLOW QUESTIONS CONSISTENCY - QUESTIONS REFINED AS TRIAL PROGRESSED I TIMING f APPROXIMATELY 1 HOUR l - FLEXIBILITY r i lNTERVIEWERS i AT LEAST 2 USED l l C-5 l USME Research Prordizaton [ p__ prepared by Sckcaech, k T i
CONDUCT OF INTERVIEWS SET INTERVIEWEE AT EASE NO PREPARATION NECESSARY EXPLAIN PROCESS IN DETAll TRIAL TAPES ONLY USED BY PANEL MEMBERS A ND FOR ARCHIVE [ l I e START BY ASKING ABOUT NEW RESEARCH OUESTIONS/NEEDS LOOK TO FUTURE - ll FOCUS ON CURRENT RESEARCH f SAFETY ASSURANCE USEFULNESS APPROPRIATENESS p 4 RESOURCES L WRAPUP 1 COST-BENEFIT STRATEGIC PLAN l l I i l! : 1728'88:MACPSY.RP19/R6 l C-6 { usne. nesearch Priontization gog pr. par.a by sc.ni.ch u l
l DOCUMENTING OF INFORMsTION g-e TAPE ARCHIVE o WRITTEN NOTES e INDIVIDUAL REVIEW BY PANEL MEMBERS - DEVELOP POSITION ON CRITIERA RATING e PANEL CONCENSUS ON RATINGS - RELATIVELY EASY NO MAJOR ARGUMENTS - PROCESS SEEMED TO WORK
SUMMARY
PAGE ON ACTIVITY - EVALUATION OF ACTIVITY ) i tr28/88 MACPSYJtP20417 l C-7 g ustmc nesearch erioraizz:en g m, z p, par.o dy sci.niecn. inc.
i PROBLEMS ENCOUNTERED e [! e SCHEDULING OF INTERVIEWS FLEXIBILITY I INTERVIEW ROOM NOT ALWAYS AVAILABLE ENVIRONMENT IMPORTANT i CAMERA VIEW, SOUND gl l e LANGUAGE 1: jl OVERALL VERY MINOR PROBLEMS i! Il l i i h l I l i! 1/28/88 MACPSY:RP21/R8 i l! i l C-8 USNBC Research Predizaton [3 prepared by Scientech, Inc. g
-. - - - - ~ ~ - - _ _ ~~* TRIAL PROCESS CONCLUSIONS l EASY TO PERFORM REPEATABLE RETRIEVABLE MULTI-DISCIPLINARY PANEL VERY IMPORTANT EASY TO COMMUNICATE ON CRITERIA DISCRIMINATES VERY WELL e TAPES WERE VERY USEFUL INCREASED INTENSITY ADDED FOCUS l I i tr28.S83AAC.PSYJ1P22/R9 C-9 usvac nesearcn erornizaton p,. par.o by sc.ni.ch. inc. g,,
.m a
_m 6 .a- ~- O W ~ D. RESULTS OF TRIAL APPLICATION f i l by I R. MATTSON, SCIENTECH, INC. l l !l l' i i l i D-1 j uste n.x earca emniuato" I i/28/88 - N'****
_. +. __m. _ -. ~ - - - ll PRESENTATION OF RESULTS CRITICAL TO SUCCESS OF RES PRIORITIZATION PROGRAM 1 i APPLICABILITY i SCRUTABILITY i I [ USEFULNESS/ EFFECTIVENESS l i PLANNING RESOURCE ALLOCATION t 1 i i COMMUNICATION i ii l I! II D-2 l { USNGC Research Priontizaton g, prepared by Soentech. Inc.
=_n I I TRIAL PROGRAM PROVIDED OPPORTUNITY TO l EVALUATE THE RESULTS i l
SUMMARY
OF ACTIVITY i - HIGHLIGHTS IMPORTANT CONSIDERATIONS OF CRITERIA J AND RANKINGS l i
SUMMARY
OF RESULTS OF EVALUATION e i I - ACTIVITIES VERSUS SAFETY QUESTIONS j il - ACTIVITIES VERSUS CRITERIA 4 n i D-3 l usnm nesearen eriori"' '*" t ir28ma = 9 1
l i ONE PAGE
SUMMARY
OF ACTIVITIES PROVIDES BASIC RATIONALE FOR EVALUATION t SUPPORTS RANKING PROCESS EFFECTIVE DOCUMENTATION PROVIDES OPPORTUNITY TO REVIEW AND REVISE EVALUATION l i i EASY TO UNDERSTAND j l l l i l D-4 USNGC Research Prontizaton NN q ! 1/28/88 =
c_ i e i: O
SUMMARY
OF ACTIVITIES VERSUS SAFETY QUESTIONS ( PROVIDES FOCUSED VIEW OF RESEARCH PROGRAM l l l l IDENTIFIES THE EFFECTIVENESS OF THE PROGRAM IN l ADDRESSING SAFETY QUESTIONS i IDENTIFIES THE ACTIVITIES FOR WHICH THERE ARE NO l SAFETY QUESTIONS r l IDENTIFIES THE SAFETY QUESTIONS FOR WHICH THERE I 4 ARE NO ACTIVITIES l i D-5 usrJGC Research Prontizaton l1728/88 NN l
i l
SUMMARY
OF ACTIVITIES VERSUS CRITERIA l PROVIDES INFORMATION TO SENIOR MANAGEMENT i ) FOR PRIORITIZATION l 1 "CONSUMER GUIDE APPROACH" i l l 1 CONCISE l i 1 . DIRECT COMMUNICATES RESULTS j SELF-EXPLANATORY l FAMILIAR FORMAT l i 1 l D-6 j usnac nesearen er.c,rnirai. " t ir28/88 i
"t, - - - - - - - - - - J INSIGHTS INTO RESULTS OF TRIAL PROCESS l CRITERIA PROVIDED GOOD DISCRIMINATION OF ACTIVITIES - IDENTIFY CRITICAL SAFETY ISSUES AND AGENCY NEEDS l l - ENABLE CONSISTENT EVALUATION OF PROGRAMS l i li h l - EASY TO COMMUNICATE i - FACILITATE IDENTIFICATION OF CONSENSUS AND DISAGREEMENT l D-7 ustmc nesearch Proraiiaton g g, - _ _ , p, pa,.d by sc.atech. Inc. W- \\
_= f INSIGHTS INTO RESULTS OF TRIAL PROCESS k CONTINUED PANEL OF EXPERTS INTERACTION VERY IMPORTANT i - MULTI-DISCIPLINARY JUDGEMENTS - EXPERIENCE 4 - BIAS i L PROCESS IS FLEXIBLE p I - ACCOMODATES REVISION FOR PROGRAM l EXPANSION OR CONTRACTION I - ACCOMODATES AGENCY REDIRECTION FOR POLICY OR ~ l RESOURCES l t j 0-e usnenescarcn erornuten ,,,p.,,o ey w% u g,, _ _ - __ _ _ _
_____._~.~.Zl_._ ~~ ~~- -' ~ -~ ~' E. CONCLUDING REMARKS i i ti il I i by i i i J. PITTMAN, NRC l n i t I I L i E-1 u 'c nesearch Prontuaton g ig,33 - prepared ty sown.ch. inc. 7
. _ = _ _ il c FUTURE ACTIVITIES L l. t 4 k PRESENTED 1 l I TO THE !i 4 ACRS SUBCOMMITTEE ON SAFETY RESEARCH PROGRAMS l l JANUARY 29,1988 I I i i l i E-2 N* h* * - ac nescarch erioritizaron [ 1/28.s8 i l
-a2,--a a aa n-- A s --,--asa m-_ e-.,-- 4 m e g --- ~-- - - - - - - - - - - L )( FUTURE ACTIVITIES i L l COMPLETE THE REVIEW OF THE TRIAL EFFORT (NRC) i U MAKE NECESSARY CHANGES TO THE PROCESS (SCIE) l l 1 GATHER AND ASSESS DATA ON ALL RES ACTIVITIES (NRC/ SCIE) REVIEW AND COMMENT BY USER OFFICES & RES DIVISIONS (NRC) t ,l REVISE ACTIVITY DATA SHEETS AND RATINGS (NRC/ SCIE) j l PRIORITIZATION BY DIRECTOR, RES (NRC) i l E-3 [. t7C Hesearch Prioritization g g _ prepared by Soentech,Inc.
_ _ _ =__ = = = - g (=_=_====RCH PRIORITIZATION METHOD PROVIDES RESEA ) A MEANS OF EFFECTIVELY COMMUNICATING THE i ll lMPORTANCE AND RELEVANCE OF THE I RESEARCH PROGRAM PROCESS IS BASED ON NRC STRATEGIC PLAN AND AGENCY i GOALS i i i RESEARCH GOALS ARE TRANSLATED INTO SAFETY ASSURANCE l l QUESTIONS j i j - CRITICAL CRITERIA ARE USED TO EVALUATE RESEARCH l ACTIVITIES -SAFETY ASSURANCE l - USEFULNESS l - APPROPRIATENESS 1 - RESOURCES ( E-4 l t t- ~9c nescarch Priordization ggg prepar.d by soent.ch. ine.
f I (CONTINUED) i ( COMPREHENSIVE AND CONSISTENT PROCESS IS USED TO GATHER l INFORMATION SENIOR MULTIDISCIPL! NARY PANEL IS USED TO EVALUATE AND JUDGE ACTIVITIES 1 PROCESS IS EASY TO USE, REPEATABLE, RETRIEVABLE, AND FACILITATES COMMUNICATION t f i ) l E-5 i -,%% u l ur-, neeao nmaan g_ - -}}