ML20151D257
| ML20151D257 | |
| Person / Time | |
|---|---|
| Issue date: | 02/10/1988 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | Advisory Committee on Reactor Safeguards |
| References | |
| ACRS-2551, NUDOCS 8804140073 | |
| Download: ML20151D257 (43) | |
Text
(
vLL Le DATE ISSUED:
2/10/88
SUMMARY
/ MINUTES OF THE ACRS SUBCOMMITTEE MEETING ON THE SAFETY RESEARCH PROGRAM JANUARY 29, 1988 WASHINGTON, D.C.
INTRODUCTION The ACRS Subcomittee on Safety Research Program held a meeting on Friday, January 29, 1988, at 1717 H Street, N.W., Washington, D.C., to discuss the proposed methodology to be used by the Office of Nuclear Regulatory Research (RES) Staff in prioritizing NRC research activities.
The entire meeting was open to public attendance. Mr. Sam Duraiswamy was the Cognizant ACRS St'aff Engineer for this meeting. A list of documents submitted to the Subcomittee is included in Attachment A'.
A copy of the presentation schedule for the meeting is included in Attach-ment B.
ATTENDEES ACRS:
C. P. Siess (Subcomittee Chairman), J. C. Ebersole, C. Mark, C. Michelson, D. W. Moeller, F. J. Remick, P. G. Shewmon, D. A. Ward, and C. J. Wylie Sam Duraiswamy (Cognizant ACRS Staff Engineer)
Principal Speakers NRC:
Z. Rosztoczy and J. Pittman Scientech, Inc:
R. Mattson and L. Ybarrondo SAIC:
R. Blond EXECUTIVE SESSION Dr. Siess, the Subcomittee Chairman, convened the meeting at 8:30 a.m.,
and stated that the purpose of this iweting was to discuss the follow-ing:
g 41 g 3 000210 2551 PDR
Safety Research Program Minutes, January 29, 1988
- Proposed methodology to be used by the RES. Staff in prioritizing research activities.
- Results of the trial application of the prioritization methodology to certain research activities.
He stated that the Subcommittee had received neither written comments i.or requests for time to make oral statements from members of the public.
PROPOSED RESEARCH PRIORITIZATION METHODOLOGY Introduction - Mr. Z. Rosztoczy, RES Mr. Rosztoczy stated that the purpose of prioritizing the NRC research activities is to:
- Provide input for research planning and allocation of resources.
- Facilitate communication with other NRC Offices, Commission, ACRS, other federal agencies (such as the Departnent of Energy (DOE) and the Office of Management and Budget (OMB)), and the industry.
He stated that several attempts were made previously to prioritize the NRC research activities (Attachment C, Pages 1-2); however, they were not successful.
Some of the difficulties encountered with these previ-ous efforts were:
- Infermation needed for prioritization was not documented.
- Information presented was not factual and did not represent a consensus.
- Quantification of risk reduction potential was uncertain.
e y
g
,,4
Safety Research Program Minutes January 29, 1988 i
l
- Participants had limited knowleage of some research programs.
- Criteria and ratings were interpreted differently by some partici-pants.
- Some of the assessments were partial.
Mr. Rosztoczy stated that based on the experience gained from the previous prioritization efforts, they have formulated the ground rules for the new prioritization effort (Attachment C, Pages 3-4).
Some of the ground rules for the new effort are:
- Develop a simple to use process.
- Document information in an impartial manner.
- Determine the contribution of research to achieve agency goals delineated in the NRC Strategic Plan.
- Determine who should do what.
i
- Present the results of the prioritization in scrutable form.
- Include the results in a living document, subject to periodic change.
l Mr. Rosztoczy mentioned that Scientech, Inc. has the primary respon-sibility for developing the research prioritization methodology, docu-menting all information, and ccnducting trial application of the method-ology to certain research. The results of the prioritization effort will be given to the Director of RES who has the overall responsibility for assigning priority rankings to various research activities.
o Safety Re, search Program Minutes January 29, 1988 i
Dr. Siess comented it seems that the proposed effort is to prioritize the existing research projects rather than to prioritize the questions of the research needs.
This approach might work if the infonnation is going to be used in deciding which projects shoulo be cut to accomodate a possible budget reduction.
However, if they havt to decide what research should be done, the questions on research needs should be prioritized. Mr. Rosztoczy responded that the results of the priori-tization will not only be used to docide which projects should be cut to accomodate a possible budget reduction, also it will be used for other purposes. The first step in the development of the prioritization methodology is to formulate questions that must be answered to ensure that NRC can fulfill its goals defined in the Strategic Plan. RES and Scientech, Inc., had already developed such questions and will be presented to the Subcomittee at a later part of the meeting.
Dr. Moeller asked whether, in the proposed priori.tjzation effort, they are seeking. technical deficiencies or administrative problems within the research program. Mr. Rosztoczy responded that the main effort is to look at the technical nature of the program to make sure that it will provide necessary technical information to achieve the agency goals.
In response to some questions from Dr. Shewmon, Mr. Rosztoczy stated that one aspect of the prioritization effort is to evaluate whether the research program will provide useful and timely information to accom-plish the agency goals.
Further, the purpose of the prioritization effort is to ensure that the research program will provide necessary information to accomplish the agency goals at present as well as in the future.
Dr. Siess comented that they cannot ignore the research done in the past when judging the future.
If a research is done well, the results will be useful.
If it is not done well, the results will not be useful.
Until the result of a specific research is available, they may not know
a d
Safety Research Program Minutes January 29, 1988 whether it will really be useful. Therefore, somewhere in the process they have to make a judgment as to what the probability is that they will get good results out of a specific research project. Mr. Rosztoczy responded that they normally assume that a specific research project will provide good, meaningful, and useful results. However, if they have any doubts about the usefulness of the results after receiving them, the process allows them to reevaluate and reprioritize that particular research project.
Mr. Ebersole commented that prior to the TMI-2 accident major emphasis was placed on the research associated with a hypothetical large-break LOCA.
He always felt that the amount of money spent on this matter was not directly proportional to the benefits expected of this research. He believes that a major part of that money should have been spent on more important issues. Mr. Rosztoczy responded that owing tc severe budget constraints he does not believe that they will beuable to do such things anymore. Under the proposed process, they will ask questions periodically on the need for, and usefulness of, a specific research and make adjustments as needed.
Mr. Michelson asked at what level that they plan to prioritize the research program. Mr. Rosztoczy responded that the NRC research program is subdivided into 5 program categories; these are divided into 19 programs. These 19 programs consist of 72 activities and there are about 300 contracts supporting these 72 activities.
Prioritization will be performed at the activity level.
Mr. Michelsor comented that assigning priority at research activity level may have some ramifications.
Each activity involves several research projects.
If an activity is assigned with a low priority ranking, it implies that all projects under that activity will also receive a low priority. Realistically, it may not be true, because some of the projects under such activity may be dealing with very significant
Safety Research Program Minutes January 29, 1988 issues.
He asked how the Staff plans to handle such a situation. Mr.
Rosztoczy responded that all activities will be grouped into 3 groups:
Top group, Middle Group, and Bottom Group. Activities in the Top Group are considered to be dealing with important research issues. The prioritization process will provide a brief summary of all research projects under each activity that will include their purpose, information expected of them, and their significance.
If there is a budget cut, they will first look for candidates in the Bottom group.
Then, they will look for candidates in the other groups.
Mr. Michelson commented that if budget decisions are made at the activi-ty level as stated by Mr. Rosztoczy, there is a possibility that the activity with low priority ranking might be dropped owing to budget constraints.
If that happens, some important projects under that activity will also get dropped. Mr. Mattson responded that the proposed prioritization process takes cognizance of such things happening in the past and tries to prevent similar things happening again. The new process will help them to find important projects under a low ~ priority activity as well as not so important projects under a high-priority activity.
Mr. Ward asked whether the Technical Assistance Programs will be fac-tored into the prioritization process. Mr. Rosztoczy responded that those Technical Assistance Programs under RES will be factored into the process. Those under NRR will not be prioritized. However, the prioritization process will be reviewed by NRR and other user offices.
If NRR feels that some of its Technical Assistance Programs should be factored into the prioritization process, RES will consider doing so depending on the urgency and significance of those programs.
e w
y.-.-
Safety Research Program Minutes January 29, 1988 Description of the Research Prioritization Methodology - Dr. L.
Ybarrondo, Scientech, Inc.
Dr. Ybarrondo discussed briefly the development and description of the prioritization methodology (Attachment C, Pages 5-8).
He stated that the agency goals are articulated in its Strategic Plan. Usine the information in the NRC Strategic Plan as a main source, they have developed the prioritization methodology. Some of the steps involved in the methodology are:
- Translate the research goals delineated in the Strategic Plan into questions that must be answered by the NRC to ensure nuclear safety.
- Assess the questions in accordance with their safety assurance (AttachmentC,Page6).
- Vital to enhancing safety
- Important to enhancing safety
- Relevant to enhancing safety
- Map current research back to the questions identified previously to determine whether:
- Current research is directly related to the questions
- There are some questions for which research is not currently being performed
- There are some research efforts that do not appear to answer questions that need to be answered j
n
__n
,.,n_-,.
-v,.
Safety Research Program Minutes January 29, 1988
- Assess the current research to determine its usefulness for resolv-ing research questions using the following categories:
- Highly useful
- Useful
- Somewhat useful
- Assess research in accordance with its appropriateness for the NRC funding using the following categories:
- Appropriate for the NRC funding
- Appropriate for joint funding
- Questionable for the NRC funding
- Assess research in accordance with the resources required to carry it out:
- Low cost
- Moderate cost
- High cost
' Combine all research assessments and determine the priority ranking based on:
- Safety assurance significance
- Usefulness
- Appropriateness for the NRC funding
- Resources required Dr. Ybarrondo stated tSat review of several documents such as NRC Strategic Plan, the report issued by the National Research Council on Revitalizing Nuclear Safety Research, information on previous priori-tization efforts, and discussion with several personnel, indicated that
i Safety Research P ogram Minutes January 29, 1988 the relevance and importance of the research program need to be comu-nicated better.
He believes that the proposed prioritization process provides a means of comunicating effectively the importance and relevance of the research program. Also, it is easy to use, repeatable, and retrievable. He said that in the development of the methodology:
- Comprehensive and consistent process is used to gather necessary information.
8 Senior multi-disciplinary panel is used to evaluate and judge activities.
With reference to the statement by Dr. Ybarrondo that research goals are.
translated into safety assurance questions, Dr. Siess comented that this approach seems to be the reverse of the process.
It should say that the safety assurance questions are translate.d.into research goals.
Mr. Ward commented that since the safety assurance questions are derived from the agency goals delineated in the Strategic Plan, they should say that the agency goals are translated into safety assurance questions.
Mr. Ward asked whether they would have been better off or worse off in develesing the prioritization process if they hadn't had the Strategic Plan to start with. Dr. Ybarrondo responded that they would have been much worse off without the Strategic Plan.
He believes that previous prioritization efforts had failed because there was no Strategic Plan available at that. time to provide the basic information.
Stating that the probability of success in getting an answer to a l
question depends on the nature of the question as well as the ability of the persons who are actually doing the research, Dr. Siess asked whether both of those factors were taken into account in the development of the l
prioritization process.
Dr. Ybarrondo responded that although the
Safety Research Program Minutes January 29, 1988 capability of the people conducting the research was discussed by the panel, they did not actually factor that into the process.
Dr. Siess asked whether the panel talked to personnel in the user offices, such as NRR, to obtain their opinion on the appropriateness of the safety assurance questions. Mr. Rosztoczy stated that the questions developed by the panel are being reviewed by user offices. Any comments by the user offices will be factored in prior to issuing the final document on the prioritization process.
Dr. Blond stated that they had discussed the safety assurance questions with the E00's Staff and senior level management of the NRC Offices and incorporated their comments.
Trial Application of the Research Prioritization Process - Dr. R. Blond, SAIC Dr. Blond discussed the trial application of the research prioritization process (AttachmentC,Pages9-14). The main purpose of the trial application process was to determine whether the process is:
- Adequate
- Easy to conduct
- Repeatable He discussed briefly the research activities selected for use in the trial application process and the steps involved in the interview process (AttachmentC,Pages10-12).
Dr. Blond stated that based on the results of the trial application of the prioritization process, they have concluded that:
' The process is easy to perform, repeatable, and retrievable.
- Multi-disciplinary panel is very important.
- The process discriminates adequately between research activities.
.e,
Safety Research Program Minutes January 29, 1988 Dr. Siess asked whether they have made any changes to the questions during the course of the trial application process. Dr. Blond responded that after the first or second interview, they made some significant changes to the questions. He said that they plan to add some more questions to the existing list for future use.
Dr. Siess asked who actually conducted the interview.
Dr. Blond re-sponded that Dr. Mattson, Mr. Meier, and himself conducted the inter-view. Dr. Mattson said that they had a rule that every member of the panel should observe every interview and they followed that rule in the trial application process. Since they realized that it is a time consuming process, they are thinking about changing it for the future.
Mr. Michelson asked at what frequency that they plan to repeat this process. Mr. Blond responded that it is up to the RES management. Mr.
Rosztoczy stated that the current plan is to prioritize all 72 research activities. After that, they plan to apply the process to those activities where there are some changes or to new research activities.
He requested that the ACRS provide its opinion on this matter.
Results of the Trial Application of the Prioritization Process - Dr. R.
Mattson, Scientech, Inc.
Dr. Mattson discussed the results of the trial application of the prioritization process (Attachment C, Pages 15-24).
Some of the in-sights gained from the results of the trial application process are:
- Presentation of the results is very critical to the success of the RES prioritization program.
- The results are useful and effective for the purposes of planning, resource allocation, and comunication.
Safety Research Program Minutes January 29, 1988
- One page sumary (Attachment C, Page 22) on each research activity that includes information such as the objective, usefulness, required resources, source of funding, etc., of that specific activity:
- Is easy to understand
- Supports ranking process
- Provides an effective documentation of results
- Provides opportunity to review and revise the evaluation
- Sumary of activities versus safety assurance questions helps to identify:
- The effectiveness of the research program in addressing safety questions.
l
- The research activities for which there re no safety ques-tions.
- Safety questions for which there are no research activities.
' Sumary of research activities versus criteria provides concise and explicit information for use by the senior level management in prioritizing research activities.
1 In response to a question from Mr. Michelson, Dr. Mattson stated that when they say that the process is "repeatable" they mean that they could use a different set of people (panel members, research program managers) and come up with the same conclusions.
Dr. Siess asked how do they know that the process is repeatable.
Dr.
Mattsen responded that since they haven't repeated the process they don't.
Safety Research Program Minutes January 29, 1988 Stating that some of the NRC research managers might have provided biased opinion on their programs during the interview, Mr. Ebersole asked how did they factor this issue into the process.
Dr. Mattson responded that the panel members were aware of the possibility of receiving biased opinion from certain research managers.
If the panel members are knowledgeable about the NRC research programs, they would be able to detect whether someone is providing a biased opinion.
Dr. Mattson suggested that the ACRS review the adequacy of the sr.fety assurance questions derived from the Strategic Plan by the panel mem-bers.
Dr. Rosztoczy stated that if the Subcommittee wants to discuss the safety question, in detail in one of its future meetings, RES would be happy to participate and provide necessary information. Mr. Ward stated that it would be worthwhile to review the safety questions in detail.
Dr. Siess stated that this issue may be scheduled for discussion at a Subcomittee meeting in the future.
Mr. Wylie asked at what stage of the process they plan to obtain input frUm NRR. Mr. Rosztoczy responded that at the time of developing safety questions based on the information included in the Strategic Plan they had talked with NRR and obtained their input.
Further, the draft report that contains safety assurance questions and information on the trial application of the prioritization process and the associated results has been sent to NRR for review and coments. Subsequent to completing the prioritization of all 72 research activities, they plan to send the results to NRR for review and coment.
Dr. Siess asked whether they plan to issue a final document that will describe the entire prioritization process.
Dr. Mattson said yes.
Safety Research Program Minutes January 29, 1988 Mr. Ward asked how the recently established Nuclear Safety Research Review Committee is expected to interact with the prioritization pro-cess. Mr. Rosztoczy responded that at this time it is not clear how that Committee plans to deal with this process.
Documents available so far on this matter had been given to that Comittee for review and coment.
With reference to the list of questions vital to safety assurance, Mr.
Ebersole stated that there are questions associated with the adequa:;y of plant maintenance, and quality of construction and operation. He asked why they did not include even a single question on the adequacy of the plant design. Dr. Rosztoczy responded that these questions are derived from the information in the Strategic Plan.
He believes that since design is not emphasized in the Strategic Plan, they did not include any questions on design. Dr. Mattson stated that it was an oversight by the panel and he believes that they should have included some questions on the adequacy of the design.
Stating that fire is a significant contributor to risk, Mr. Michelson comented that the Strategic Plan should have included some information on this issue.
Dr. Mattson responded that during the interview when asked about areas where research should be done, it was pointed out by some that fire protection research needs to be done.
Dr. Siess comented that if there is no information included in the Strategic Plan on fire protection and if we think it should have been included, we should bring it to the attention of the Commission.
Future Activitics Associated with the Pricritization Process - Mr. J.
Pittman, RES Mr. Pittman stated that some of the activities that need to be done in the futurc are:
i
Safety Research Program Minutes January 29, 1988
- Complete the review of the trial effort.
- Make changes to the process as necessary.
- Gather and assess data on all research activities.
- Obtain coments from user offices and RES divisions.
- Revise activity data sheets and ratings as necessary.
- Provide necessary information to the RES Director for use in prioritizing research activities.
SUBCOMMITTEE REMARKS Dr. Siess stated that in his opinion the prioritization process is an worthwhile effort and it should be pursued further until its completion.
The ACRS should look at the adequacy of the safety questions, derived by the panel, in relation to the information in the Strategic Plan and provide coments to the Comission on items that should have been included in the Strategic Plan.
He solicited coments from members of the Subcommittee.
Some of the coments provided by the Subcomittee members are given below.
Mr. Michelson: He believes that the prioritization process is logical and well-structured.
j Dr. Moeller:
' Since the process starts with the information included in the l
Strategic Plan, it should be ensured that the Strategic Plan is comprehensive and incluoes information on all important matters.
I
Safety Research Program Minutes January 29, 1988
- Although he feels that the oral interview approach is good, he is interested in knowing what other options are available.
- He believes that one of the important aspects of the process is that it helps to identify important projects within a low-priority research activity and not so important projects within a high-priority research activity.
- He is interested in knowing how the Nuclear Safety Research Review Committee fits into this effort.
Dr. Shewmon:
- He believes that it is an interesting exercise.
' He wondered whether a different set of panel members would come up with the same conclusions as the present panel.
Mr. Ward:
' He believes that it is a good and logical process.
- It is a good idea to leave the final decision on assigning priority rankings to the RES Director.
- Since the agency goals defined in the Strategic Plan are translated into safety questions, he believes that the ACRS should review the adequacy of these questions to determine whether the Strategic Plan includes all important issues.
- He is also interested in knowing what other options are available other than the oral interview approach.
Safety Research Program Minutes -
January 29, 1988 Mr. Wylie:
He agrees with the comments made by other members. He also believes that it is a good and logical approach.
Dr. Remick:
- It is a worthwhile process.
- He would like to see this process applied to Technical Assistance Programs.
- Summarizing the results of the evaluation in one page is a good idea.
- It would be interesting to see how certain people would react when their favorite projects got cut based on the results of this process.
Mr. Ebersole:
It is a good effort.
Dr. Siess stated that he plans to provide a report to the full Committee on this matter during the February 11-13, 1988 ACPS meeting. Also, he 1
may submit a letter on this matter to the full Committee for considera-tion. He suggested that the Subcommittee members look at the adequacy of the safety questions and provide their opinions.
Dr. Siess thanked all participants and adjourned the meeting at 12:12 p.m.
l i
NOTE:
Additional meeting details can be obtained from e transcript I
of this meeting available in the NRC Public Document Room, 1717 H Street, N.W., Washington, D.C., or can be purchased l
from ACE-Federal Reporters, 444 North Capitol Street, Suite l
402, Washington, D.C. 20001, (202) 347-3700.
l
LIST OF DOCUMENTS SUBMITTED TO THE SUBCOMMITTEE 1
SAFETY RESEARCH PROGRAM JANUARY 29, 1988 1.
Presentation Schedule.
2.
Research Assessment Methodology. Task II - Trial of Methodology, prepared by Scientech, Inc., January 1988.
3.
Presentation Material submitted to the Subcomittee During the Meeting.
1 i
I j
1 i
ATTACHMENT A
i PRESENTATION SCHEDULE ACRS SUBCOMMITTEE MEETING ON THE SAFETY RESEARCH PROGRAM JANUARY 29, 1988 ROOM 1046, 1717 H ST., N.W.
WASHINGTON, D.C.
l ACRS CONTACT: Sam Duraiswamy 202-634-3267 NOTE:
Presentation tire should not exceed 50% of the Total Time allocated for a s>ecific item. The remaining 50% of the time is reserved for tie Subcommittee questions and answers.
8 Nurber of copies of the presentatien materials to be submitted to the Subcommittee:
25 copies l
I TOTAL PRESENTATION ITEM PRESENTER TIME ACTUAL TIME i
15 min 8:30 - 8:45 am 1.
EXECUTIVE SESSION 2.
RES PRESENTATION a
a.
Introduction Zoltan Rosztoczy 30 min 8:45 - 9:15 am (RES) b.
Description of the Larry Ybarrondo 45 min 9:15 - 10:00 am Research Prioritization (Scientech)
Memedebyy
- BREAK ***
15 min 10:00 - 10:15 am c.
Trial Application of Roger Mattson 40 min 10:15 - 10:55 am the Research (Scientech)
Prioritization d.
Results of the Trial Roger Blond 40 min 10:55 - 11:35 am Application (SAIC) e.
Concluding Activities Jim Pittman 10 min 11:35 - 11:45 sm (RES) 15 min 11:45 - 12:00 Noon 3.
SUBCOMMITTEE REMARKS J
i i
- ADJOURN ***
12 Noon ATTkentAT 9
.W
BACKGROUND p!
l n
SEVERAL PREVIOUS RESEARCH PRIORITIZATION EFFORTS HAVE l
BEEN UNDERTAKEN
- 1. RISK BASED PRIORITIZATION l
- INTERNAL STAFF EFFORT
- USED ANALYTICAL HIERARCHY PROCEDURE (AHP)
- QUERIED BRANCH CHIEFS AND DIVISION DIRECTORS
'l
- VERY OUANTITATIVE n
[
- HARD TO EXPLAIN, TEDIOUS TO IMPLEMENT e
- 2. SIMPLIFIED RESEARCH PRIORITIZATION
- INTERNAL STAFF /BATTELLE COLUMBUS LABS
<} [L 4
- USED COMBINATION OF DECISION METHODS (AHP,"LIGHTYEAR")
- QUERIED DIVISION DIRECTORS
{.
- INTERNAL BIAS
- RESULTS VALIDITY OUESTIONED C-I o
c e
g M
,j USNRC Research Priorittration r
! 1/28/88 P'P*'N M'* k
li i
BACKGROUND (CONT'D) l
- 3. STAFF RESEARCH PRIORITIZATION i
- INTERNAL STAFF
- USED TABULAR QUANTITATIVE APPROACH
!)
- INTERNAL STAFF "EXPERT" TEAMS AS JUDGES
- STRONG INTERNAL BIAS
- RESULTS QUESTIONED l
j;
- HARD TO EXPLAIN 0
'i 0
il h
ll h
g
?
C-2, N
1
=
j USNRC Research Prioritization ! 1/2&88 pr% Nente, k
7 ll ll s
ll GROUND RULES OF NEW EFFORT ii i
I j
METHODS
- SIMPLE TO USE PROCESS l
- COLLECT ALL NEEDED INFORMATION l
- DOCUMENT INFORMATION IN IMPARTIAL MANNER j
- EXTENSIVE REVIEW AND DISCUSSION 4
- SYSTEMATIC DISPLAY OF INFORMATION I
k.
[
. CRITERIA
- AGENCY GOALS (STRATEGIC PLAN)
]
- CONTRIBUTION TO ACHIEVING GOALS (l
- RESOURCES NEEDED
- APPROPRIATENESS OF NRC INVOLVEMENT c
!i i
il i!
lO l!
C-3 a
l
( USNRC Research Prioritization ! 1/28/88 N'
e
)
)
GROUND RULES (CONT'D) i h
i i
l i
L AUTHORITY
- ASSEMBLE INFORMATION - DIRECT DRA, RES
- ACCUMULATE USER OFFICE & RES DIVISION COMMENTS -
i l
- PRIORITIZE - DIRECTOR, RES t
- RESULTS
- RESEARCH ACTIVITIES (72 TOTAL) WILL BE PRIORITIZED
- EVALUATION AGAINST CRITERIA WILL BE DOCUMENTED
- RESULTS WILL BE PRESENTED IN SCRUTABLE FORM 1
- LIVING DOCUMENT, SUBJECT TO PERIODIC CHANGE
!l H
L j
C-4
!i b
s
~
h M
USNRC Research Prientization ! 1/28/88 pepared by Scientech. Inc. p_
~
IMPORTANCE AND RELEVANCE OF NRC RESEARCH 1
l J
PROGRAMS NEED TO BE BETTER COMMUNICATED e
j STRATEGIC "O
)
PLAN
'f f
[
c h
q RESEARCH "a
f A5ENCY PHILOSOPHY A?D m[
q GOALS PRINCIPLES i
m it t-U 1 f SAFETY ASSURANCE l
QUESTIONS l
M t
)
)
i f
' f y
S ARCH ACMTV PRI R TIZAT DN 3
l J L i
INPUT TD:
PLANNING "l
7 ALLOCATION OF RESOURCES COMMUNICATIONS l
b--h 8
f%
4 M
i
{
UStJRC Research Priontization ! 1/28/88 "
1 i
i RESEARCH GOALS ARE TRANSLATED INTO SAFETY ASSURANCE QUESTIONS
- VITAL 1
i
- DIRECT AND IMMEDIATE IMPACT ON HEALTH AND SAFETY i
l i!
l l
3 lMPORTANT l
I
- EVENTUAL IMPACT ON HEALTH AND SAFETY j
I t) i L
RELEVANT 1
L L
L
- USEFUL INSIGHT ON HEALTH AND SAFETY i
1 l
)
c-6 i
O ii
-
- SAFETY QUESTIONS CHANGE IN RESPONSE TO AGENCY STRAT.EGIC PLANNING PROCESS s
i M
Os i
L USNRC Research Proritization j prepared by Soentech, k
... - ~ -.-
i CRITICAL CRITERIA ARE USED TO EVALUATE RESEARCH ACTIVITIES i
l i
u j
h
- e a v m aare:
r l
-*=ar., semy I
~ ~ -* ~ ~,.~v 2
u.
1 I
i l
i
\\
,"* =-_,Pd='*** d
['
j h.=arx c in-e. ane
'd**"""-
poi.i,, w -
i r
i l
I i
i
- - r 1 c
=-
!E~
iM
- CM e.edd O. !
C-7 I
E ustmc nesearch erenteaton ;m prepar o by sc.nt ch. w w
l
I
-a l
l COMPREHENSIVE AND CONSISTENT PROCESS O
IS USED TO GATHER INFORMATION j
h l
ASSESSMENT STEPS PRODUCT OF STEP e
j 1
-s 8
l Teped intervtews R
4 veesc' E m wtth unc.tef f y
int.rvt.w.
g e
eooot r....r c h n
VHSC ac t t ve t tes
[
_u 4
l l
f Judges f ormut ete list R e s e arc h At trtbut e of research guse t tone
"**t8'"
Def tne t tone end c ommon de f 9 nt ( t on,
.f t he e s t,t eut e.
..e.
In
.,,. e. u, n...
essee stng researc h (Queet tene the e Aeprwo ot m. <
h ac t t we t tee (J o e f u l ne s s,
og*nC W.'eus t y,,yg,
h A pproprt et ene s s, one wer t o l
no oo,ce.
continue rnooien et..ne..een>
p g
I,i i
l I
'I l
l
- 4..
i.
Ac t.n2 9.
e,anoi the,uoge.
sh, A.,
f ormulate e c et tec t t wo
$he A
- 8 * *
- N* RI wh nuwd uttWtg en e s s es sment of the research Sheet terms of usefulnes s.
eooroertetenos e f or unc fundt
....ng, and re g ourC #
_ _ - - " ~ ' - - - - - -
e
, e
,e., s.
l l
1 i
C-8 Q
N 33$
USimC Research Prio<deation pfcpared by Scientech, Inc.
g
r_
2 f TRIAL DEVELOPED TO TEST PRIORITIZATION PROCESS i
AND PROVIDE USEFUL RESULTS PURPOSE OF TRIAL DETERMINE IFCRITERIA ADEQUATELY DISCRIMINATE BETWEEN ACTIVITIES i
e DETERMINE IF PROCESS IS EASY TO CONDUCT e
PROCEDURES ADEQUATE?
CONSISTENT APPROACH?
RESOURCE INTENSIVE?
NRC STAFF PANEL i
DEVELOP USEFUL PRESENTATION OF RESULTS 8
COMMUNICATES RESULTS PROVIDES RESULTS IN A FORM WHICH CAN BE EASILY USED PLANNING RESOURCE ALLOCATION.
COMMUNICATION DETERMINE IF PROCESS IS REPEATABLE IS OUTSIDE GROUP REQUIRED 7 IS IT SUFFICIENT TO INTERVIEW ONE PERSON / ACTIVITY O
tr28.es mcrsyneiss2
{
USNRC Research Priontization ! 1/28'88 NN
~
f ACTIVITIES WERE SELECTED TO PROVIDE MEANINGFUL TEST OF PROCESS
~
[
SIGNIFICANT PROGRAMS k
- LARGE BUDGETS (>$20 MILLION PROGRAMS)
- AT LEAST TWO DIVERSE AREAS
-8 ACTIVITIES SEISMIC AND FIRE PROTECTION EARTH SCIENCES COMPONENT RESPONSE TO EARTHOUAKES VALIDATION OF SEISMIC ANALYSIS SEISMIC DESIGN MARGIN t
9 REACTOR CONTAINMENT CORE MELT PROGRESSION AND HYDROGEN GENERATION CORE / CONCRETE INTERACTION DIRECT CONTAINMENT HEATING CODES, MODELS, VALIDATION, AND ANALYSIS r
C-Io o
i b [it the USNRC Research Prioritization g 1728/88 NN h-3
[
DESIGN OF INTERVIEWS 1
~
PRE-INTZRVIEW ENVIRONMENT SET UP l
l ORIENTATION
~
i ON-CAMERA ACTIONS i
+
PROLOGUE f
INTERVIEW QUESTIONS TRY TO FOLLOW QUESTIONS L
i 1
CONSISTENCY
~
- QUESTIONS REFINED AS TRIAL PROGRESSED 1
i 1
I TIMING APPROXIMATELY 1 HOUR
- f 1
- FLEXIBILITY l[
l INTERVIEWERS
[
AT LEAST 2 USED g
c-il O
f M
L USNRC Research Priordization prepared by Scientech Inc.
5 a
i
CONDUCT OF INTERVIEWS l
SET INTERVIEWEE AT EASE NO PREPARATION NECESSARY EXPLA!N PROCESS IN DEf All TRIAL l
TAPES ONLY USED BY PANEL MEMBERS A ND FOR ARCHIVE e
START BY ASKING ABOUT NEW RESEARCH OUESTIONS/NEEDS LOOK TO FUTURE-FOCUS ON CURRENT RESEARCH q
SAFETY ASSURANCE USEFULNESS APPROPRIATENESS l
RESOURCES I
WRAPUP lj COST-BENEFIT STRATEGIC PLAN
'l 1
C-12 Q
1r28/88MACPSY_RP19/R6 L
N tihe prepared by Soentech, Inc.
USNRC Research Poontization g
i V
r- --
!I DOCUMENTING OF INFORMATION l
e TAPE ARCHIVE i;
e WRITTEN NOTES e
INDIVIDUAL REVIEW BY PANEL MEMBERS
- DEVELOP POSITION ON CRITIERA RATING ll e
PANEL CONCENSUS ON RATINGS RELATIVELY EASY e
- NO MAJOR ARGUMENTS PROCESS SEEMED TO WORK
SUMMARY
PAGE ON ACTIVITY EVALUATION OF ACTIVITY e
t 1728/88.MACPSYJ1P20717 L
, lu Jbt ustmc nesearch Proraizaron ;3__
g.p,w w sm. inc.
- - - - - =.
i:
PROBLEMS ENCOUNTERED i
e SCHEDULING OF INTERVIEWS FLEXIBILITY L
INTERVIEW ROOM c
NOT ALWAYS AVAILABLE ENVIRONMENT IMPORTANT L
i e
CAMERA VIEW, SOUND i
e LANGUAGE OVERALL VERY MINOR PROBLEMS o
ir28/88MAC:PSYRP21E8 L
+
GE8 S usranc nesearcn eriornizaron ; s, p, g.,.a e sc <n.ca. ioc.
r
l i
i h
i l
l PRESENTATION OF RESULTS CRITICAL TO
~
[
SUCCESS OF RES PRIORITIZATION PROGRAM i,
i APPLICABILITY ll j
i-SCRUTABILITY c
I l
i t
- USEFULNESS/ EFFECTIVENESS s
l i
i' PLANNING
{
c I
RESOURCE ALLOCATION l
i j
l:
i n
COMMUNICATION i n C-/5 t
s M
g{ USTJRC Research Proritizaton 'U 1/28/88 NN
TRIAL PROGRAM PROVIDED OPPORTUNITY TO i
EVALUATE THE RESULTS l
d
SUMMARY
OF ACTIVITY i
i
- HIGHLIGHTS IMPORTANT CONSIDERATIONS OF CRITERIA t
t i
AND RANKINGS l
SUMMARY
OF RESULTS OF EVALUATION
- ACTIVITIES VERSUS SAFETY QUESTIONS l
0
- ACTIVITIES VERSUS CRITERIA i
j O
C-16 n
_L De prepared by Soentech, k g ustmc nescarch Prontizaton g
a
i i
i
.i ONE PAGE
SUMMARY
OF ACTIVITIES PROVIDES BASIC RATIONALE FOR l
EVALUATION j
I
[
SUPPORTS RANKING PROCESS I
f EFFECTIVE DOCUMENTATION t,
PROVIDES OPPORTUNITY TO REVIEW AND REVISE EVALUATION I
EASY TO UNDERSTAND 1
i i,
h C-17 O
I I
k W
UStJRC Research Prontizaton 5
l g_
pepa ed by Soentech. Inc.
m c-i:
SUMMARY
OF ACTIVITIES VERSUS~ SAFETY QUESTIONS i
PROVIDES FOCUSED VIEW OF RESEARCH PROGRAM 1
i l
IDENTIFIES THE EFFECTIVENESS OF THE PROGRAM IN ADDRESSING SAFETY QUESTIONS L
IDENTIFIES THE ACTIVITIES FOR WHICH THERE ARE NO SAFETY QUESTIONS i
IDENTIFIES THE SAFETY QUESTIONS FOR WHICH THERE ARE NO ACTIVITIES l
I l
c-IB
.o
$\\
USNRC Research Proritiraton $ 1/28/88 _
NN ** *
~
SUMMARY
OF ACTIVITIES VERSUS CRITERIA l!
PROVIDES INFORMATION TO SENIOR MANAGEMENT
~
FOR PRIORITIZATION l
"CONSUMER GUIDE APPROACH" s
i CONCISE o
. DIRECT COMMUNICATES t
Il RESULTS
- SELF-EXPLANATORY
{
[
- FAMILIAR FORMAT I!
l!
il 9Il C-/9 l
it 4
{ USNRC Research Prordtraton$ 1/28/88 -
~
- ~ ~
_i _ Z 1 _Z -__ Z r_ z - - ~~ --- -
1-~-~~~
~ ~ - ~~~ Z i!
INSIGHTS INTO RESULTS OF TR!AL PROCESS CRITERIA PROVIDED GOOD DISCRIMINATION OF ACTIVITIES l'
- IDENTIFY CRITICAL SAFETY ISSUES AND AGENCY NEEDS
- ENABLE CONSISTENT EVALUATION OF PROGRAMS
- EASY TO COMMUNICATE s
- FACILITATE IDENTIFICATION bF CONSENSUS AND DISAGREEMENT l
L
>n C-Zo D-7 O
c USTJRC Hesearch Prordizaton g
prepared by scoreech, h
?2.?-
-=
INSIGHTS INTO RESULTS OF TRIAL PROCESS CONTINUED
[
PANEL OF EXPERTS INTERACTION VERY IMPORTANT
- MULTI-DISCIPLINARY 4
JUDGEMENTS l
- EXPERIENCE 1
l
- BIAS PROCESS IS FLEXIBLE
- ACCOMODATES REVISION FOR PROGRAM EXPANSION OR CONTRACTION
- ACCOMODATES AGENCY REDIRECTION FOR POLICY OR C-ZI RESOURCES o
- s N
asa.
j ustmc nesearch erorsizaton ;,,
pr par.d w sc=ww.ch. inc.
..s.
t
SUMMARY
OF ACTIVITY SEISMIC AND FIRE PROTECTION EARTH SCIENCES The Earth Sciences Research Activity provides the basic data required to understand the frequency and severity of earthquakes in different areas of the country. It consists of geological / geophysical studies, efforts to revise Appendix A to 10 CFR 100, and installation of the National Seismographic Network in cooperation with the US Geological Society to repiace the current seismic networks. Several programs are in place to advance understanding of the frequency, transmission, and effect of seismic events.
Safety Assuranca - The Earth Sciences programs are relevant to safety assurance.
Significant work in this area has been going on for more than a decade. These programs have had major effects on nuclear power plants. The current program is designed to supplement the existing data and to answer additional questions conceming the seismological risk to nuclear facilities (R-1). The ris'K significance of these programs is highly uncertain, with the general consensus being that there are significant safety margins in current methods of seismic design. Thus, these programs probably will not have a major safety impact on licensed facilities. Although the current body of seismic information probably is sufficient for characterizing the seismic risk at existing sites, these programs should piovide a better basis for such characterizations if additional seismic risk analyses are to be required in the future. The Earth Sciences Activity will advance the state of knowledge of seismic risk, but the extent and nature of NRC's future role in this area is uncertain.
Usefulness - These programs have been highly useful to the NRC. They have been very well coordinated with NRR through weekly staff discussions and have been directly used in making regulatory decisions. The programs have been well coordinated with USGS, the Corps of Engineers, and foreign govemment # forts. The program managers have succeeded in focusing their program on NRR needs and have been responsive to requests to address licensing problems.
Acorocriateness - The objective of the Activity is to advance understanding of seismic risk, even though much is already known. The USGS is assuming Federal government responsibility for the new seismic networks, and NRC is in the process of phasing out of its supportive role in that area. Utilities and EPRI have seismic monitoring programs at individual sites. It is questionable whether NRC should continue to fund significant research in this area. However, NRC must keep abreast of improving knowledge developed by the scientific community in this area and adjust its requirements and practices as necessary to reflect contemporary knowledge and residual uncertainties.
Resources - The Earth Sciences Activity has required a high expenditure of resources.
About $50 million have been spent to date, and another $20 million will be spent by NRC in repiacing the network and handing it over to USGS. The FY 1988 budget request is for more than $4 million.
1 l
f 1
C-22; L
d l
l
.e
.e a.
5 mi m
m e
+
~ =.
m.
i i3 E.
E.'
E RESEARCH ACTIVITIES E
1:
.g g
a e
o t
y
=.
s.
m, m.s 2
t 3e
.f es o
=a e
c c
3 l1 25 s
ii
"_!! )j jf
?
ui x2 NRC RESEARCH QUESTIONS
~
b
>fI i E eT o es or o_
o>
w o
Wu V.1: Measure op erstlenal readines s Z
i m
V.2: Type of reliablerty assurance progra m D,
e 89 v.3: Me a sure performa nce trenssingt offsettuness Y.4: Set.mie espebility of e.f ety equimment V 8: Me86vre sgtng et safety tempenema, system.,.tructures et O
V 4: Oversome ladiviewel plant unas s ept a tie vulnera bi stles M....,.............,=..............
.O..O.._.
-i v.7: Au e ia..
.ui.e.
u.a.intyrus i s.amnm.nt t-Y.4: Acceptable sceldent s e p a bilttylle s sendenser V 9: Overcome Indiviewal plant e he rt term c e nt. f anvr e V.10: Aseldent m gtla n aly s is,t r aining,p roc edu res
..,=...,.......,.,e.,..,,..,,.............
.o-~..
O.
V.,
. eve e accl.ent eme ne R is evememe
..,,...,..,e.......,e..................
.Q.
V.
, e n. ts.
..e s y he at mov.1
..,,...,,..,........,o.,.,...............
.G... Q..O. O.O.
.Q.
Y.
~e e...n ehen..e n rs
..,.........,e V.
..a swre een.t est.n.
- t. ~no e..
..,...,s
..,.,,e..,,,...,...,..............
V.
..nt s te I t.
a<
..ev.e.. p.s.
V.i t: Hostile thre eVprev entallen et f acillues
. ht: T.M response of Ra te semples evente wu
=
5:: usium
.oh.vi.,
in,i.nt.avi,.n me nt n.v..
=
.O.
E l 3: g ela nc e betwun S.A. p re ve nuen, m g t, mitig ation g
..S.O.
.O.
.Q.O..O. _S.
i: e.m p m e ne ur,,um.u,e n.ne e
.f enA.
g....,,.....,e.
.,s.
.. Im eve m,
.stu sent i, Al smpreu Ac in.eeeuen i.,..., o per, ufeiy
. i.e :
O b7: Im p ro v e/im ple me nt safety goale y.................................
os: nun.no e.nu.venen fu s. Ace
>g g
40: Sewese term sensideratione for seeign, seting, emers. plan 2
* = = = = = = = = = = = = = = = = = * * * * * * = = = = = = =
=
=
- ===
==*-
=
.O.
.O.
.8.
ble: Capablety for Matt irs, large dry sent.
l
. conges in.es ~e,e,.uens is a o,es, m.
bit: Improve regulatory structure bia: Ne w t eennisal regwiresients 814: Saale for e:Isting sleense er ens.ene b18: HiglVlow le vel waste s af ety previsie n s i-16: Nwe6 ear f eellity f ec o mmis sio ning
.R.1 :.N.we. lea.r.f e.elu.ty. s.e is.m ie.ha.a s.t e................
wg
............. e. f e, m e..
- n. : ossiguepe,enen enu airs
<z R 3: Redwee ALARA level. la fuel eyete C
.R-4:
R olla ble desimetry tec hniques y
n.s: upturoi. iul. nota en u,uw,e i. usi.
,ei....
g A-4: Sest toenmques te measure /sentrol reciesetive omleslene O-A.7: Ce84 eff eeUve mRigallen of nws. lee. e nviro nm ental depast p
W a e-.- e--
o -__
1 C-23
F 7
.... r o.
c4 U
E i
2 et
.s 8
g :-
n4
{3 a,3 8-a ia
&8 3
w u2 12h" y
1 3
?
3-1.{3la 3
?
8 31 i
i ai>5 as
.e a
73 33 a3 P 3
$. $. 3 0 k.
w" 6
d, '
-t zy
{}
jo g)=
s3 5 3 4
-i i.t t-s 8
!i es i
is 18 il II
- r. !
j r egi } j} 2 8
3 11 el_
1 ga
$1 la 3
Is*i lE
~
fr
]
$* 1
.i i.i.
1 e
N N2,N2 9
5 "a a
2 y=
4,xss g
g
>J De
.N' Nd3 y
>k g
>34>
i e
f,-
f.-
y v?. v isdes;,
C
>a
>a
>>2
>>ags;S 2
]y 9
e, N
N e
n n
g o
8E 3
.,N 2
8 3
=
K 9
3 5
! ifs
% bb
& 1b b'
~
e e
e e
e e
e e
L <
2 000 3
- i!l a
m3a 8
=
3
=sa
.5 000 e
e o
e O
e e
O 3:
E
=
g 5
E 9
- 5.& g 2 =
r 52 8
>=
s eeO 8
O e
o e
5 e o
e e
m
=
s I
f 3
3 l
~
W 4
- e l
f b
e 2
G C
N8 C
3 A
C 1
s c-3 o
o U
i h
k2h 2
6e
. g p,g 4
d
-1
.W *e g 2
g.
I 5
O
- Cf W
- 5 3
3
)
d$
b 5
b5 jS o
o o
e 4
~
e
~
e W
W Q
d d
d d
d d
c3 C-29
.