ML20155G278
| ML20155G278 | |
| Person / Time | |
|---|---|
| Issue date: | 11/02/1998 |
| From: | NRC COMMISSION (OCM) |
| To: | |
| References | |
| REF-10CFR9.7 FACA, NUDOCS 9811090004 | |
| Download: ML20155G278 (123) | |
Text
.._.
nr p.:,e l
V IT i O !!'./ _
UNITED STATES. OF AMERICA NUCLEAR REGULATORY COMMISSION l
l
Title:
BRIEFING ON REACTOR OVERSIGHT PROCESS IMPROVEMENTS PUBLIC MEETING Location:
Rockville, Maryland Date:
Monday, November 2,1998 Pages:
1 - 123 0
I h
l l
l t( l' 3
ANN RILEY & ASSOCIATES, LTD.
i
-n4 1025 Connecticut Avenue,NW, Suite 1014 Washington, D.C.20036 (202) 842-0034 i
9811090004 981102
^
7 PDR
.. _ = -. -
DISCLAIMER This is an unofficial transcript of a meeting of E..
the United States Nuclear Regulatory Commission held on November 2, 1998, in the Commission's office at One White l
Flint North, Rockville, Maryland.
The meeting was open to public attendance and observation.
This transcript has not been reviewed, corrected or edited, and it may contain inaccuracies.
The transcript is intended solely for general informational purposes.
As provided by 10 CFR 9.103, it is not part of the formal or informal record of decision of the matters discussed.
Expressions of opinion in this transcript do not necessarily reflect final determination or beliefs.
No pleading or other paper may be filed with the Commission in any proceeding as the result of, or addressed to, any statement or argument contained herein, except as the Commission may authorize.
e t
[
- ~. -. - -
1 1
UNITED STATES OF AMERICA 2
NUCLEAR REGULATORY COMMISSION 3
4 BRIEFING ON 5
REACTOR OVERSIGHT PROCESS IMPROVEMENTS 6
7 PUBLIC MEETING 8
9 10 Nuclear Regulatory Commission 11 Commission Hearing Room 12 11555 Rockville Pike 13 Rockville, Maryland 14 15 Monday, November 2, 1998 16 17 The Commission met'in open session, pursuant to 18 notice, at 2:07p.m., the Honorable Shirley A. Jackson, 19 Chairman, presiding.
20 21 COMMISSIONERS PRESENT:
22 SHIRLEY A. JACKSON, Chairman of the Commission 23 NILS J. DIAZ, Commissioner 24 EDWARD McGAFFIGAN, JR.,
Commissioner 25 JEFFREY S. MERRIFIELD, Commissioner ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut' Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
2 1
. STAFF AND PRESENTERS SEATED AT COMMISSION TABLE:
2-JOHN C. HOYLE, Secretary of the Commission 3
KAREN D. CYR,. General Counsel 4
WILLIAM D. TRAVERS, EDO 5
FRANK GILLESPIE, NRR
' _7 MICHAEL JOHNSON, NRR
.8 PATRICK BARANOWSKY, AEOD 9
JAMES LIEBERMAN, Office of Enforcement
'10 BRUCE MALLETT, Div of Reactor Safety, 11 Region II 12 JOHN FLACK, RES 13 RALPH BEEDLE, NEI 14 DAVID LOCHBAUM, UCS 15 16 17 18
'19 20 e
'21 22 23 24 25 ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
3 1
PROCEEDINGS 2
(2:07 p.m.]
3 CHAIRMAN JACKSON:
Good afternoon, everyone.
I am 4
pleased to welcome members of the NRC staff to brief the 5
Commission on the progress of planned improvements to the 6
reactor oversight process and plans for and results of an 7
initiative to improve the NRC assessment, inspection and 8
enforcement processes for operating commercial nuclear 9
reactors.
10 Before we begin, however, I would like to take a 11 moment to recognize the return of Ms. Greta Dicus to the 12 Commission.
While she could not be with us today, we do 13 welcome her back.
She was missed this summer.
14 In addition, I would like to recognize and to 15 welcome and to introduce to you Mr. Jeffrey Merrifield to 16 his first Commission meeting.
Commissioner Merrifield, my 17 colleagues and I look forward to working with you.
We have 18 a lot to do, as you will get an inkling of this afternoon.
19 Today's meeting represents a continuation of a 20 dialogue which has existed between the Commission and the 21 NRC staff since 1996 when, due to concerns over the 22 subjectivity involved in the senior management meeting 23 process, I directed the staff's attention toward seeking an 24 external review of that process, which was the Arthur 25 Andersen study.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
- - -. _ ~
. - -.- - -.~
4 1
Since that time and pursuant to Commission 2
direction, the staff has developed proposals to modify and 3
to improve the entire power reactor regulatory oversight 4
process.
Not only the senior management meeting process, 5
but power reactor performance assessment, which includes all 6
of the constituent pieces, including SALP, as well as the 7
inspection and enforcement processes.
8 The reactor oversight process is intended to 9
independently assess reactor plant performance, to 10 facilitate the early identification of plants which require 11 increased regulatory attention, and to direct regulatory l
12 actions towards such plants before the reasonable assurance 13 of public health and safety is compromised.
14 Our ultimate goal is to attain a clear, coherent 15 picture of performance at operating reactor facilities in a 16 way that leads to objective, consistent and predictable 17 regulatory actions.
Through the reduction of subjectivity 18 that can be afforded by the use of performance indicators 19 and through the use of risk information, it is our attention i
20 to reduce unnecessary regulatory burden to the extent l-l 21 possible.
l l
22 The staff has quite properly considered the 23 individual components of the reactor oversight process as an 24 integrated whole in which components of the process work 25 synergistically to achieve our objectives.
Today the staff ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
_~.. _ _. _ _
t i
l 5
1 will describe its current activities to support these 2
objectives and also should describe any incremental 3
improvements to the process that already have been 4
accomplished.
5 We welcome this update which represents an amalgam 6
of both staff and stakeholder thoughts on the subjects 7
obtained through a number of NRC-stakeholder interactions, 8
culminated in a well attended and, I am told, fruitful l
9 workshop conducted during the week of September 28.
The
~10 workshop was sponsored by the NRC and was attended by 11 numerous representatives of the NRC, licensees, the power 12 reactor industry, public interest groups, and congressional 13 staff.
14 The Commission applauds the cooperative efforts of 15 all involved at the workshop.
16 At the conclusion of the staff's presentations, 17 two stakeholders will provide brief remarks on the NRC 18 efforts concerning the assessment process.
To represent the 19 Nuclear Energy Institute (NEI), Mr. Ralph Beedle will 20 present remarks.
To represent the Union of Concerned 21 Scientists, Mr. David Lochbaum will provide remarks.
And I 22 will call them to the table at the appropriate time.
23 Copies of the slide presentation are available at 24 the entrances to the meeting.
So unless my colleagues have 25 any introductory comments, Dr. Travers, please proceed.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
6 1
MR. TRAVERS:
Good afternoon, Chairman Jackson, 2
Commissioner Diaz, Commissioner McGaffigan, Commissioner 3
Merrifield.
As the chairman mentioned, we are here today to 4
discuss the status of the staff's efforts to develop 5
improvements in the NRC's inspection, assessment and 6
enforcement processes.
7 With me here at the table are Sam Collins, Frank 8
Gillespie, and Mike Johnson of NRR; Pat Baranowsky of AEOD; 9
Jim Lieberman of the Office of Enforcement; John Flack of 10 the. Office of Research; and, of course, Bruce Mallett from 11 Region II.
12 You mentioned an integrated whole, Chairman.
I 13 think the example that I would offer you is the kind of 14 cooperative effort that we have internally put together to 15 work on these processes, as demonstrated by the different 16 organizations, regions, and major program offices that are 17 involved in this effort.
18 Since receiving the initial tasking memorandum 19 from Chairman Jackson on August'7th, the staff has given a 20 high priority to furthering changes that are intended to 21 better utilize risk information, clarify NRC requirements 22 and expectations, and improve the predictability, 23 objectivity and timeliness of NRC decisions.
24 Particular emphasis has been placed on addressing 25 specific aspects of the reactor oversight program.
The ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 L
7 l
I 1
EDO's August 25th memorandum to the Chairman provided the 1
2 staff's short-and long-term plans, including detailed 3
milestones and deliverables for a number of the most 4
important issues.
5 In the two months since the initial response, the 6
staff has increased its level of effort in order to 7
accelerate ongoing improvements in the performance 8
assessment, inspection and enforcement programs.
We remain 9
substantially on track in our efforts.
10 As you know, on September 15th the Commission 11 approved the suspension of the systematic assessment of 12 licensee, or SALP, program, for an interim period until the 13 staff completes a review of its process for assessing 14 licensee performance.
15 The suspension of SALP has freed staff resources 16 to work on this project, and, as a result, region based 17 managers and inspectors have been able to be assigned as 18 dedicated members on each of the teams assigned to develop 19 the technical framework, inspection and assessment models.
20 This is truly an integrated effort.
21 There has been a significant amount of interaction 22 with stakeholders for this effort.
Chairman, you mentioned 23 the workshop.
At that workshop we were able to achieve our 24 goals for the workshop by obtaining alignment of the 25 participants on the basic framework for the process and its ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
8 1
defining principles.
2 Although significant progress has been made, it's 3
really just the first step and a significant amount of work 4
remains to address the details.
Between now and the middle 5
of December there are meetings scheduled with stakeholders 6
at the working group level nearly every week to continue to 7
refine and add to the progress we have already made.
We 8
view these continued interactions with stakeholders as a 9
critical factor in develcping an acceptable overall 10 inspection, assessment and enforcement framework, and these 11 interactions will continue to be a priority for us.
12 As you know, we are working to provide the 13 Commission with the results of our work, including a staff 14 recommendation, by January.
15 At this point, I would like to turn it over to 16 Frank Gillespie, who is going to begin the process of 17 describing what we have been up to.
18
[ Slides shown.]
19 MR. GILLESPIE:
Good afternoon.
We are here this 20 afternoon to present to the Commission a brief background 21 review of the oversight process improvement effort completed 22 to date, a status of current staff activities, near term 23 goals, and to discuss long-term activities required to 24 implement process improvements.
25 While these efforts were originally focused on ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
1 9
1 improvsm:nts to the assessment process, the task has evolved L
2 to a more broadly based effort involving the close 1
3 integration of inspection, assessment and enforcement.
4 In addition, there are several other process ties 5
to these efforts which have been recognized, such as the l
6 allegation process, licensee reporting process, and 7
risk-informed regulation.
8 We last briefed the Commission on April 2nd on the 9
staff proposal which resulted from the integrated review of 10 the.nssessment procens (IRAP) effort.
The objective of the 11 IRAP review was to develop a single integrated assessment 12 process which provided greater-objectivity, predictability, 13 and scrutability.
14 The fundamental concepts which formed the basis of-15 the IRAP proposal were that:
16 Inspection results provided the basis for the 17 assessment.
18 Inspection findings would be categorized by 19 performance template areas.
Scored based on safety 20'
. significance, assessment would be accomplished by totalling
~21 the scores in each template area and comparing them against.
e 22 thresholds; and NRC actions would be taken based on a 23 decision model.
24 Since the submittal of the IRAP proposal in 25 SECY-98045, the staff has received feedback on a proposal ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
e 10 1
from the ACRS and the Commission.
2 In a letter to the Commission dated March 13th 3
ACRS recommended the staff take a top-down approach to 4
developing improvements in the assessment process.
5 In a staff requirements nemorandum dated June 30th 6
the Commission expressed concerns with the use of 7
enforcement as a driving torce for the assessment process, 8
the quantitative scoring of PIM entries, and the use of 9
color coding for performance ratings.
Horever, the 10 Commission did approve the solicitation of public comments 11 on the IRAP proposal.
12 In parallel with the development and consideration 13 of the IRAP proposal, the industry developed an independent 14 proposal for improvement of the assessment process.
This 15 effort, led and coordinated by the Nuclear Energy Institute, 16 resulted in a proposal that was fundamentally and 17 philosophically different from the IRAP proposal.
18 This proposal took a top-down approach and 19 established tiers of licensee performance based on 20 maintaining barriers to radionuclide release, minimizing 21 events that could challenge these barriers, and ensuring 22 that systems can perform their intended function.
23 Performance in these tiers would be measured through 24 reliance on high level objective indicators with thresholds 25 set for each indicator to form a utility response band, a ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
11 l
1 regulatory response band, and a band of unacceptable 2
performance.
3 So in response to the IRAP SRM, the NEI proposal, 4
input from July 17th Commission meeting with public and l
5 industry stakeholders, and the July 31st hearing before the 1
6 Senate, the staff set out to develop a single recommendation 7
for improvement to the regulatory oversight process which 8
places an appropriate regulatory burden on licensees.
9 This recommendation is intended to preserve the 10 core values of regulatory oversight which are to carry out 11 the agency's mission of protection'of the health and safety 12 of the public and to do this in a risk-informed and 13 performance-based manner, and to account for the NRC's 14 principles of good regulation:
independence, openness, 15 efficiency, clarity, reliability.
16 This recommendation should further reduce the 17 burden for good performing plants but retain the ability to 18 provide a strong focus on those licensees with significant 19 performance problems.
20 The approach taken by the staff to develop a 21 framework for regulatory oversight which uses a top-down 22 approach.
The staff started with the mission of the agency 23 and then worked down to identify those cornerstone areas 24 which provide the foundation for meeting our mission.
The 25 staff then identified and addressed those key issues which l
I ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
12 1
1
' form the defining principles to be used in the redesign of 2
the regulatory oversight process.
3 For the cornerstones of safety the staff is 4
. applying a' set of defining principles and a risk-informed, 5
performance-based perspective to identify what is important 6
to measure'in each cornerstone and how it can be measured.
t 7-During this process the staff identified important ties to 8
other. key processes such as enforcement, allegations,
.9 licensing, which should be addressed in the oversight 10 framework.
11 CHAIRMAN JACKSON:
Frank, could you go back to 4.
12 Have you had any discussions about the role or continuing 13 role of what have been the elements of the oversight and 14 assessment process, namely,.SALP, PPR, SMM?
Have you come 15 to any discussion about whether they would be retained or t
16 retained in their current form, or is that premature at this 17 stage of the game?
18 MR. GILLESPIE:
I think it would be premature to 19 give you details,'but if I could take you back to one of our 20 first presentations on the key attributes, positive and 21 negative, of those processes, some of those positive key 22 attributes definitely are going.to affect our assessment 23 process group and how we interface, for example, with the 24 public.
Public meetings of some frequency appear to be an 25 important public attribute that we want to retain.
This ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
.~.
13 1
also cams up when we suspended SALP.
So there are some important' positive attributes to what we were doing which we 2
3 would intend to factor into how we carry out this process.
4 While working through the process to develop the 5
cornerstones to regulatory oversight, the staff recognized 6
the importance of both internal.ad external input.
As 7
directed by the IRAP SRM, a 60 day comment period on the 8
IRAP and cornerstone concept was completed on October 6.
9 The staff received 26 submittals in response to the public 10 comment to the Federal Register notice and is reviewing and 11 evaluating them during the continuing development process.
12 There have been numerous public meetings with the
'13 industry, ACRS, regional, headquarters staff, to obtain 14 feedback on development of the cornerstone concepts.
These 15 interactions with both internal and external stakeholders i
16 are continuing throughout the development of recommendations 17 for improvement in the oversight process.
18 A four day workshop, as was mentioned, was 19 conducted by the staff on September 28 through October 1 to 20 enable the staff to interact with industry, the public, and 21 the NRC itself, our own staff, to obtain and evaluate input 22 on improvements to the regulatory oversight process.
23 Over 300 people attended the workshop, with broad 24 participation from the NRC headquarters and regional staffs, 25 individual licensee representatives, INPO, NEI, and ANN RILEY & ASSOCIATES, LTD.
Court Reporters J
1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C.
20036 l
(202) 842-0034 I
- - ~
14 1
participation from the Union of Concerned Scientists, GAO, 2
state regulatory. agencies, foreign regulatory agencies, the 3
Office.of the Inspector General, and Senate staff.
4 There were several significant accomplishments 5
achieved at the workshop which have contributed to the 6
continued development of improvements to_the regulatory 7
oversight process.
8 I use the next words very carefully and very 9
deliberately, because there is a scaling, as you will see in 10 this viewgraph.
11 Workshop consensus was reached on the overall 12 framework for regulatory oversight and the objective 13 definitions for each cornerstone of safety.
14 Good alignment was achieved on the defining
~
15 principles for the oversight process, with two significant 16 issues remaining open:
the integration of data and the 17 nature of the data reporting program.
These issues will be 18 further discussed later.
19 CHAIRMAN JACKSON:
Have you had any discussions at 20 all or any preliminary discussions with INPO'about data that 21 INPO collects?
22 MR. GILLESPIE:
Not to date, but it was one of the 23 agenda items that we just put in.
There is an INPO senior 24 management meeting, I think next week, and this is one of 25 the agenda items that we are suggesting.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
-- ~. -. _ -
15
.1 Finally, significant progress was achieved in 2
developing a process for selecting performance indicators, 3
thresholds, and for identifying necessary inspection areas 4
for each cornerstone.
5 CHAIRMAN JACKSON:
Maybe you had better back up 6
for a minute.
Can you be a little bit more explicit?
I 7
-think we all know what consensus is.
Tell us about i
8 alignment and progress.
I'm sure we are going to hear from' i
9 our'other stakeholders.
I'm interested in their perceptions 10 about where we are in those areas.
11 MR. GILLESPIE:
We had set ourselves some 12 objectives before the meeting.
Our main objective was to 13 try to develop a consensus on the framework, the highest 14 level.
The framework evolved and actually changed at the 15 meeting.
The picture, if you would, looks different today 16 than it did before the meeting, and that has evolved.
17 There is a general agreement on what those 18 cornerstones are, and more importantly, a very intense i
19 discussion on the objectives that are attached with each 20 cornerstone and what it means.
This became very, very 21
'important because it takes you to the next step.
If you 22 have.your objectives stated, what information is needed to 23 say that you have reasonable assurance the objectives is 24 met?
25 Going down to alignment, our next desire was to L
i i
ANN RILEY & ASSOCIATES, LTD.
Court Reporters l
1025 Connecticut Avenue, NW, Suite 1014 L
Washington, D.C.
20036 l
(202) 842-0034
j 16 l
1 achieve alignment.
You might say that in the simplest sense 2
this was a broad majority.
In trying to come up in a 3
plenary session where we summarized each of the working 4
session findings, this in general would represent about an 5
80 percent sense of alignment.
We stated on the first day 6
of the meeting that that is about what we were trying to get 7
on this topic.
This was an area where it was clear that in 8
a four day workshop you weren't going to achieve the 9
perfection.
10 The defining principles are the basic sNructure 11 that set the stage for how inspection integrates with 12 assessment, assessment interacts with enforcement, and sets 13 the stage for the basic assumption that you in fact believe 14 you can set thresholds and have performance indicators.
So 15 alignment there.
16 Progress was made in discussions -- and now we 17 were on about the last two days of the workshop -- on what 18 are the performance indicators.
Generally they are 19 perceived to be more than what is in the INPO indicators or 20 what are indicators today, but clearly some of those are 21 included in the indicators:
Where will we inspect?
How 22 much will we inspect?
That clearly did not get decided, but 23 we made significant progress, and we will be going through 24 that a little later on.
25 What are the kinds of things we should inspect?
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
.- ~ -.
1 17 I
1 One of the rules we had coming out of this was there is a 2
minimum risk-informed baseline' inspection that would be done 3
everyplace.
4 The last one, how do you select a threshold?
What 5
is the logic you use?
Is it risk-informed?
There are two 6
thresholds in each indicator.
There is a threshold which is 7
the operating threshold at the top level, where you would 8
start to get the regulator to.come in and start to take a 9
gradually increasing action.
Then there is a threshold as 10 might be represented in the NEI paper, a threshold of
.l 11 shutdown, that ultimate regulatory threshold.
l
- 12 The focus of this meeting was really on more time 13 was spent on the upper threshold than on the lower
- 14 threshold.
. 15 COMMISSIONER McGAFFIGAN:
On the thresholds, I'm 16 trying to understand the concept.
We're going to have 17 multiple items that get graded, as I understand the NEI 18 scheme, green, white, red, in utility response space,
. 19 regulatory response space, unacceptable space.
How do you
. 20 integrate all of the indicators into an overall green?
21.
Is it fair for us, if' a plant is sailing along in 22 green in most indicators, but, using SALP terms, not doing 23 so well in engineering, for us to take regulatory action in 24 the engineering area or the operational area.
25 I know that isn't the concept anymore; you break 4
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 y
e
7 18 1
these things down differently, t
2 Have you talked through in practice how this thing 3
works and where the regulatory threshold is?
4 MR. GILLESPIE:
Mike is going to cover that a 5
little bit in assessment.
I'm really trying very hard not 6
to prejudge where the teams are going to come out.
7 COMMISSIONER McGAFFIGAN:
My concern, and I'll say 8
it to the stakeholders as well, is the idea I think sort of 9
embedded in a lot of our thinking is that this thing is 10 going to be ready to go in January.
That's about two months 11 from now.
If it's going to be ready to go in January or 12 February or March, you'd already be wanting to train the 13 people out there in the field, right?
14 CHAIRMAN JACKSON:
There is a schedule in here.
15 MR. GILLESPIE:
I'm going to go over the schedule.
16 This is a very, very important point on expectations.
Each 17 task member is going to cover what we hope to have and how 18 far we have gotten to deliver in January.
If you look way,
'9 way ahead to the schedule, our January deliverable is the 20 concepts that if applied to the inspection program, if would allow us then to 21 applied to the assessment program, move forward and then rewrite the inspection procedures,and 22 then write the procedures on how we are going to do 23 24 assessment.
So everything will not be done by January 25 relative to implementation.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C.
20036 (202) 842-0034
19 1
CHAIRMAN JACKSON:
Actually, his question segues 2
into a question that I have.
Do you have other such 3
workshops planned and scheduled that are as robust as the 4
one that happened the week of September the 28th?
5 MR. GILLESPIE:
There are none planned right now.
6 There are some under discussion.
7 CHAIRMAN JACKSON:
I think that may be a good 8
thing given the Commissioner's question.
It's something I o
9 think you ought to think about.
10 COMMISSIONER McGAFFIGAN:
I'just want to 11 understand where we are going to be in fiscal 1999.
We are 12 going to be trying to develop this process -- I did glance 13 ahead at the viewgraph -- but very little of this w: 11 14 actually be being practiced in 1999.
So next spring
..'e will 15 have the typical senior management meeting, the roll-up to 16 the senior management meeting.
You guys will do whatever 17 you do.
Is that the thought?
18 MR. GILLESPIE:
Yes.
This April there will be the 19 typical senior management meeting, and we are continuing on 20 through the process we currently have in place.
The 21 inspection portion would be implemented between January and 22 October, and the assessment process would be going out to 23 June of 2000.
24 I'd like to actually leave until Pat talks.
Pat 25 is going to talk where he thinks he will be relative to l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
20 1
performance indicators.
Otherwise, I will end up giving --
2 CHAIRMAN JACKSON:
Stealing his thunder.
3 MR. GILLESPIE:
Yes.
4 CHAIRMAN JACKSON:
They have it all worked out.
5 MR. GILLESPIE:
Then we can come back.
I think l
6 expectations need to be honest and up front.
7 CHAIRMAN JACKSON:
Okay.
8 MR. GILLESPIE:
Going on to slide 7, which is our 9
outline, as previously stated, we feel that we have good
~10 external and internal stakeholder consensus on the framework 11 for an improved regulatory oversight process.
12 This framework was developed using a top-down 13 approach.
It starts at the highest level, with the NRC's 14 overall mission.
This mission statement is based on the 15 Atomic Energy Act of 1954, as amended, the Energy 16 Reorganization Act of 1974, as amended.
The mission of the 17 NRC as it applies to commercial nuclear power plants is to 18 ensure that these facilities are operated in a manner that 19 provides reasonable assurance of adequate protection of 20 public health and framework and the environment, protects 21 against radiological sabotage, and theft and diversion of 22 special nuclear materials.
23 The mission of protecting the public health and 24 safety is a responsibility that we also share with the 25 licensees.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
21 1
Given this mission, the,next step was to identify 2
those aspects of licensee performance that are important and p
3 therefore merit regulatory oversight.
4 The NRC's strategic plan identifies the 5
performance goals to meet for nuclear reactor safety and 6
includes:
7 Maintain a low frequency of events that could lead 8
to a reactor accident.
9 Zero-significant radiation exposures resulting 10 from civilian nuclear reactors.
11 No increase in the number of offsite releases of 12 radioactive materials for civilian nuclear reactors that 13 exceed 10 CFR Part 20.
14 No substantial breakdown of physical protection 15 that significantly weakens protection against radiological 16 sabotage or theft or diversion of nuclear materials.
17 These performance goals reflect those areas of 18 licensee performance for which the NRC has regulatory 19 responsibility in support of our overall agency mission.
20 These performance goals were represented in the framework 21 structure as the strategic performance areas of reactor 22 safety, radiation safety, and safeguards, and form the 23 second level of the framework.
24 For each of those strategic performance areas 25 there are many regulatory requirements.
However, with a t -
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 i
Washington, D.C.
20036 l
(202) 842-0034
22 1
risk-informed perspective, it was possible to identify those 2
most important elements in each strategic performance area 3
which formed the foundation for meeting the overall agency 4
mission.
5 These elements are identified as a cornerstone 6
from the third level of the framework.
As an example, the 7
objective of initiating events cornerstone is limit the 8
frequency of events that upset plant equilibrium and 9
challenge critical safety functions.
10 Acceptable licensee performance in this and other 11 cornerstones should provide reasonable assurance that the 12 overall mission of adequate protection of the public health 13 and safety is meet.
14 The cornerstones provide the fundamental building 15 blocks for regulatory oversight process and provide 16 reasonable assurance that the overall safety mission is met.
17 However, there are other aspects of licensee performance 18 such a human performance, safety conscious work environment, 19 problem identification resolution, which are not captured as 20 cornerstones but are equally important to meeting our safety 21 mission.
4 22 The staff concluded that these items and others 23 generally crosscut the affected areas and manifest 24 themselves as causes of performance problems.
25 Licensee performance in these crosscutting areas ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
1 23 1
should therefore be dealt with in each of the cornerstone I
2 areas as contributors to performance as measured by
(
3 indicators and as observed through inspection.
4 Once the cornerstones and objectives were 5
established, we then had the basis for determining what 6
information was needed to provide reasonable assurance that 7
the objectives were being achieved.
This included what 8
performance attributes are'in each cornerstone, what is 9-important to measure for each attribute, what aspects of'
'10 performance can reasonably be measured with objective 11 indicators, what areas of performance should be measured 12 through inspection, and what are the appropriate thresholds 13 for NRC' interaction.
14 These cornerstones provide the foundation for 15 improvements in inspection, assessment and the enforcement 16 process.
17 Once the framework was established, key issues 18 were discussed and agreed upon which formed the defining 19 principles for regulatory oversight.
These defining 20 principles are essential to the continued development and 21 improvement to the oversight process since they form the 22.
rules against which the cornerstone details will be 23 developed.
24 Further, these defining principles also establish 25 the relationship between elements of the oversight process l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 L
Washington, D.C.
20036 (202) 842-0034
24 i
such as enforcement and inspection.
These defining i
2 principles are:
There will be a risk-informed bareline inspection 3
program that establishes the minimum regulatory interaction 4
5 with licensees.
6 Thresholds can be set for licensee safety 7
performance.
8 Performance indicators, supplemented with some 9
inspection, will form a rebuttable presumption for licensee 10 assessment.
A risk-informed baseline inspection program will 11 12 be performed for all licensees and should cover those 13 risk-significant attributes of licensee performance not 14 adequately covered by performance indicators.
The 15 inspection program will also verify the adequacy of the 16 performance indicators and provide for event response.
17 In most cases, inspection observations are 18 expected to complement the performance indicator results.
19 However, when warranted, risk-significant inspection 20 observations can be used to overturn the indicator results 21 when the inspection observations develop a compelling case 22 that the performance indicators are not accurately 23 reflecting licensee performance.
Enforcement actions taken should not be an input 24 25 to the assessment process.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
25 L
1
' CHAIRMAN JACKSON:
Frank, can you hold up one 2
second..
3 COMMISSIONER DIAZ:
Let's go back to these last 4
two bullets on page Ba.
I.just want to make sure you are 5
going to eventually add'some definition to the term I
6
" adequacy."
It's a broad' term.
I don't know whether you 7
meanLadequacy in the whole context'of what the process is or 8
you are talking about the accuracy of the indicators in 9'
predicting, or all'of the above.
i 10 MR. GILLESPIE:
It's.actually all of the above.
11 Going down to the next, "will verify the adequacy of the 12 performance indicators," which is going to be very 13 important, we are'getting consistent information. reported 14 consistently with the same definitions from all licensees.
15 The other piece, if I can take a simple example of 16 PI,.might be total scrams.
Total scrams reflect operators 17 in a control room and generation operation, how they react 18 to, but it might be that total scrams may not reflect what 19 needs to be inspected, and that's operator reaction during 20 an event.
21 We do get to observe that on a simulator.
So that 22 might suggest that.while the PI as a general oversight does 23
' touch upon human and operator performance during a 24-
' reactivity transient, we have to look at what is the risk 25
. significance of observing operators in an accident situation ANN RILEY & ASSOCIATES, LTD.
Court Reporters
-1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
~
26 1
on the simulator.
So while we may have a PI that has 2
breadth, it may not have enough depth in an area that is l
3' risk significant'.
4 COMMISSIONER DIAZ:
I just wanted to point out 5
that that word " adequacy" is an extremely ~important word.
6 It.is~the definition of adequacy where the process kind of 7
hinges.
So it is something that you might not be able to l
8 address'now,. but the Commission will be looking in January 9
at how adequacy is defined.
10 The other word is " compelling," this compelling 11 case, which is kind of the second' step.
What is a 12 compelling case?
13
-COMMISSIONER MERRIFIELD:
If I can add, what kind
~14
'of burden.
You-are using compelling case.
What kind of 15 burden of proof does that put on the person seeking to sort 16 of overturn the indicator results?
17 MR. GILLESPIE:
The burden is clearly going to be 18 on the staff to overturn the indicator results, not the 19 licensee.
" Compelling" is part of the assessment group that 20 we are still trying to work on.
That is going to be a very 21 interesting definition to develop.
Developing it may be 22
.more interesting than the final definition.
23 COMMISSIONER DIAZ:
Yes.
I just want to point out 24 that those two words need to be well defined and some kind 25-of boundaries put around it so we can actually know what we ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
- - - --~ - ---- - -
27 l
1 are doing.
2 CHAIRMAN JACKSON:
Are you going to hava 3
thresholds for event response?
Have you addressed that at 4
all?
5 MR. GILLESPIE:
Yes, we are going to address it.
6 Pat is working on it.
Chairman Jackson, I think you mean 7
the kind of event that could happen that is a high risk 8
event might trip whatever PIs we have multiple but not push 9
past the threshold.
Clearly we are going to have to have 10 and will have some definition of how we deal with the 11 exception that goes across in a high risk situation like 12 that on multiple items.
13 MR. COLLINS:
Commissioner Diaz, what is unique 14 about the process that we are in, which is a little 15 different than perhaps what we are used to historically, is 16 that we expect not only for the indicators to evolve over 17 time with experience, but we expect as the industry. matures 18 those indicators might actually change over a period of time 19 in response to either aging considerations, license renewal 20 concerns, or other challenges that are brought forth by a 21 collection of data, which might be different than what we 22 are actually measuring today.
23 The second aspect which would allow that to happen 24 is that all the information is shared.
So it is not an 25 instance as perhaps we found ourselves in the past where we ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
28 1
are justifying a SALP assessment.which has broad subjective 2
statements with very little data and criteria.
This 3
information will all be laid out in front of the licensees 4
as well as the staff.
It will be scrutable on a mutual 5
basis.
6 In any case, the words " adequacy" and " compelling" 7
will be the subject of joint considerations as far as what 8
is it going to take to make this process work, both for the 4
9 industry, the NRC, and for the other stakeholders.
So we 10 would expect this development not to end when the process is 11 put in_ place on trial in June of 1999, but for the 12 stakeholder involvement and for the evolution to continue 13 and for it to be a very scrutable process.
14 COMMISSIONER McGAFFIGAN:
I'm a little concerned 15 about managing performance indicators and getting overly 16 dependent on just a bunch of numbers because they happen to 17 be what you can measure.
Maybe this is an area that is 18 covered by one of these categories where PIs are 19 inadequately covered.
20 We constantly run into cable separation issues, 21 fire protection issues at plants.
We went looking at a 22 couple of plants that came out of start-up that had cable 23 separation issues in their cable spreading rooms, and they 24 had to go in and take actions.
I don't know how you have a 25 performance indicator for whether they have adequately ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut' Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
29.
i.
1 handled fire protection.
2 Are these indicators so good that you are working 3
with?
At this point in the-process you should have at least 4
an existence proof.
If you had had these indicators in play, 5
over the last.five years, how good are they predicting NRC 6
regulatory actions?
7 I'm not saying that all of our regulatory actions 8
were perfect and I'm not saying all of our scoring was 9
perfect, but are there SALP 1 plants that are in the red t
10 zone, and are there SALP 3 plants that are firmly at the top li of the green zone.
What do the data tell you when you look 12 at some of these, looking backward at the regulatory actions 13 that we took, whether they were the right actions?
14 MR. GILLESPIE:
You asked two questions.
15 The first one is we are still developing specific l
16 indicators-and thresholds.
So we have not taken an 17 independent retrospective look.
But NEI has provided us 18 with some insights on some work that they did.
So we are 19 going to have to take a retrospective look at these various 20 plants once we get the indicators done.
21 The second question is partially addressed in our 22 thought process on backup slide number 2.
If you look at 23 mitigating systems and you through desired result, important 24 attributes -- this was a straw man we kind of used at the l
25 meeting --
l 1
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 l
30 1
COMMISSIONER McGAFFIGAN:
That's going.to be 2
helpful.
3 MR. GILLESPIE:
Yes.
You're not going to see it 4-up there.
You really have to see it on the paper.
5 What we came'down is a realization that 6
performance indicators were not going to cover all the 7
blocks.
If you look.at adequate controls to maintain plant 8
design, you will see on the left-hand side inspection, 9
design programs.
10 So part of our logic is asking the question, how 11 much information do we need about high risk systems and 12 components relative to mitigation systems, and where can 13 that information be made available?
It is clearly not all 14 going to be available from performance indicators.
15 This is an illustration of design inspection 16 needed, validation of PIs needed, and potentially some work 17 on licensed operator recall program, which is personnel 18 during accidents.
19 COMMISSIONER McGAFFIGAN:
Where does inspecting 20 adequate fire protection come in?
21 MR. GILLESPIE:
In the separation?
22 COMMISSIONER McGAFFIGAN:
Cable separation or 23 other fire issues, where does that come in.
24 MR. GILLESPIE:
Pat.
25 MR. BARANOWSKY:
The way thing like inspecting ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
l 31 1
risk-significant areas that are associated with the fire 2
program would come in as we go'through the indicators and 3
determine what their capabilities are, we are going to 4
identify things that they can't_do, and then the 5
risk-informed inspection program is supposed to be focused 6
on the most significant areas that we would do our audit 7
inspections, either ourselves or oversee what the licensee 8
is doing.
There is not going to be indicator that can find 9
design flaws or tell you what the implication of design 10 flaws is before send an inspection team out there.
I think i
11 this is part of the information that has to be integrated 12 with the indicators in order to' perform an assessment of the 13 licensee's-performance.
14 CHAIRMAN JACKSON:
So you are saying that the 15 inspection program really has two purposes.
One has to do 16
. with what you call the verification or the validation of the 17 PIs, but also separate and apart from that, to get at issues 18 such as what the Commissioner mentioned, that a performance 19 indicator may not so easily pick'up per se.
20 MR. BARANOWSKY:
For instance, if we take the Quad 21 Cities high risk fire situation, the current regulatory 22 program, without assistance from PRA, didn't-find it, and 23 the performance indicators by themselves couldn't indicate 24 it, but it required a risk-informed look in that area to 25 find the issue.
Then we took an appropriate regulatory ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
32 1
response to that situation.
I think we are going to see 2
some of that.
3 COMMISSIONER McGAFFIGAN:
So a possible goal is 4
freeing up resources that at the moment may be focused on 5
less risk-significant things and will allow people to do 6
smart sampling in areas that are more risk significant.
Is 7
that one of the goals?
8 MR. BARANOWSKY:
NRC and licensee resources.
d 9
COMMISSIONER McGAFFIGAN:
You said since you don't 10 have the indicators yet you can run a' truth test on them.
11 The world should know that when we had you all in the senior 12 management meeting process adopt some of the Arthur Andersen 13 indicators, and Arthur Andersen had selective ones in their 14 public report without plant names attributed to them, that 15 the Commission at least had all 105 plants.
This was not 16 something that would have been outside the reach of NEI 17 probably to replicate either except for one of the 18 indicators.
So we could look at hits.
19 The Arthur Andersen model, we could look at it; we 20 could see how well it replicated the past and understand the 21 differences.
And the GAO, of course, with the benefit of 22 also seeing what we had seen, could come in and say, as 23 Arthur Andersen said', that there had been, if anything, 24 according to the Arthur Andersen model, a little bit of a 25 bias to pull our punches.
Plants were getting hits reaching ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
1 33 Andnrsen thresholds before we took action, with only one or 1
2 two in the other extreme where we took action and the hits 3
never got very high.
4 I hope that the_ Commission can have that sort of 5
thing.
In this case, since it's a public process involving 6
the stakeholders, you might as well all be working of the 7
same data sheet when you decide whether this new set of 8
indicators, whichever initial set you use, is a good way to 9
go or not.
I would hope that we would have a backward
'10 looking look at how well these things do.
11 MR. GILLESPIE:
Absolutely.
In fact, this is one 12 of the reasons we feel, I don't want to say c'onfident, but 13 at least comfortable, and I guess I should say confident, 14 that we can develop a set of performance indicators because 15 of the Arthur Andersen and some of the work, without plant 16 names, that NEI has shared with us in'public meetings; that 17 we do have a success path.
But we are going to have to 18 select specific indicators and retrospectively look and test 19 those exact indicators.
20 CHAIRMAN JACKSON:
I had a question somewhere that 21 was going to ask you that question, about how you were going 22 to go about actually testing the indicators.
I think 23 without some algorithm as to how the indicators get folded 24 together to help you reach some decision threshold, you can 25 look at whether they are red, green or blue, or one, two or ANN RILEY & ASSOCIATES, LTD.
7 Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 i
(202) 842-0034 l
l l
34 1
thren until you are blue in the place.
So the real question 2
becomes, how do you actually meld them to make some 3
decisions?
4 MR. COLLINS:
Chairman, I think in fairness to 5
Commissioner McGaffigan's question, the premise about 6
fitting the data over the new system will have to include 7
the understanding that the new system allows for a response 8
mechanism to kick in when the data starts to trend, which 9
our previous process does not.
10 When we overlay data on to the new window system, 11 if you will, it will show trends, but it will not include 12 what we would expect the licensee's reaction to be if that 13 trend is declining and how that would have gone under the 14 new system, and if it continues into the white, because the 15 assumption would be that there is no licensee action because 16 you are overlaying an old system on'to a new system, that 17 the NRC engages, that we engage in a way that creates a 18 turnaround.
So we are going to have the same information 19 overlaid on different processes, and I think we need to be 20 fairly careful with that given the fact that there is 21 already an end result, depending on the plants we look at.
22 If we look at the plants which are " problem 23 plants," it may actually be more informative if we pick a 24 case where we know where a licensee picked up a problem 25 early and in fact responded to it, or we picked up a problem ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
l 35 1
early and the licensee responded to it, which is probably 2
not a problem plant situation but maybe.a different type of 3
performer, to see whether the curve actually reflects the l
4' improvement in performance.
So we will have to look at the 5
case study to ensure that we are really proofing the system 6
under the new system, under the new processes.
7 COMMISSIONER MERRIFIELD:
At what point in the 8
schedule do you think you will have defined those 9
performance indicators?
10 MR. GILLESPIE:
January.
And hopefully by January 11 we will have defined at least the process for doing 12 thresholds.
13' COMMISSIONER McGAFFIGAN:
I will just interpose a 14 comment if I could.
Typically the way this place works, if 15 a paper is due to us in January, you've got a first draft 16.
drafted at the moment and it is probably going around in-17 some sort of concurrence.
So you are going to have some 18 very fast drafting and some very fast concurring, I assume.
19 CHAIRMAN JACKSON:
Bill is working all that out.
20 COMMISSIONER McGAFFIGAN:
Okay.
21 MR. TRAVERS:
We're going to give you an update i
22 every couple of weeks.
23
'MR. GILLESPIE:
Going on to slide 9.
Although 24-there was good alignment on most of the defining principles, 25 there were still two key issues on which-consensus was not 4
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 I
36 1
reached.
The first involved how indicator results, 2
inspection observations, and information sources such as 3
FEMA results and LERs will be integrated'in the assessment 4
conclusion.
5 Although there was no consensus on this topic, 6
there was good agreement that indicators and other 7
information sources should not be artificially merged.
i 8
It has also been acknowledge that while current i
9 assessment processes such as the semiannual plant 10 performance reviews and annual senior management meetings 11 may be able to accomplish this integration, objectivity and 12 scrutability would then to be a challenge.
13 COMMISSIONER MERRIFIELD:
I take it that that 14 integration will also be available in January.
15 MR. GILLESPIE:
Yes.
We would have a proposal on 16 how the indicators would interact with the inspection 17 results.
18 CHAIRMAN JACKSON:
When you say that, you mean 19 that you intend to also be presenting some analysis 20 methodology that would fold in the indicators and the 21 inspection results?
22 MR. GILLESPIE:
We are doing our best to present 23 an analysis methodology as part of the assessment task, yes.
24 CHAIRMAN JACKSON:
It would also be interesting to 25 know what expected change in licensee regulatory burden you ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 1
37 l
1 would expect to see due to an improved process in how you i
2 arrive at that.
3 MR. GILLESPIE:
Yes, Chairman Jackson.
4 MR. COLLINS:
We may not have that o n e..
5 MR. GILLESPIE:
We may not have that one for 6
January.
7 CHAIRMAN JACKSON:
So you want to be honest.
8 Truth in advertising, right?
9 MR. COLLINS:
Right.
I think'we need'to 10 understand a little better once the system is developed, and 11 perhaps it comes with overlaying the information on it from 12 past plants.
13 CHAIRMAN JACKSON:
That's why you need at least an 14 analysis methodology.
15 MR. COLLINS-What we hope to have, though, is a 16 connection.
Perhaps this is at the root of your question.
17 A connection between how the inspcction program interacts 18 with the assessment process, 19 CHAIRMAN JACKSON:
Right, but also how they met to 20 arrive at some judgments.
21 MR. COLLINS:
Yes.
22 CHAIRMAN JACKSON:
That's what I mean by analysis 23 methodology.
24 MR. GILLESPIE:
Going to slide 10.
25 COMMISSIONER McGAFFIGAN:
You didn't stop very ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C.
20036 (202) 842-0034
38 1
.long on the second bulletHon slide 9, voluntary reporting 2
program is preferable to rulemaking.
Are we going to have 3
performance indicators for some plants and not have them for 4
others because it's all voluntary as to whether they bother 5
to give them us?
What is implied in that sentence?
6 MR. GILLESPIE:
This is exactly that.
We are 7
going to have to work with our stakeholders.
To do 8
something in a very timely fashion is going to require a 9
voluntary program.
10 CHAIRMAN JACKSON:
This is not fair to Mr. Beedle, 11 but it would be interesting when you come to the table if 12 you could speak to that issue about how voluntary programs 13 work that would cover the waterfront relative to what the 14 regulator needs to have.
15 MR. BEEDLE:
We'll do that.
16 COMMISSIONER McGAFFIGAN:
I'm just perplexed as to 17 how you have a program where say they come up with 18 --
18 I'll just make up a number -- performance indicators, and 19 for 43 plants we have 18; for 21'we have 16; for 22 we have 20 10.
I'm sure we have some minimal that we control ourselves 21 that we have the data on.
That would be a pretty wild 22 program.
23 MR. GILLESPIE:
Clearly that's not the intent.
24 MR. TRAVERS:
That's not what is envisioned, of 25 course.
So we recognize this question of how you get what ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
39 1
you n:td to do that.
We are struggling now with whether we 2
can get it voluntarily.
3 CHAIRMAN JACKSON:
I'm going to make an 4
advertisement that even predates myself and my colleagues, 5
but it continued into my tenure, and that had to do with the 6
struggle relative to the need for a reliability data rule, 7
were we going to get the data or not, how were we going to 8
get it, would it be disaggregated or would it be aggregated.
9 It's not clear to me how it ultimately turned out.
So we 10 struggled and struggled.
When I say we, I mean the agency, 11 for years and years and years.
If all of this is going to 12 wreck upon the rocks of not being able to get the data 13 through a voluntary program, then I think we are going to 14 have to grapple with what the implications of that really 15 are.
16 COMMISSIONER DIAZ:
Would it be fair to say that 17 the nore comprehensive the voluntary program the less 18 prescriptive will our requirements be?
Is there a 19 correlation there?
If have a very thorough, complete, 20 comprehensive set of indicators, or whatever, I think we 21 could say that the less prescriptive we could be.
Is that 22 correct?
23 MR. GILLESPIE:
That's correct.
24 COMMISSIONER DIAZ:
So in response to Commissioner 25 McGaffigan, there is somebody with a two-by-four sitting l
l l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 t
I l
g 40 1
around here.
2 MR.' COLLINS:
We are receiving the full 3
cooperation of the industry at this point.
I think there is 4
a mutual appreciation for the goals, as was articulated 5
earlier, of this process, which is to allow the licensees to 6
focus their resources on areas where they believe it's
'7 important and less overlaying of the NRC processes on top of 8
that.
9 CHAIRMAN JACKSON:
Theoretically, one could say 10 that if one didn't have the indicators that one needed to 11 make a judgment, that might trigger the need for a look 12 before inspection above the baseline.
13 MR. GILLESPIE:
It's a tradeoff.
14 MR. TRAVERS:
The advantage of it is obvious.
15 It's up-front understanding and agreement; more or less a 16 contract, or who's got the burden and what are these 17 indicators telling us about performance, and how should the 18 regulatory scheme be structured based on that.
19 CHAIRMAN JACKSON:.So'the two-by-four is send us 20 the data or we'll send you more inspection.
21 MR. GILLESPIE:
I think if you go back to one of 22
-our basic defining principles and the idea of an objective 23 being stated and agreed upon for each milestone, the first 24 question for each cornerstone is, what information do you 25 need to make the judgment that there is reasonable assurance ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
41 1
this objective is met?
Whether that comes from a 2
performance indicator or inspection, we are going into this 3
asking that first broad question first.
4 COMMISSIONER DIAZ:
And that's the balance between 5
the inspection and the indicators.
How they integrate is 6
the whole key.
7 MR. GILLESPIE:
That's where it starts coming in, 8
right there.
9 The efforts completed to date and just discussed 10 were intended to provide the framework for the regulatory 11 oversight of commercial nuclear power plant licensees.
12 The current scope of activities inc'lude developing 13 improved processes within this framework to address 14 inspection, assessment and enforcement.
15 As described in the August 25th Chairman tasking 16 memorandum, the activities in these areas are being closely 17 coordinated to ensure that the process improvements remain 18 integrated.
19 The work in these three areas will form the basis 20 for the recommendations for improvements to the regulatory 21 oversight process that will be submitted to the Commission 22 in January of 1999.
23 In addition to work in these three areas, there 24 are several other regulatory oversight processes which need 25 to be addressed and evaluated within this cornerstone ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
= -.
42 1.
framework.
Most of this work is.longar term in nature and 2
will not be part of the January 1999 recommendation.
3 Specifically, there will be a substantial effort required to-4 revise the' inspection program documentation to support any 5
new approach to regulatory oversight.
6 Definitions developed within the framework for 7
risk-significant inspection observations will need to be 8
applied to the enforcement program to help characterize 9
inspection findings.
10 The allegation program needs to be evaluated for 11 appropriate changes within the framework structure to 12 determine whether allegations should be handled in a 13 risk-informed manner.
14 As previously discussed, how assessment results 15 affect enforcement will continue to be evaluated beyond 16 January of 1999.
17 Based on the results of the cornerstone framework 18 and the identification of risk-significant performance 19 areas, changes to licensee reporting requirements may be
'20 warranted.
21 CHAIRMAN' JACKSON:
Let me ask you a couple quick 22 questions.
Will different inspection skill sets be required 23 to implement this program?
24 MR. GILLESPIE:
Bruce and I have talked about 25 that.
I'd like to let Bruce address that.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
43 1
MR. MALLETT:
Thank you, Frank.
I 2
[ Laughter.)
3 COMMISSIONER DIAZ:
That.didn't sound'very l.
4 enthusiastic.
I 5
MR. MALLETT:
We believe that there will be some L
6 skills.
They may be lined up different than we have now.
.7 Now in the inspection program you may have a specific skill 8
for someone who may be in operations, may be in maintenance, l
9 or may be in electrical engineering.
I.think in the future 10 you may need a different skill.
We aren't too sure what 11 that looks like yet.
So we will have to factor into this 12 implementation some training aspect to develop people for 13 those skills.
14 As far as being able to inspect, that basic skill 15 will be there.
16 MR. COLLINS:
Chairman, I would say that overall 17 we would be looking at a'more perfo'rmance-based approach to 18 inspection.
In other words, as you go down through the 19 cornerstone into the tiers, we would be looking more at the 20 cause of performance, whether it be good performance or an-21 area that needs attention, and we would track that back 22 through and come to a determination of whether the action 23 the licensee has taken or proposed to be taken are 24 appropriate.
25 So we will need a better understanding of the i
l ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
44 1
corrective action system and corrective action processes.
2 Potentially, and this is yet to be determined, a more 3
refined skill set on human performance, because that may end 4
to be an area that is raised to a different visibility.
And 5
then probably better training in the risk-informed, 6
performance-based inspection area and less specific 7
disciplines, unless we have a specialist inspection, which 8
would probably be reactive inspection.
9 CHAIRMAN JACKSON:
When you talk about reporting, 10 you mean LERs, or do you mean these performance indicators, 11' or what?
And will the LERs be part of the assessment 12 program?
13 MR. GILLESPIE:
We are looking at how we are going 14 to integrate LERs.
Right now certain things reported in 15 LERs would be picked up as part of the indicators,'and we 16 don't want to do a double count of LERs and indicators.
So 17 we are going to be stepping back and looking at LERS and 18 what information is reported and how you integrate that in 19 with the hard indicator information being reported from the 20 licensee.
So, yes, we do mean both, the potential for a 21 rulemaking on reporting the PIs and the potential that this 22 could change the LER reporting rulemaking in the longer term 23 once we refine probably the second generation of performance 24 indicators, quite honestly.
25 CHAIRMAN JACKSON:
You had earlier shown the slide ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
l 45 1
showing oparator licensing and requalification.
How do they f
2 affect assessment?
3 MR. GILLESPIE:
What we are showing there is l
4 operator licensing and requalification.
Specifically, the 5
requalification piece is right now part of our baseline 6
inspection program.
That came out when we were going down 7
through the attributes.
The question was, what is not 8
specifically covered that would be high risk by'a PI?
So 9
that dropped out as an example, and that was operator 10 reaction under accident conditions in'a control room.
11 CHAIRMAN JACKSON:
Okay.
12 COMMISSIONER McGAFFIGAN:
This chart was titled 13
" Longer Term."
For some of these actions -- I'm looking at 14 Mr. Lieberman -- the threshold for minor violations, that 15 isn't too much beyond January, is it?
My recollection from 16 the ongoing response to the Chairman's tasking memo is that 17 that is an early spring deliverable from you; isn't it?
s 18 MR. LIEBERMAN:
That's c'orrect.
In the past the 19 enforcement program drove the assessment process.
We want 20 the assessment process to drive the threshold process, and 21 we need that to be done.
Right' after that we will be ready 22 to go.
23 COMMISSIONER McGAFFIGAN:
So these are longer term
- 24 but quite near term in NRC time?
25 MR. GILLESPIE:
Yes.
Early spring.
If you look ANN RILEY & ASSOCIATES, LTD.
l Court Reporters l'
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
46 1
at.our schedule, longer term tends to be before June for 2
most milestones.
3 CHAIRMAN JACKSON:
Is it fair to say that that 4
second bullet under longer term relates to ensuring that 5
there is an appropriate alignment between thresholds for 6
inspection and thresholds for minor violations?
7 MR. LIEBERMAN:
That's right.
We want to make 8
sure that we are not collecting in inspection space 9
information that we don't need for assessment and we are not 10 enforcing things which aren't important for the bigger 11 picture oversight issues.
We want to have these things 12 integrated together.
13 CHAIRMAN JACKSON:
So in addition to having 14 assessment come ahead of the curve, it is also meant to 15 ensure that there is an alignment here; is that correct?
16 MR. GILLESPIE:
Yes.
If we re successful in a 17 risk-informed baseline inspection, then inspectors in. theory 18 should not even be looking at things that we would put in a 19 minor violation category today.
20 CHAIRMAN JACKSON:
Okay.
21 MR. GILLESPIE:
Slide 12.
There are currently i
22 four short-term activities in progress to develop a 23.
recommendation to the Commission on improvements to the
.24 regulatory oversight process.
25 The technical framework group, led by Pat ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
1
l l
47 1
Baranowsky, is responsible for building en the work started 2
in the public workshop, to complete the development of the 3
cornerstones by~ identifying appropriate performance 4
indicators, establishing criteria for thresholds, and 5
establish the basis for risk-informed baseline inspection.
6 The inspection task, led by Bruce Mallett, is 7
responsible for developing a process that addresses scope, 8
depth, and frequency of a risk-informed baseline inspection.
9 The scope and basis for inspection were based in part on 10 input received from the framework group.
11 The assessment process group, led by Mike Johnson,
.12 will determine methods for the integration of indicator and 13 inspection data, develop criteria for NRC actions based on l
14 assessment results, and determine the best method for 15 communication of the results to licensees and the public.
16 The enforcement activity, led by Jim Lieberman, is 17 working with and participating in these tasks to ensure that 18 the enforcement process changes are properly evaluated in a 19 framework structure and the changes to the inspection l
20 assessment program are integrated with changes to the 21 enforcement program.
22 All these activities are fully coordinated and i
23 integrated and consist of broad participation from all four 24 regions, NRR, OE, Research, and AEOD.
25 With that, I would like to turn it over to Pat i
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
l
48 1
Baranowsky to address the technical framework.
2 MR. BARANOWSKY:
The technical framework group, as 3
Frank mentioned, does have representatives from a broad 4
spectrum of the NRC's. offices.
Let me mention some of the 5
disciplines that are involved.
6 We have people with field inspection and 7
inspection program development background; maintenance rule 8
implementation; performance indicator development analysis; 9
emergency planning; health physics; security; human 10 performance; risk assessment; and enforcement.
11 That is a pretty broad-based group.
If you look 12 at the cornerstones, you will see they cover pretty broad 13 indications.
14 We have about a dozen full-time and about a half 15 dozen half time staff involved in this activity.
16 The charter for this' group is to develop the 17 details of the technical framework for a more objective, 18' risk-informed and performance-based approach to licensee 19 performance assessment, and to provide related bases for 20 inspection activities.
Therefore, the information that is 21 developed by this group will be used in the development of 22 the risk-informed baseline inspection program and for the 23 performance assessment tasks.
24 The work of this group, as Frank mentioned, will 25 follow and build on the defining principles and the l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
l I
49 1
cornnrstone development effort that was begun at the 2
performance assessment workshop in late September of this l
L l
3 year.
4 Also, as was mentioned by Sam Collins, we 5
recognize that this is really the first phase of an activity 6
that is going to evolve over several years through 7
implementation, feedback, and improvement of the process.
l 8
Nonetheless, it's our intent to develop with sufficient 9
detail information that will allow the Commission to make a 10 decision on the efficacy and direction of this new approach 11 to licensee oversight for potential near term implementation 12 even though there may be some future development in the j
i 13 years to come.
14 COMMISSIONER DIAZ:
I'm sorry, my major concern 15 with this paper is in some of the definitions of 16
' cornerstones.
Are you going to cover that because they are 17 in the appendix, or should I just jump right in to it?
18 MR. BARANOWSKY:
I think we should jump right in.
19 CHAIRMAN JACKSON:
Backup slide la.
20 COMMISSIONER DIAZ:
I'm sure we are going to this 21 from the principle that when you look at these. things you 22 first determine what is the desirable outcome, and second, 23 how you are going to regulate to make sure that that outcome 24 is there.
25 If I look at the definitions in here, I do have a l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
50 1
little problem with the way thDy are stated.
Let me start 2
with initiating events:
limit the frequency of events that 3
upset plant equilibrium.
I'm a little leery about the
-4 words, because plant equilibrium is upset in many different 5
fashions, and those might not be initiating events.
So I 6
would encourage the staff to look at that.
7 Words might be something like " initiating events 8
that create deficiencies in plant balances (reactivit/, heat 9
transfer, and coolant inventory)"
That would be very 4
10 specific.
11 I think it's important to know what we mean by 12 upsetting plant equilibrium, because if we have a scram due 13 to losing a transformer, say 50 percent load is gone but the 14 plant is still responding very well, it would be an event, 15 but it it might not be an initiating event that would create 16 a response.
17 Fundamentally, there are 'hree things that we are t
18 always looking at when we look at critical safety functions, 19 and that is reactivity, heat transfer, and coolant 20 inventories.
I don't know of any other.
Some definition on 21 that might be appropriate to avoid people getting upset 22 about upsetting plan equilibrium.
23 CHAIRMAN JACKSON:
There is actually a definition 24 of initiators in a PRA sense.
.25 MR. BARANOWSKY:
I wonder if I could respond to l
l l
ANN RILEY & ASSOCIATES, LTD.
l Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
51 1
thnt.
2 COMMISSIONER DIAZ:
Please.
3 MR. BARANOWSKY:
I hope you will be happy 4
eventually when you see what we are putting together.
Our 5
job is to take these bullets that were basically put on 6
paper as a result of the workshop and detail them out just 7
to cover the kind of concerns that you are raising.
In 8
fact, for each of the cornerstone areas we are going to have 9
a fairly substantial discussion of what the cornerstone is.
10 what the performance concern is, how the performance 11 indicators relate to those things what the performance 12 indicators can't do.
13 COMMISSIONER DIAZ:
I have absolutely no doubt 14 that you will do that.
Again, the summary is something that 15 people look at and they form their own images.
I think this 16 process has to be so transparent, so well defined that some 17 of those things are important.
18 Quickly, because I know we are time constrained, 19 when you go to mitigation systems, there is some things that 20 we need to state that have to be according to our rules.
21 This definition still has something missing.
For example, 22
" ensure that those systems required to prevent and/or 23 mitigate core damage perform at a level commensurate with 24 their safety significance."
It has to include " perform or 25 are capable of performing," because if they are not capable ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
52 1
of performing that function even if they were not challenged 2
by an initiating event, that might be sufficient to be a 3
cornerstone.
A lot of our things are established on the 4
capability to perform the function.
So " perform or capable 5
of performing."
You have to have it.
If not, we are not 6
compatible with other things.
7 The same thing on barrier integrity:
" Assure that 8
the physical design barriers protect 'or are capable of 9
protecting.'d In other words, the capability has to be 1
10 there not only being able to do it.
l 11 With just those minor corrections, your summary i
l 12 actually becomes very inclusive.
13 MR. BARANOWSKY:
Thank you.
14 COMMISSIONER DIAZ:
You're welcome.
15 CHAIRMAN JACKSON:
Sam.
16 MR. COLLINS:
Let me raise a fundamental issue 17 here.
Not to resolve it here, but there is a difference in 18 looking at the level ~of engagement of the regulator.
In 19 your own words, things happen at' power plants, and you do 20 have random failures, random events.
21 We become concerned at a different level when 22 those random events result in actual challenges to our 23 safety safety, as opposed to challenges to a safety system 1
1 24 that the safety system does not function as required, as 1
25 opposed to it doesn't function as required when you ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
53 1
challenge a barrier versus a barrier failing.
So it is a 2
graduated approacn.
3 Although eyatems that may not have worked are a 4
concern to us, they are not of the same concern under this 5
scheme as those that are actually challenged.
So what we 6
have to decide as an agency at what level we are going to 7
engage versus at what level we are going to ensure that it's a
8 understood and that the licensee is approaching the issue 4
9 appropriately.
10 In other words, if I can paraphrase Jim, does luck 11 count?
Does the fact that you had a potential but didn't 12 have a circumstance that has a nexus that is close to one of 13 our strategic goals more important?
We are still working 14 our way through that to some extent.
15 CHAIRMAN JACKSON:
Actually, the two things do tie 16 in.
Commissioner McGaffigan raised the issue of cable 17 separation and other fire protection issues.
You could 18 argue that cable separation relates to the capability line 19 as opposed to did they work if there were a fire, or what 20 have you.
I think the real answer is that you have to'give 21 some specificity that relates to these fundamental barriers C
22 and you have to be clear on what this graduated approach is l
23 that you are talking about related to that.
24-COMMISSIONER DIAZ:
I agree.
It's the specificity 25 that will avoid the problems.
Voluntary or involuntary, ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
M 54 1
whatever it is, there is still soma things that are low, and 2
we still-have to be able to maintain the capability to 3
. perform the function.
If we want to be specific about what 4
grade we are going to risk inform those functions, that's 5
fine, but you still have to have that.
6 CHAIRMAN JACKSON:
That's a good point.
That's a 7
good way-to put it, I think.
8 MR. BARANOWSKY:
The product that we are going to 9
have is basically part of the paper that will come up here 10 January 1999 and will document the principles, bases, logic, 11 and technical information that supports all these areas at 12 that time.
13 Let me go to number 14 and talk a little bit more 14 about some of these specific tasks that we have.
We had 15 covered some of this stuff in pieces,. parts and chunks 16 earlier as we had a question and answer session, but I will 17 talk about a few of these.
18 The cornerstone task,.as I said, is primarily to 19 detail out the~few bullets and charts that we have, to cover 20 the scope, key definitions, and relationship to other 21 activities.
We are going to have operating events that are 22 significant by themselves, how does that relate this, and 23 what do we do about that.
Reporting, generic issues, and so 24
-forth, all have.to be looked at in terms of this framework.
25 Enforcement philosophy as it relates to the ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 l
(202) 842-0034 l
l 55 1
defining principles and development of the performance 2
indicators, inspection bases and thresholds, will also be 3
considered in this technical framework development task.
l 1
4 The performance indicators are intended to be risk 5
informed to the extent practical at this time.
6 The performance indicator task involves evaluation 7
of the performance indicators that were proposed at the 8
workshop, and also includes the identification of other C
9 performance indicators where either there were some holes 10 identified or limitations.
11 However, I should note that not all the 12 performance indicators are going to be so amenable to 13 risk-based or risk-informed thinking.
For instance, the 14 radiation protection cornerstone area is not so much based 15 on risk as it is ALARA and other regulatory principles that 16 haven't been evolved through the kind of risk analysis that is associated with initiating events and mitigating systems.
17 18 In general, the PIs are meant to be a broad sample 19 of performance in some of the more risk-significant areas 20 and those areas that are delineated by the cornerstones.
21 For instance, in the mitigating systems, although 22 we are not going to look at all risk-significant mitigating 23 systems, at this time we are thinking about four or five key 24 systems, ones that we have some form of indication available at this time that would be easy to develop, because we are 25 ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
56 1
working'under some time constraints.
We think that would be 2'
practical, and moreover, we think that we are going to get a 3
large chunk of what we need to get in terms of risk-informed 4.
information from that set of indicators.
5 In the future, our work will involve developing 6
' improved indicators or additional indicators, and we will 7
also look at that in this activity.
8 Validation of the performance indicators is also a 9
part, and that is related to the adequacy discussion that we 10 had earlier.
11 Evaluating their limitations.
Their limitations 12 are significant in terms of the development o'f inspection 13 bases.
We want to make sure that the risk-informed 14 inspection program takes advantage of the information 15 generated by the performance indicators but that we don't 16 misunderstand the capabilities of performance indicators to 17 give us relevant information, i
18 The inspection bases will include identifying 19 areas where verification an'd validation needs to continue, and of course the risk-sig'ificant aspects of performance 20 n
i 21 not adequately govered by the PIs.
i 22 The threshold task involves the identification, 23 definition and evaluation of the performance indicator 24 thresholds.
These thresholds are intended to provide a 25 clear demarcation point or points for identifying fully l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C.
20036 (202) 842-0034
l 57 l
1 acceptable parformance, areas of declining performance, and l
2 unacceptable levels of performance.
3 We will be evaluating the thresholds proposed by 4
the Nuclear Energy Institute for several of the proposed l
5 indicators, and we will perform some of our own independent i
6 analyses of PI response, the benchmarking, sensitivity to 7
risk, and that kind of thing.
t 8
An important aspect of the evaluation of 9
performance indicator response and thresholds is determining l
l l
10' the ability of indicators to identify declining performance, i
11 allowing the staff time and the licensee time to evaluate 12 problem areas, and initiate corrective actions before 13 reaching an unacceptable threshold.
This would allow 14 licensees time to do what they have to do and the NRC to 15 implement a graded response to declining performance.
16 In this regard, we will be considering regulatory 17 as well as safety implications of crossing a threshold and 18 possible mandatory actions associated with the. unacceptable 19 threshold.
20 The enforcement philosophy and implementation are 21 primarily being addressed in the performance assessment and 22 the' inspection baseline groups.
However, we are going to 23 look at this philosophy in terms of its logical connection 24 and consistency with the technical framework as it's E
25 developed.
l l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036
-(202) 842-0034
58 1
Questions.
2' COMMISSIONER McGAFFIGAN:
Could I go back to a 3
question I asked earlier and was told, I think, this is the 4
time to ask it?
Do we take regulatory action in a single 5-area?
Even they.are green everywhere else, if they dip into 6'
the white, in that area, is that the notion?
7 MR. BARANOWSKY:
That's the notion that is 8
currently being proposed, although Mike Johnson is going to-9 be looking at whether or not that is going to be our final
'110' posture and how we should look at groups of indicators 11 changing in one way or another.
We are not planning at this 12 point to have an integrated indicator like the one we 13 recently developed and put out with the IRAP public comment 14 paper.
15 COMMISSIONER McGAFFIGAN:
If you are in the green 16' zone and we have an inspection finding that belies the 17 indicator, the burden of proof is on us, but we still can 18 take regulatory action in the green area if you pass that 19
. burden?
How does that burden get manifested in terms of 20 staff processes?
Is there a higher level of approval 21 required to take a regulatory action if, despite the burden 22 of proof being on the regulator, it's passed?
23 MR. TRAVERS:
Commissioner, I think this is 24 exactly right.
We would look on that compelling argument 25 that we were speaking of earlier as one that would be a ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 t
l I
l i
'59 1
burden on us to make if they are in the green zone, but one 2
which we very well could make'if we felt by virtue of our i
31 inspection program or any other information that we had that 4
we needed to engage on an issue, and how we would engage on 5
the issue would be determinant on just what the issue is and 6
the extent to which it looked to be a problem.
We could 7
have a meeting; we could issue an order; we could do a l..
j 8
special inspection; we could put more resource on the issue.
9 COMMISSIONER McGAFFIGAN:
I'm just trying'to
~
10 understand what burden of proof means.
If I'm a licensee 11 and I get an inspection and at the exit interview it's clear 12 that despite sailing along in green in whatever category or 13 set of categories this might fall under, I really do have 14 some problems here, and I should expect regulatory action.
15 Do I petition to the EDO to say, I'm on top of it now, I 16 appreciate what your staff found, but let me fix it and
'17 don't do anything, because I'm in green?
And how is this
~
18 process scrutable to me, that you make a decision that
'19 despite the green, you are going to have a public meeting,
~20 you-are going to have an order, you are going to do
'21 something?
22-MR. TRAVERS:
That's part of the challenge that we 23
.have to yet' develop, but the expectation would be if you are 24 in the green that we wouldn't be in a position to engage.
25 COMMISSIONER McGAFFIGAN:
Despite the fact you've l
l l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters
[
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 l
(202) 842-0034
~-
.-.~.-
60 1
just found some stuff in an inspection report, I might leave 2
the meeting thinking, well, I'm still in green.
Then three 3
months later NRC takes an action and comes out of the blue.
4 How do we make sure that it doesn't come out of the blue, 5
that we signal to them early on that we regard these 6
findings of such magnitude that we may take regulatory 7
action despite their being in the green, and that that'is 8
being considered?
9 MR. TRAVERS:
I think engagement at its earliest 10 phase would include dialogue with the~ licensee.
I think 11 that is what is envisioned.
12 MR. COLLINS:
It's really no different than our 13 processes provide for now.
14 MR. TRAVERS:
That's right.
15 CHAIRMAN JACKSON:
So you would engage because you 16 have dissonance between what the indicator says and what 17 your inspection results might say.
18 MR. TRAVERS:
Correct.
I would assume that what 19 we would have in place -- it's early yet, and we haven't 20 really developed the process -- but that we would have an 21 internal process that would guide us in developing the 22 compelling case.
We would base it on whatever information 23 was at hand, including information from the inspection 24 program, and so forth.
The internal process would make it 25 one that is very carefully managed.
So again, the ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
61 1
expectat. ion would be that things do happen at these complex f
2 plants and that a few things or normal kinds of events or l
3 issues would not result in engagement.
If we felt we needed 4
to, we could do it, but it would be a very carefully managed 5
process.
6 COMMISSIONER McGAFFIGAN:
All I am suggesting is l
7 that it's probably going to have to be a relatively 8
scrutable and transparent process from the point of view of 9
the licensee.
We get criticized today.
One of your early 10 viewgraphs talks about more scrutability, more transparency, 11 et cetera.
Once we divert from the model, we can't make it 12 inscrutable and less' transparent.
13 MR. TRAVERS:
I absolutely agree.
That's why when
.4.
Frank Gillespie was speaking he was talking about' developing 1
.15 the thresholds that we would use for that kind of 16 engagement.
17 CHAIRMAN JACKSON:
This is a question I want to 18-put on the table.
If it's best addressed by Dr. Mallett, 19 that's fine.
If not, if you could answer it now.
It i
20 actually relates to this, particularly if there is ever an 21 area where there may be a dissonance between what a 22 performance indicator or set of indicators seem to say
(
23-versus inspection findings.
24 Are you looking at, perhaps both in indicator land 25 and inspection land, at sample size, and that it be L
l ANN RILEY & ASSOCIATES, LTD.
Court Reporters l
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
62 1
established in a way that provides a demonstrable level of 2
confidence?
Or is there any kind of statistical or sampling 3
protocol that is under consideration in the selection of the 4
number, types and thresholds of indicators as well as 5
inspection observations?
6 MR. BARANOWSKY:
I don't think we have a 7
statistical sampling process in mind for the performance-8 indicators.
The approach there is to try and capture the 9
bulk of the risk as we understand it and to use indicators 10 that are broadly understood to be important to risk, which 11 means we are going to take insights that we get across the 12 industry in selecting these indicators as opposed to being 13 very plant specific and picking details for the indicators.
14 There may be some plant-specific elements to the 15 indicators such as performance thresholds that make sense in 16 either peer group, or certain design features that would 17 benefit from a plant-specific approach, but when it comes to 18 the inspection program, I think we are talking about some 19 sort of sampling.
Bruce might want to address that.
20 CHAIRMAN JACKSON:
If you are going to talk about 21 it as part of your presentation, I'm willing to wait, but I 22 want to be sure that you are going to talk about it.
23 MR. MALLETT:
Okay.
24 MR. GILLESPIE:
One other element.
If we have 25 this compelling case, we have two problems.
One is the ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
63 1
plant-spncific problem we have the compelling safety case 2
on.
The other is the feedback ~1oop that says this 3
challenges having selected the right PIs and how we are 4
using them.
So there are two elements to that when we come 5
across it.
Our intention would be to have that feedback 6
loop in to take on that challenge if it occurs.
7 COMMISSIONER DIAZ:
If I might be able to confuse 8-myself, if you think of this as you being a controller in 9
the sense of controlling processes, I think what 10 Commissioner McGeffigan is saying is,'if any one of the 11 inputs or the desired outcome has a significant delta or 12 error margin ~from what you expect, then immediately the 13 process gets more focused, and you might take action.
j 14 There are two ways in which that could happen.
It 15 could happen very suddenly.
All of a sudden you have 16 inspection finding on something that shows you that you are 17 out of whack.
Or it could be a degrading process which is 18 slowly changing.
Either one of those could actually trigger l
19 our actions.
Is that correct?
20 MR. GILLESPIE:
That's correct.
21 MR. COLLINS:
One the' process would accommodate by l'.
22 the levels of engagement and the shift in the burden of I
23 proof.
The second would be the more. extreme case where we 24 would have a data point which is abnormal, if you will.
25 That would engage a scrutiny of the system as well as a ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 L
.~
.. ~ _.
64 1
reaction to the issue.
2
~ COMMISSIONER McGAFFIGAN:
One of the issues that I 3
'see'is the performance indicators are always going to be 4
They will be lagging even if they-are close to 5
concurrent.
An inspection finding is here and now.
You 6
came across something end, like you say, it may be abnormal, 7
'but the performance indicators are backward looking, at best 8
concurrent, the ones I've seen.
An inspection report, as I 9
say, the person was in the plant yesterday and he found such 10 and such, and it's either a big concern or it isn't.
If it 11 is a big concern, despite their being in the green, you have 12 to have some mechanism for dealing with it.
13 MR. GILLESPIE:
Right.
The big concern is 14 probably the easier one to deal with, when it is something 15 that is recognized as verye very significant.
It's the 16-accumulation of small things that is going to take us a lot 17 of thought on how to deal with building a compelling case 18 when there is an accumulation of small things.
19 Also, there are two thresholds we are dealing with 20 here.
One is an operating threshold which, if successful in 21 the system, will be set high enough to give us some, you 22 might say, margin, accounting for a one quarter or lag time 23 when the trend shows up and crosses the threshold, but yet 24 leave a utility sufficient freedom so that they can catch a 25 trend and reverse it themselves before we have to interdict ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
=.
..~ -. -.
65 1
ourselves.
Much lower, we would hope, is the safety margin 2
where much sterner action would have to be taken, and in 3
between these there is a gradual engagement with us.
4 COMMISSIONER McGAFFIGAN:
The plant that keeps 1
5 coming to mind is D.C. Cook.
I don't think there were lot 6
of indicators before that inspection occurred, and then it 7
fell off.
Mr. Lochbaum might believe, and I'm sure does, 8
that he had concerns with the ice condenser plants earlier I
9 that hadn't been fully addressed.
That plant inspection 10 comes along and the plant goes from non-regulatory 11 difficulty to-regulatory difficulty all at once.
Tile old l
12 inspection program doesn't catch it; performance indicators 13 don't necessarily catch it.
How does it work in this new 14 system *:
It's probably in the green.
15 CHAIRMAN JACKSON:
It's in the green if you don't 16 look at it.
17 COMMISSIONER McGAFFIGAN:
It's in the green, 18 according to performance indicators, in this area where 19 performance indicators perhaps don't capture very much.
I 20 think you showed that backup slide earlier.
This is i
21 probably one of the areas where you have inspection still 22 and don't rely on indicators.
23
'I think one of the things the industry is looking 24 for is predictability in the regulator, the notion that not 25 everybody is one day away from regulatory difficulty.
You ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C.
20036 (202) 842-0034
66 1
still have these hard cases.
Unless you can tell me how 2
this new systems would catch D.C. Cook without having an 3
engineering inspection.
4 MR. GILLESPIE:
In discussions we've had, even 5
with NEI, we would fully expect that there are four to six 6
things a year that seem to occur by exception that we are 7
still going to have to deal with outside the defined 8
performance indicator risk baseline inspection program, 9
whether it's an event that has multiple items that still 10 stay in the green or whether it's significant design 11 problems.
And design is one of the areas which was 12 crosscutting through all the cornerstones that came up 13 needing to still be inspected.
14 CHAIRMAN JACKSON:
That's the question.
Will the 15 risk-informed baseline inspection program capture 16 risk-significant design issues or not?
17 MR. MALLETT:
Let me answer that, Frank.
18 We are planning to do that.
What we are also 19 planning to do is; when we get m6re inspectable items 20 defined and what we want to look at, we are going to go back 21 and benchmark some of these events that occurred or some of 22 these issues that came up, like a D.C.
Cook, to see if we 23 have covered in our inspectable items those sort of things 24 in the baseline program.
I'm not saying we may pick them l
l 25 all up, but we want use that a way of a self check to make l
l I
l l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters l
1025 Connecticut Avenue, NW, Suite 1014 i
Washington, D.C.
20036 l
(202) 842-0034 l
1 l
67 1
sure wn have bean all-inclusive in our inspectable items.
2 MR. COLLINS:
Chairman, I think the direct answer 3
is the risk-informed core baseline inspection will probably 4
not contain an in-depth design engineering review.
That's l
5 not to say it wil' act be done either as a result of the i
6 supplemental _or as a result of licensees doing it i
7' themselves.
Much of the development of the inspection l
8-programs is going to depend on the role of the licensee as
~
L 9
far as either routine reviews or corrective action as a 10 result of findings, as well as those areas that we believe 11 periodically in order to ensure that the performance 12 indicators are giving us accurate information, we will go in 13 and delve into.
That may be different than what the 14 risk-informed core baseline inspection is.
15 CHAIRMAN JACKSON:
I understand the point you are 16 making, but going back to a couple of the examples, one has 17 to come out of this.
There is some additional effort that 18 is not part of today's discussion having to do with 19 definition of design basis and design basis information.
20 Somewhere along the line there has to be'some kind of 21 scoping or risk ranking in that arena, and you have to be 22' able to say how you deal with those things that show up at 23
'the top of the list.
24 It's not necessarily everything, but you have to 25 be able to say how you are going to de.41 with that:
Where i
1 ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
68 1
is'the information coming from?
Is is licensee 2
self-assessment?
Is it some inspection that is or is not 3
part of the risk informed?
4 But if it's important, if it's high in a risk 5
sense, do you not have to address it at some level so that 6
you-don't come along and here's a surprise that shuts a 7
plant down for a year, two years, et cetera, because all of 8
a sudden this was something that was discovered?
Maybe it 9
was self-revealed, maybe not, and now it shuts the plant 10 down for X period of time.
11 That's one of these kind of sudden surprises that 12 on the one hand is very unpleasant for the licensee, and on 13 the other hand, makes us look bad if it warrants a plant 14 being shut down for two years and we've been going along 15 saying all the time it was fine.
So I feel somehow you have 16 to get at that.
It's not something you can walk away from.
17 COMMISSIONER DIAZ:
It's going back to the same 18 thing.
We have to have the capability to perform those 19 functions that we believe are essential to be performed.
20 All we are going to do is going to risk rank them to a point 21 that we know which ones those are, and that's where the
- 22 specificity comes in.
D.C.
Cook will be captured by 23 capability to perform.
24 CESIRMAN JACKSON:
Right.
This will be a 25 performance expectation that those things that come up high ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 t
69 1
on that risk ranking are dealt with, but secondly, there is 2
a question of how does it get dealt with in this process.
3 If it's not covered by performance indicators, if you are 4
telling us it's not covered by a risk-informed baseline 5
inspection program, then how is it going to be covered?
l 6
It either has to be covered by a risk-informed 7
baseline inspection program suitably defined, or by some 8
licensee self-assessment.
Perhaps that's the way to go.
9 But there has to be something for those things 10 that show up at the top of the risk list that have to be 11 dealt with.
If you don't d'eal with it, you've left a big 12 hole, number one, from the point of view of s'afety 13 oversight, but secondly, you're left the big surprise, and 14 that also is unacceptable.
15 MR. MALLETT:
Let me make a comment.
One of the 16 things Pat Baranowsky and I've talked about is he's going to 17 give us a list of things he believes do not have adequate, 18 to use your phrase, performance indicators.
We are 19 approaching it from a different standpoint.
We're looking 20 at everything that we believe we need to have in the 21 inspection program first to get this baseline assessment.
22 Then-we will modify that, depending on whether we have 23 adequate performance indicators, and modify based on risk, 24 we hope.
I think that will address your issue, or we hope 25 it will.
l l
ANN RILEY & ASSOCIATES, LTD.
l Court Reporters l
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
70 1
CHAIRMAN JACKSON:
And.thereby address his issue.
2 MR. MALLETT:
It won't address the green coloring.
3 CHAIRMAN JACKSON:
I'm talking about the surprise.
4 MR. MALLETT:
It should address all-inclusive in 5
the program if we've done our job right..
6 COMMISSIONER MERRIFIELD:
It seems to me acute.is 7
going to be the point at which you can do the benchmarking 8
of the performance indicators.
You are going to be making a 9
presentation to us in January of the performance indicators.
10 At what point will you be able to do some of that 11 benchmarking to give us some greater assurance that you've j
12 hit the mark?
What's your time line for testing those 13 performance indicators to determine, based on past 14 performance, that they would have picked up the concerns 15 that Commissioner McGaffigan has raised?
16 MR. BARANOWSKY:
The benchmarking that is part of 17 this effort is starting now.
The Nuclear Energy Institute 18 has done some of their own work, and we are laying out our 19 activities that we want to do.
20 I just want to mention that we b've done 21 benchmarking of performance indicators that are similar to 22 the ones we are talking about here as part of the Arthur l
23 Andersen work over the last several years.
I have pretty 24 reasonable belief that these things are going to have 25 capability of indicating poor performers.
t ANN RILEY & ASSOCIATES, LTD.
l Court Reporters l
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 i
71 1
There is going to be a question of whether you 2
want to have false positives or false negatives and 3
statistical issues like that that we have to address.
4 That's what we still have to delve into with these indicators that we haven't worked with day in and day out, 5
6 but they are similar to ones that we have worked with in the 7
past.
8 CHAIRMAN JACKSON:
I think you have to present the 9
documentary evidence to the Commission.
If it's based on 10 the Arthur Andersen type algorithm using the indicators you 11 come up with, you need to present that even if the names 12 have been changed to protect the innocent.
13 MR. BARANOWSKY:
It's going to be part of what we 14-provide in our January paper, and I think you will see 15 information coming out as we go to the ACRS in December.
So 16 it will be coming up shortly.
17 COMMISSIONER McGAFFIGAN:
The point for 18 Commissioner Merrifield is that there are some areas that 19 just aren't going to be covered.
You're not going to have a 20-performance indicator for the capability of the ice 21 condensers.
l*
22 MR. BARANOWSKY:
We're not saying we can do that.
23 That was a problem with the old system.
24 COMMISSIONER McGAFFIGAN:
It was a problem with l
25 the old system; it's going to be a problem with the new 4
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C.
20036 (202) 842-0034 i
72 1
systems; we're going to try as best we can to work the 2
inspection program to fill the hole, as I understand the 3
answer.
4 CHAIRMAN JACKSON:
Inspection and/or other things.
5 It could be licensee self-assessments, or required 6
self-assessments, or whatever.
Agreed upon 7
self-assessments.
What you have to really lay out is how 8
the pieces flow together.
The performance indicators, you 9
have to know where they start and where they stop and where 10 the inspection goes and where self-assessment comes in.
11' Nonetheless, you have to be assured that you have covered 12 the waterfront.
13 MR. GILLESPIE:
Going back to our first principle, 14 the objective statements, what information do you need that 15 reasonable assurances objective is being met?
That was the 16 importance of the objective statements.
That is where Pat 17 is taking off from, Design shows up in every single 18 cornerstone and different aspects of design.
We will have 19 some risk-significant approaches to it.
We are trying to 20 grapple with that problem and what comes out and how it 21 comes out the bottom.
22 CHAIRMAN JACKSON:
Okay.
Let's move on.
23 MR. BARANOWSKY:
That is the end of my talk and 24 time for John Flack from the Office of Research to tell you 25 a little bit about risk lists.
l l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 hashington, D.C.
20036 (202) 842-0034 I
73 1
CHAIRMAN JACKSON:
I was just thinking about you.
2 You're-an SR squared A as oppo' sed to an SRA; is that 3
correct?
4 MR. FLACK:
I am a risk assessment engineer, not 5
an SRA.
6 CHAIRMAN JACKSON:
Right.
So you are senior risk 7
and reliability analyst.
I'm calling you an SR squared A.
8 MR. FLACK:
Okay.
In any case, risk will be 9
considered in all these issues, burden of proof and 10-inspection.
So let me go on to that.
11 Before I begin to describe how we are utilizing 12 risk insights from both the IPE programs and PRAs that are 13 available to support the development of risk' informed 14 oversight process, I'd like to highlight a few issues up 15 front that are important to the development stage.
16 The first is the generic versus plant specific 17 issue.
This is really a question as to the extent to which 18 we can capture generic risk insights in formulating a 19 risk-informed inspection process ~.
20 On the next viewgraph I'll summarize the approach 21 we've taken to address this issue.
22 The next issue we will specifically consider risk 23 in inspection and decision making and what metric and 24 criteria we are going to use for this.
25 Although guidance still needs to be developed to ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
i 74 1
. support these activities, we expect that that guidance would 2
be consistent with Reg Guide 1.174, and that the risk 3
information would be used in conjunction with other 4
considerations such as defense in depth and safety margin.
5 The third issue, treatment of items not modeled in 6
PRA, is really to keep us aware of the fact that PRAs do not 7.
cover everything and that we will not overlook important 8
issues like acts of commission, complex system interactions, 9
and transition risk.
10 Finally, the fourth key issue involves resource 11' allocation and use of risk to prioritize and guide the 12 inspection process.
The risk significance and availability
-13 of the PI data that. Pat Baranowsky just described and risk 14 will both play a factor in assessing our inspection needs.
15 This is being addressed in the ongoing research 16 right now, which I am about to go over on the next slide.
17 CHAIRMAN JACKSON:
You are saying there will be a 3
18 plant-specific inspection program that is tied to the actual 19 elements of risk presented by a given facility?
20 MR. FLACK:
We are looking at it from two 21 perspectives, generic and plant specific.
The 22 plant-specific aspect would probably involve more of the 23 maintenance rule development, information from the 24 maintenance rule, which is plant specific.
We are trying 25 from both perspectives.
What we are trying to look at is ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
'75 1
from the generic perspective what we can capture, and then l
2 what would need to be supplemented with some plant-specific 3
insights, but I'll get into that in a minute.
l 4
In fact, our first step was to identify and 5
priorities sources or risk and link these to the 6
cornerstones using the generic PWR and BWR insights and 7
plant-specific insights.
In this process we utilized the 8
IPE insights and findings contained in the various NUREG 9
reports as well as the IPE database to identify those 10 contributors found to be most important by licensees.
11 By scanning across the top ten sequences of each 12 plant we were able to take a broad look at whIat is driving 13 the risk at nuclear power facilities.
In general, these top 14 sequences can capture 80 to 90 percent of the contributors 15 to core damage frequency at any one plant.
Sequences that 16 shows up to be in 50 percent or more of the plants was 17 considered high and generic.
18 At the same time, we took a vertical slice using 19 the Surry IPE and'NUREG-1150 results to gain insights into 20 what would not be captured using the generic approach.
21 Taking this approach, we found that about 50 percent of the 22 core damage frequency for internal events could be captured 23 generically, but that a deeper understanding of 24 plant-specific features would be needed to capture the full 25 range of contributors.
l ANN RILEY & ASSOCIATES, LTD.
)
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 i
Washington, D.C.
20036 (202) 842-0034
i 76 1
Once the risk insights were identified, they were 2
arranged into a matrix so that we could link them to each of 3
the cornerstones.
In a similar fashion, the risk was linked 4
to underlying attributes which could then be used as a focus 5
of the oversight activities.
6 Together, these form what we call the risk matrix 7
and a framework for bringing into the process risk 8
information.
9 Now that we have the risk matrix, the next step is 10 to link the risks to the identified performance indicators, 11 and as they become developed, the identification is still 12 ongoing, but the above approach provides a means by which we 13 can accomplish this task.
14 To summarize our first phase work, we were able to 15 capitalize on information generated by the IPE program and 16 NUREG-1150 to formulate a risk matrix and establish an 17 approach that links risk insights to the cornerstones, 18 attributes and PIs as they become available.
19 Also, we are now in a better position to put in 20 perspective generic versus plant-specific risk information.
21 We plan to continue our effort to capture external events 22 and shutdown risk and insights from other risk importance 23 measures.
24 We will also be looking at the application of 25 inspection resources using risk-informed approaches where ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
77 1
PIs do not cover the area.
2 This summarizes the research work to date.
If 3
there are no other questions, I'll pass it on to Bruce 4
Mallett.
.5 MR. MALLETT:
Turn to slide 17.
I believe I can 6.
still say good afternoon.
I want to provide you a 7
perspective on what the risk-informed baseline inspection 8
group is doing as part of their project.
We discussed some 9
of these issues earlier, so I'll just try to highlight a few 10 of them in the interest of time.
There are a few points I 11 want to make.
12 The overall objective of the project is to 13 describe a program of how the NRC will conduct its baseline 14 inspection program at all power reactor facilities.
We 15 anticipate providing this in a Commission paper, which I
~
16 believe Frank Gillespie, we'll issue sometime in January or 17 late December of this year.
The anticipation is that we 18 will provide it to the Commission in January 1999.
19 In establishing the project, as Pat Baranowsky 20 indicated, we also recognized that we needed certain 21 expertise on our inspection group.
This is a monumental
' ~
22 effort.
We have 12 members on our inspection group team.
l,,
23
.It consists of individuals from various groups.
We have two 24 risk experts.
We have a senior reactor analyst on the 25 group; we have individuals from Research.
We also have ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 i
Washington, D.C.
20036 I
(202) 842-0034 1
l
78 1
representatives of the senior resident inspector program, 2
those who are currently senior resident inspectors and those 3
who have been in the past.
We have representatives who are 4
currently regional inspectors in the program.
5 We also felt it important we have individuals who 6
are experts in each of the cornerstone areas.
For example, 7
we have a person with expertise in radiation safety, another
+
8 person with expertise in mitigating systems, and we also 9
have a representative from the Office of Enforcement on our 10 team.
11 With regard to the charter and deliverables, I 12 would first say, if you turn to slide 18, we divided the 13 charter and tasks to reach our end product of describing 14 this program into several key tasks.
15 The first one is to look at the scope.
We felt it 16 was important to first decide what the program should do 17 overall.
We discussed here today that the purpose of a 18 baseline inspection program is to achieve an indicator of.a 19 licensee's performance, that they are operating safely, but 20 there are two key pieces to that program.
I believe, 21 Commissioner McGaffigan, you launched us into that 22 discussion.
23 One is that we will emphasize risk-informed 24 inspections in areas where there are no clear performance 25 indicators at this point in time.
However, we will also ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
l 79 l
l 1
have baselina inspection program in areas where there are t
2 limited performance indicators.
That gets to some of the l
3 discussion we had earlier.
4 Another key piece of the program will be where we 5
do have performance indicators verifying that they are still 6
providing us with the indicated results.
7 As far as the question of sampling, I'll address j
8 that when I get down to the process attributes.
9 When you look at the scope of the program, we 10 first embark upon deciding what we want to inspect.
As I 11 indicated earlier, we are calling these inspectable items.
12 We are providing a complete set of those.
Our plan is then 13 to modify those, depending on the outputs from Pat i
14 Baranowsky's group whether there are adequate performance E
15.
indicators, and depending on the experience that we've had, j
16 The next step I have down there is a basis for the 17 linking those to the NRC mission on risk.
Let me give you l
18 an example of how we plan to do that.
19 If you take the cornerstone back on slide 7, 20 mitigating systems, the concept is similar to the improved l
21 tech spec documentation we have out there now.
We'll have 22 mitigating system.
You might have a characteristic that l,
23 that perticular item is functional.
In other words, it's i
l 24 capable of performing its intended function or design l
.25 function.
l ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C.
20036 (202) 842-0034
80-l 1
For each inspectable item we would list its basis.
21 Let me give you an example.
If you take post-maintenance.
- 3 testing as an inspectable item, as a basis for that we might j4 include why you would why you wouldLinspect post-maintenance 5
testing.
We are envisioning that we would have a 6
relationship with that to the cornerstone as to why it 7
produces a desired result.
We might talk about whether we 8
have a performance indicator in there'and what~our 9'
inspection program is going to show versus that indicator.
10 Most importantly, as John Flack indicated, we will 11 put in some risk information.
Right now it's only on L
12 concept, but we envision using some-kind of risk hierarchy i
l'3 to guide the inspector to what are the important systems or l'
I 14 components to look at when you are looking at l
l 15 post-maintenance testing.
16 I recognize that's only a concept, but we wanted 17 to give you that as an example of wh'at we are looking at as l
18 how we might link this to mission and risk.
19 CHAIRMAN JACKSON:
Don't you really want to look 20 at it kind of in a -- you could come out at the same place 21
-- converse way, where you would look at some system, and 22 you could ask if the performance indicator tells you what 23 you need about the system?
Then you ask your three 24 questions.
Is there something that the performance 25 indicator doesn't tell you?
If it turns out to be that L
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 1:
81 1
post-maintenance testing will tell you that, then that tells 2
you;what you are going to look at.
If the indicator tells 3
you what post-maintenance testing would tell you, you 4
wouldn't necessarily do it except insofar as wanting to 5
validate the indicator.
Is that correct.
6 MR. MALLETT:
That's correct.
7 CHAIRMAN JACKSON:
So you don't start that you i
8 need to do post-maintenance testing; you start with the 9
systems and then you ask, what do I need to know in order to 10 verify that the systems performs it's intended function?
l 11 MR. MALLETT:
When you are designing what you want 12 to look at in your program, that's correct.
You might also 13 use it in a different method.
If a certain event came up, 14 we envision you might go back and also use this as a way of 15 saying, do I need to even look further into this event?
16 If we move to the other items we put on here as 17 key items, address stakeholders issues.
We felt it was 18 important early on to talk to the various internal NRC 19 stakeholders and external to ses what the issues are that 20 they believe need to be addressed in a baseline inspection 21 program.
Once we have given the concept to paper, we intend 22 to go back and use this as a check list to say, did we 23 address all those issues?
24 John Flack mentioned some of those when he 25 discussed it on slide 15.
So I won't elaborate anymore.
l l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036
.(202) 842-0034 o
82 1
As far as process attributes, the next step is to 2
decide how do we use this process.
We have all these 3
inspectable items; we have their basis now; we've linked 4
them to our mission.
How do we use this?
How do we tell 5
the inspector how to use this?
l 6
We haven't got the answer to that question yet, 7
but some considerations we have is, how much inspection do 8
you need to do?
How often do you need to do it?
9 Another thing we have considered is how we are 10 going to put sampling -- we call it selecting inspectable i
11' items -- into the process.
Chairman Jackson, you asked that fo'mulated yet 12 question about sampling.
We don't have it r
13 but we do intend to include that in our description of our 14 program.
15 Another item we are addressing is how can you get 16 some generic risk information and also guide the inspector 17 to get specific risk information based on plant specificity.
18 How do you address type of plants, for example?
19 CRAIRMAN JACKSON:
Mr. Flack is going to tell you 20 that.
21 MR. MALLETT:
He's working with us.
We are using e
22 his group heavily.
Mr. Baranowsky is also going to tell us 23 some of that.
l 24 I also have some senior reactor analysts on my 25 group that are discussing with them and interfacing to t
i ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
_.m._...
83 1
provide that result.
2 The last item I put as a key issue.
We are also 3
benchmarking some other agencies to see what their programs 4
are, what they use as a baseline inspection program, and to l
l 5
see if we can learn any lessons from them, or issues.
6 You did ask one other question I would like tx) add l
7 one other comment to, about the skills.
One skill we do see 8
is the inspectors and managers are going to have to have
~
9 more understanding than they do today about risk information
{
l 10 and how to use risk information.
I don't know that that's i
11 necessarily a' skill,.but there may be some training involved 12 in.how to do that.
13 If there aren't any more questions, I would like 14 to turn it over to Mike Johnson, who is going to talk about 15 the assessment group and their project.
16 MR. JOHNSON:
Thanks, Bruce.
17 Slide 19, please.
18 I will discuss the role of assessment, the 19 deliverables, and finally, the team composition.
20 We envision that the role of assessment within the 21 oversight framework and based on the defining principles 22 will be to consider the results of licensee performance as 23 measured by the objective indicators and thresholds
.24 developed by the framework group and the information that 25 results from the implementation of a risk-informed ANN RILEY & ASSOCIATES, LTD.
l Court Reporters l
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
84 1
inspection program and other insights as developed by the 2
inspection group to arrive at a view of licensee performance 3
within the framework.
4 Then, based on the licensee's performance, the 5
' role of the process will be to identify appropriate 6
regulatory actions that range from^ conducting just the 7
baseline, up to and including issuing an order.
8 To communicate the_ assessment results along with 9
planned regulatory actions to. licensees, the public, and 10 other stakeholders.
11 To provide follow-up and to verify our regulatory 12 actions to ensure that they are successful.
13 And to provide a quality check and feedback, a 14 process for continuous self-assessment, to ensure that the 15 effectiveness of our other oversight processes, the 16 inspection process, the enforcement process, continue to 17 improve.
18 In developing the staff's final recommendation 19 that.we will provide at the'end of the year, we will 20 consider questions such as:
21 How do we integrate the information inputs from 22 each of the cornerstones?
23 At what frequency and over what interval will we 24 roll that information up?
l.
25 What will be the methodology where we compare the l'
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue,.lM, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
l
85 i
1 objectiva. insights or the objective indicators with the insights coming from the risk-informed baseline inspection 2
i 3
and other inspection and other insights?
4 What does that methodology look like?
5 What actions should be taken and what is the 6
process with decision criteria to allow us to determine the l
7 appropriate regulatory response based on licenser.
8 performance in a manner that is scrutable and p::edictable?
9 Because are concerned about scrutability and predictability.
10 How should we communicate the results of the 11 assessment in actions to the licensees, the public and other 12 stakeholders?
This will include issues such as how do we 13 provide an opportunity for licensee input and feedback as a 14 part of the assessment process.
15 What should be the relation between the assessment 16 process and enforcement?
As we talked earlier and as Jim 17 Lieberman will talk in a minute, we do recognize that there 18 is a relationship between the-assessment process and 19 enforcement.
So what should that relationship be?
How will 20 it work?
21 How should we phase in the recommended process 22 with' our existing processes, including the senior management 23 meeting and the other things that we do in terms of 24 assessment today?
25 And how will we measure the assessment process L
(-
l ANN RILEY & ASSOCIATES, LTD.
t Court Reporters 7
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 lI}
.., - _~ _
_-..m
86 l
1
' post-implementation to ensure that it meets our 2
expectations, to ensure that a year from now the process 3
that we have recommended and we hopefully are beginning to 4
implement does meet the success criteria that we laid out 5
for ourselves?
6 We believe that the assessment process will be of 7
great interest to licensees, the public, and other external 8
stakeholders; arguably,,perhaps of more interest than even 9
the inspection program.
10 Because the assessment process will provide the 11 primary communication vehicle for the agency on the 12 performance of utilities, it will have a great ability to 13 impact licensee activities, public awareness and confidence 14 in the NRC and its licensees.
And as we learned with the 15 SALP process, it could have a potential for unintended uses 16 and consequences.
17 The assessment process will also be of great 18 interest to internal stakeholders who will be the process 19 implementers.
20 Given the importance of the process to both 21 internal and external stakeholders, we assembled a task 22 group of experts made up of representatives from key 23 internal stakeholders, including the regional offices, who 24 will be the heavy lifters in the implementation of the 25 assessment process, as well as members from AEOD, NRR, ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
87 l'
Rosaarch, and the'~ Office of Enforcement.
2 Parti,cipants have implemented the previous 2
-3 assessment processes ~
Several participated in the'IRAP 4.
process and understand the challenges of. developing an 5-assessment process, and all have participated in the a
6 workshop or are members of the inspection group or the 7
framework group,' and therefore understand the philosophical
)
8 approach that we are embarking on and will-be in a position
~' '-
9 to ensure that the assessment group activities are properly integrated with the activities of the other groups.
10 1
11 Finally, as is important with.the other groups, we 12
.have.already conducted and plan to conduct sbveral-
-l 13 additional meetings.with the industry, the public, and other 14 stakeholders'in order to-get early input and involvement in i
'15 our development of the assessment process.
16 COMMISSIONER McGAFFIGAN:
Could I ask a practical 17 question?
-18 CHAIRMAN JACKSON:
Please.
19 COMMISSIONER McGAFFIGAN:
You talked about the 20-transition.
Is there likely to be an 'nnual briefing to the a
21 Commission on the four regional administrators' and the 22 director of NRR's view as to how the plants are doing?
Is l
23 that likely to still remain part of the process?
24 MR. JOHNSON:
First of all, let me preface this by l
25 saying that we haven't really talked about it and done the L
l' i
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 i
88 1-development.that would enable me to answer your question
'2 conclusively, but let me just tell you that it is our 3
feeling, based on the conversations that we've had in 4
staffing the group, that there would be some periodic 5
briefing of the Commission on the status.
l 6
COMMISSIONER'McGAFFIGAN:
I also assume that, i
l 7
based'on what I read of the stakeholders interactions, that 8
the watch list concept may go by the boards.
I'm just 9
gaming this, and I hope you guys do.some gaming.
If I'm an 10 enterprising reporter, how do I still~-- I've got the four 11 regional' administrators in front of me and we don't have a
.12 watch list anymore but we have the discussion list, namely, 13 the ones that they thought important enough to call to our 14 attention, assuming we don't have 15 or 20 minutes for each 15 of the 104 plants.
16 CHAIRMAN JACKSON:
We might.
17 COMMISSIONER McGAFFIGAN:
We are doing pretty well 18 today in time.
19 How do you end up having the trade press not 20 report'that last week plants A, B,
C, D,
E, and F were the 21 focus of the Commission's deliberations as they received the 22 annual briefing from the staff and you still have a watch 23 list?
24 MR. COLLINS:
I think the backdrop that we have to 25 keep in mind is that none of this should be new news to ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
--~--
89 1
anyone other than when either the agency believes we need to 2
take an action or we are confirming a licensee action.
Any 3
roll-up that we would take periodically would not be for the 4
purposes of " announcing" any action against a plant.
As in i
l 5
the past perhaps the senior management meeting that was the 6
context, this would be a review of where we have been at any 7
given point in time and have our actions been effective.
It 8
might be more of a status of what has previously been 9
announced and implemented rather than a decision-making 10 meeting.
11 COMMISSIONER McGAFFIGAN:
I don't know how you 12 control five Commissioners who are sitting here asking you, 13 or even your own regional administrators, for that matter.
14 Plant X looks like it's getting into some regulatory 15 difficulty.
I understand, Mr. Regional Administrator, you i
16 have some real concerns about X,Y,Z.
Is that going to be 17 off limits for discussion?
18 MR. COLLINS:
I don't pretend to control five 19 Commissioners.
I guess what we would have to do is 20 understand what the forum is.
I wouldn't envision that this 21 process is focused toward the staff.
The process is focused 22 towards having a mutual understanding of performance and 23 ensuring that there is an entity, preferably the licensee, 24 who is responding and reacting to those issues 25 appropriately.
If not, then we engage, we reinforce; if ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 F
u-T
90 1
appropriate, we act independently.
2 At that point, if we were to be in a meeting to 3
discuss licensee performance, I would expect the licensee to 4
be there discussing their performance and the reasons for 5
why their performance is appropriate or not, and for the 6
agency to be there to ensure that our actions are 7
commensurate with that.
That meeting, if a meeting is 8
warranted in that fashion, should not be delayed annually; 9
it should be conducted when it is appropriate.
10 Perhaps, in that context, the meeting we are 11 talking about is more to review the process itself than it 12 is to review licensee performance.
That's yet to be 13 determined.
14 COMMISSIONER McGAFFIGAN:
I believe Mr. Lochbaum, 15 who is going to speak in a few minutes, has suggested that 16 at that meeting we would focus, in his scheme, on which 17 plants are doing less well and maybe which are doing better.
18 I'll let him speak for himself.
But that there would be a 19 discussion of how plants specifically are performing as 20 opposed to how our process is working.
21 CHAIRMAN JACKSON:
You could argue that the one is 22 a test of the other.
You can't ask the staff how they would 23 control five Commissioners.
24 COMMISSIONER McGAFFIGAN:
No, but I think I can 25 ask the staff, I think you need to think about the gaming of ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
I 91 1
the= process.
All processes are gamed and you should think l
'2 about it.
3 CHAIRMAN JACKSON:
But presumably,.if one'gets at l
4 the. issue of surprises, or oneLis sailing along in good 5
shape and all of a sudden one drops off'the cliff, that's 6
the ultimate sense in which someone can " gar. t the process."
a 7
If it's an open, scrutable, continuously interactive and' 8-appropriate process with the licensees involved, it's not 9
new news.
10 COMMISSIONER McGAFFIGAN:
Most of our recent watch 11 list meetings haven't been new news either.
12 CHAIRMAN JACKSON:
Nonetheless, when we come out 13 with the list, everybody watches.
There is a balance 14 between having a process that is scrutable, objective, 15 risk-informed, and so on,-and the fact that the Commission
)
16 has to be inforned and it should be in an open process.
I 17 Mr. Lieberman, you're on.
18 MR. LIEBERMAN:. Turning to slide 21.
We've 19 already made reference to enforcement as part of the 20 development of the oversight process.
21 As Mike Johnson and others have said, to develop 22 the assessment process, we need to establish what regulatory 23 actions will be taken based on the performance levels of 1
24 licensees.
This will include consideration of the role of 25 enforcement in the oversight process and what changes, if L
l l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters
[.
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034 l
1
l
~
92 1
any, are nneded to be made in the enforcement policy.
1
-2 Specific enforcement issues that the staff :Us 3-considering in coordination with the oversight effort 4
includes developing better guidance for the thresholds 5-between minor violations and severity level 4 violations, 6
reviewing severity level examples and enforcement policy to 7
make them more risk informed, reviewing the process to
)
8 determine sanctions, and evaluating the role for regulatory 9
significance in the enforcement process.
l 10 As to the threshold between minor violations and i
11 level 4 violations, we've already mentioned that in the past 12 the enforcement process has set thresholds resulting from 13 the inspection and assessment process.
As part of the 14 integration effort, the threshold will be driven by the 15 needs of the assessment process.
16 As. to severity 3 9vels, the technical framework and 17 assessment efforts will provide insights as to what is risk 18 and safety significant for purposes of assessment.
These 19 insights should be considered in devs. loping the severity 20 level examples so that violations which are rignificant to 21 threshold issues are significant to enforcement and vice 22 versa.
23 As to enforcement sanctions, we will be 24 considering what changes, if any, should be made to the 25 process for assessing sanctions based on the levels of ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
.v
- -... ~
l 93 i
1 licensee performance.
2 Finally, the issue of regulatory significance be l
3 to be addressed.
By regulatory significance, I mean when i
4 the agency concludes that the significance of root causes 5
and the circumstances of. grouping individual severity level i
6 4 violations are greater than the actual potential 7
consequencer warranting their aggregation into a' severity i
8 level 3 prco;em.
9 The staff has not developed a final' position on 10 whether and how regulatory significance should be used in 11 the regulatory process.
Since regulatory significance is in 12
. essence an assessment effort, the staff is proposing that 13 the resolution of this. issue be deferred until it can be 14 integrated into the assessment process.
15 In the meantime, the staff intends to issue an 16 enforcement guidance memorandum and increase its oversight 17 of cases involving regulatory significance.
For example, 18 reactor cases involving escalated actions which now require 19 my approval, will require the approval of the deputy 20 executive director for regulatory effectiveness.
In
~
addition, we intend to continue the current efforts to 21 22 ensure t:.at each case considered for regulatory significance l
23 have a clear nexus to safety.
24 Apart from the adjustments to the enforcement 25 process, we have developed a proposal to address l
I i.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
94 1
non-escalated enforcement actions.
This should be delivered 2
to the Commission very shortly'.
3 "ne changes for non-escalated enforcement actions 4
have been coordinated with the oversight effort.
The 5
proposed changes will not prejudge the outcome of the 6
assessment and inspection improvements; it can accommodate 7
any needed changes.
8 COMMISSIONER McGAFFIGAN:
You are proposing to 9
postpone this paper on regulatory significance until the 10 assessment process is further along.
The next chart is 11 going to tell us the schedule.
How much of a delay are you 12 talking about?
Does it have to be already in place and 13 being implemented?
Is it sometime soon after January?
How 14 long do we wait to tackle this issue?
15 MR. LIEBERMAN:
I think, Commissioner, we will be 16 doing that in early spring.
We need to understand how the 17 assessment process works, and then we can complete the 18 effort on the regulatory significance.
Whether we can 19 implement it completely may be a' function of how long the 20 assessment process is being completed, but the concept we 21 should be able to present to the Commission after we 22 understand.the assessment process.
23 CHAIRMAN JACKSON:
Have you interacted with other 24 agencies that have an enforcement authority, such as FAA, 25 FDA, EPA, DOJ?
Have you interfaced with state agencies, ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
\\
\\
f 95 i
1 with local inw enforc:m:nt, with academicians who have done l
2 studies in criminal justice or civil justice?
Have you 3
looked at SEC?
I 4
MR. LIEBERMAN:
Some of all of that.
We've looked 5
at DOT; we have gone to FAA; EPA.
I've gone to training 6
programs and discussed issues with the academic community.
7 I haven't dealt that much wien the states, but I read of lot 8
of articles on enforcement in general.
So I understand what 9
other organizations are doing in the enforcement process.
'10 CHAIRMAN JACKSON:
Where do things stand vith 11 OSHA, with what they call cooperative compliance?
12 MR. LIEBERMAN:
The OSHA system is a little 13 different from the NRC regulatory systems.
OSHA doesn't 14 have licensees.
They have spot inspections.
They inspect a 15 group of potential safety concerns in an industry, say the 16 paper making industry in Maine.
Tney don't have the same 17 degree of oversight.
The analogy that OSHA has to the NRC 18 regulatory program on giving credit to self-assessment is 19 not exactly the same system.
20 COMMISSIONER McGAFFIGAN:
Is regulatory 21 significance used in other agencies?
Is there a category 22 that allows you to aggregate or do some of the thil.se that 23 the current enforcement policy allows?
24 MR. LIEBERMAN:
I'm not familiar with other 25 agencies using the specific term of regulatory significance, l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters l
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
96 1
but the concept of evaluating violations for the potential 2
safety significance is not unique to the NRC process.
We 3
came up with the concept of regulatory significance, the 4
grouping of violations, in part because of the history of 5
the civil penalty process.
Prior to 1980, our civil penalty 6
authority provided for a maximum of $5,000 per violation.
?
We almost had a cash register approach to enforcement back 8
in the 1970s.
There were so many violations and there was 9
so much money per violation and you added it up.
10 When we received the authority post-TMI to have 11
$100,000 per violation and we recognized that many 12 significant issues involved more than one violation, we came 13 up with a concept of grouping violations together and then 14 assessing civil penalties based on the groupings of 15 violations.
That's where the concept of regulatory 16 significance came along.
Other agencies have different 17 civil penalty schemes.
So there is not necessarily a direct 18 correlation.
19 CHAIRMAN JACKSON:
Okay.
20 MR. GILLESPIE:
Going on to slide 22, which is our 21 schedule.
Short-term actions are set to happen between now 22 and January of 1999, including public meetings more than 23 wcekly with NEI.
NEI has established two subgroups to our 24 framework and inspection groups to deal with radiation 25 protection and to deal with safeguards.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
97 1
We are hoping to have our proposal developed by 2
the end of November.
This will allow us to meet with the 3
ACRS subcommittees and ACRS full committee the first week in
'4 December so that we might get a letter from ACRS to the I
5 Commission on our proposal, and then to the Commission in 6
December.
7 Beyond January, with Commission approval and I
8 comment in the spring, develop revised enforcement guidance r
9 in the spring.
10 Start the phase-in of both the assessment and 11 inspection process in June.
12 Implement risk-informed inspection' baseline in 13 October.
I emphasize this is the risk-informed baseline,
{
14 because there are regional initiative inspections which will 15 not be. completely redone.
16 Complete the phase-in of both risk-informed 17 inspection and assessment by June of the year 2000.
18 And then a retrospective look one year later.
19
-Hopefully we will have established some credible objectives 20 when we put this place-and measure ourselves against those 21 objectives a year later.
--e-22 With that, we complete our presentation for this 23 afternoor..
24-CHAIRMAN JACKSON:
Thank ycu.
If there are no 25 further questions or. comments, let me thank the staff.
I 1
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 9-
+ >
~+
wM m
-- I'
98 I
will make some fuller remarks at.the end.
So you are 2
excused for the moment.
3 Let me, first of all, thank Mr. Beedle and Mr.
4-Lochbaum for their patience and invite you to please come to 5
the table.
6 Good afternoon.
7 MR. BEEDLE:
Good afternoon, Chairman Jackson, 8
Commissioner McGaffigan, Commissioner Diaz.
9 CRAIRMAN JACKSON:
We are pleased to have you.
We 10 are particularly interested in how you see the overall t
11 progress to reengineering the assessment, inspection and 12 enforcement program.
13 MR. BEEDLE:
First, let me echo Frank Gillespie's l
l 14 comments somewhere around two o' clock this afternoon and 15 wish you all good evening.
16 COMMISSIONER McGAFFIGAN:
It isn't sunset yet.
l 17 CHAIRMAN JACKSON:
It isn't sunset yet.
It's 18 getting close.
19 MR. BEEDLE:
Just a point of perspective.
The l
20 process that the staff described to you during the course'of 21 the last couple of hours is one that is shared by the 22 industry because it's going to help both the industry i
23 executives and the NRC staff focus on the things that are 24 important to safety in the operation of these plants.
In 25 doing that, it helps us assign our resources to the things ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
i 99 1
from a safety pr.ct of view, that are meaningful, and frees l
2 us from a regulatory burden on things that are not safety l
3 related.
One of the objectives of this process, at least 4
from the industry point of view, is to help focus on that as 5
opposed to the management of the facilities.
6 The four day. workshop was really'a very beneficial 7
workshop in that'it fostered a lot of communication between 8
the NRC staff and the industry and other stakeholders that 4
9 were present.
It helped us better appreciate the direction t
10 that the staff was moving on this particular issue.
So from l
11 an educational point of view, I think it was an immense 12 success.
1 13 Did we solve a lot of problems in the process of 14 doing that?
Perhaps not, but we did have an alignment that 15 was discussed earlier that there was a need to focus cn1 l
l 16 safety, and with that we could define parameters that would 17 help us understand that better, and'with that we could also a
18 define some thresholds that would give us the ability to i
19 then take a look at inspection and enforcement and properly 20 respond to that.
l' 21 Let me have the next slide, please.
,e 22 (Slides shown.]
23 MR. BEEDLE:
You asked earlier about data.
I'd 24 like to talk a little bit about that.
25 First of all, the nuclear officers in the i
I i
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 l
Washington, D.C. 20036 (202) 042 0034 l
100 1
community have agreed that the data is necessary for the 2
agency in order to determine where the performance of the 3
plant is, and it helps not only you, but it also helps the 4
people that are managing the facility.
5 Our thinking at this point is that that data would 6
be provided directly to the NRC in some sort of a formatted 7
process that would make it easy to digest and process, and 8
it would not involve pass-through through INPO and 9
perturbate the process that INPO has got.
So it eliminates 10 a number of the concerns.
11' CHAIRMAN JACKSON:
It would be direct.
l 12 MR. BEEDLE:
It would be direct.
Perhaps an l
13 appendage to the monthly operating report for each of the i
14 plants.
l 15 Would each of the plants participate?
I think 16 with reasonable assurance I can tell you that they would.
17 So I don't think that is really an issue.
Having defined up 18 front the parameters that we are talking about, I don't 19 think we are going to have any particular problem.
20 If the staff comes back and says we need six more 21 parameters, I think we might ask some questions and try and l
22 understand why.
If we could reach agreement on it, I think 23 all the plants would then provide those additional six 24 parameters.
25 We expect that the number of parameters for each ANN RILEf & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
m
-. _. - ~ _ _
l l
101 1
plant would be the same.
So we wouldn't have a group of 2
plant t that would provide four'and another group of plants 3
that wca 3 provide six.
I think we are going to look at i
4 consistency across that spectrum.
t 5
The three year trending curves that we have been 6
plotting would be plotted for each one of the plants.
l 7
Histograms would display worst value for each plant plotted i
8 against those indicators.
J
'4 l
9 PRA sensitivities would be run, and in the case of 10 scram and mitiaating systems, we are l'ooking at something 11 like'two times a CDF or a delta CDF of one times ten to the 12 minus five to set threshold.
13 Insights from the data that we have analyzed to 14 date.
The indicators do provide an overall perspective on i
15 safety performance.
That is certainly our assessment.
16 Barrier integrity indicators show strong plant 17 performance for almost all plants.
l 18-The initiating events and mitigation indicators 19 exhibit the most detectable vari'ations.
20 And the unplanned plant transient indicator 21 appears to be a reasonable leading indicator of plant i.
22 performance.
Indicators do not reveal any design control 23 problems.
24 I would remind you that this process is one l
25 focused on risk, but it does not exclude the fact that we ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 i
102 1
still have tech specs and regulations and design basis 2
requirements to adhere to.
So we are looking at assessment 3
process, not whether or not we in fact followed the 4
requirements of regulation.
5 Indicators do distinguish levels of safety 6
performance.
We see the excellent performing plants have 7
indicators that are high in the green band; the average 8
plants are in the low green to white band; and there does 9
appear to be declining trends that show in multiple
'10 indicators.
Recent watch list plants have several 11 indicators that show up in the white zone.
12 If I could have the backup slides, please, that 13 show the graphs.
14 COMMISSIONER McGAFFIGAN:
All of this information 15 is going to be docketed.
If it comes in the monthly 16 operating reports, it's public information.
17 MR. BEEDLE:
That's correct.
18 COMMISSIONER McGAFFIGAN:
So we can use it as we 19 see fit.
We can aggregate it, et cetera.
20 MR. BEEDLE:
You could aggregate it, but it's not 21 our intent that you would aggregate it.
We are trying to j
22 focus on_a plant's performance in those areas.
23 CHAIRMAN JACKSON:
It's going to be plant 24 specific.
25 M9 BEEDLE:
Right.
We are not going to give you l
ANN RILEY & ASSOCIATES, LTD.
j Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
103
)
1 aggregated information; it's plant specific.
1 2
In this one, I know that the graph is a little j
3 hard to see, but here are some plant transients and 4
unplanned shutdowns for a plant that has reasonably good
{
i I
5 performance.
You can see that the industry average is that 6
solid line.
The occurrence at this plant is the dotted 7
line.
So we've got a plant here that I think, by all 8
rights, would be concluded to be a good performer, and this 9
is what the operational challenges look like in plant 10 transients.
These are transients that create a power change 11 of greater than 15 percent.
12 This is a plant whose performance is trending in a i
13 downward direction.
You can see that trend developing.
14 This plant's performance has been cyclic in 15 nature, and I think this performance indicator indicates 16 that.
It was one that, coupled with other indicators, I 17 think you would have probably concluded should be on the 18 watch list.
19 Back to'the original set of slides.
20 COMMISSIONEF. McGAFFIGAN:
The data you have there 21 goes back five years.
.c 22 MR. BEEDLE:
Correct.
23 COMMISSIONER McGAFFIGAN:
You are getting three 24 year data for the plants.
Is that just a matter of 25 resources?
Do you think three year data will be enough to l
l ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
104 1
prove the point?
2 MR. BEEDLE:
This is data that we had started out 3
with sometime ago, when we were looking at this process.
We 4
think that the three year rolling average looks like about 5
the right data to look at.
I don't.think that even in this 6
one where you have that cyclic behavior that going back 7
another couple of years makes all that much difference.
8 In this slide we have some data that speaks to the 9
establishment of the thresholds just to give you an idea of 10 what we are looking in that area.
We are looking at the 11 base core damage frequency.
In plant A it's 1.47; in plant 12 B it's 4.6.
So we see a different range in there.
13 If we use a delta CDF at one times ten to the 14 minus five or 1-E to minus five, and then two times the CDF 15 as the threshold, the resulting behavior that you see would 16 be, in the case of scrams, the CDF was based on four.
You 17 could set a threshold using the delta of ten scrams, and if 18 you were going to use a two times, a doubling of the CDF, 19 you'd have to have something on the order of 14 scrams.
So 20 that would give you the range in that white band of, say, j
21 four to ten.
l 22 If you look at diesel unavailability and HPSI 23 unavailability, you get an idea of the sensitivity of this.
24 From where we typically see the plants operating, 25 to get a significant change in availability as measured by JJRJ RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
l 105 1
the core damage frequency, you are looking at a fairly 2
significant increase in unavailability. You really have to i
3 make it a point to have your system out of service for long-i I
periods of time before you start encroaching on a safety 4
5 limic in this case.
6 Plant B, where the CDF is a little bit higher, l
7 those ranges are slightly different but not significantly 8
different.
9 So there is a wide spread in this data threshold 10 that we would be looking at.
11 COMMISSIONER McGAFFIGAN:
Is the idea that we 12 would have different thresholds for different plants?
That 13 might be a'little difficult to implement.
Or is there a 14 single; threshold for a performance indicator that would work 15 for all PWRs'or all BWRs, or whatever?
16 MR. BEEDLE:
I think we are going to end up with 17 different thresholds, with some different parameters for the 18 BWR3 and PWRs.
19 COMMISSIONER McGAFFIGAN:
Is it a single threshold 20-for all plants, or does this analysis suggest that you have 21 a different threshold for each plant?
22 MR. BEEDLE:
I think it will be a different 23 threshold for each plant.
24 CHAIRMAN JACKSON:
What is consistent is whether 25 the trigger is a specific delta in core damage frequency or ANN RILEY & ASSOCIATES, L7D.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 i
i e
.p 106 1
two times or some specific multiple of core damage 2
frequency, or a change in the' base to cored damage 3
frequency.
That's the commonality of approaches.
Is that 4
correct?
5 MR. BEEDLE:
That's correct.
'6 We are currently working on the trend graphs and 7
histograms and would expect that a little later this week 8
we'll have those available for about two thirds of the 9
plants.
10 PRA sensitivity results will be provided sometime 11 later this month for a representative set cf plants, 12 approximately 25 of them.
So I think we are reaching some 13 consensus on what the data collection effort should be and 14 what those thresholds should be.
15 We are working closely with Frank and his various 16 task forces on tlis, and we think that we are reaching 17 agreement on some technical issues that help us understand 18 safety at the plants.
19 COMMISSIONER McGAFFIGAN:
One of the conversations 20 we've had with ACRS in the past is that we probably have 21 pretty good confidence on delta CDFs.
I'm still stuck on 22 this notion that we might have for plant A so many safety 23 system actuations that get you into the white zone and for l
24 plant B have a different number based on IPE that weren't 25 all done in a standardized way.
It's sort of taking my ANN RILEY & ASSOCIATES, LTD.
l Court Reporters 1
1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
107 l
1 breath away at the moment, i
2 CHAIRMAN JACKSON:
But the delta CDF --
3 COMMISSIONER McGAFFIGAN:
The delta CDFs I can 4
understand.
Delta CDFs is part of it.
But part of that was 5
two times CDF.
We were using the actual number.
6 CHAIRMAN JACKSON:
That's the point.
You have to l
7 settle on which of those is the acceptable metric, right?
S MR. BEEDLE:
Go back to the slide with the plant A 9
and B table on it.
In this slide we had, for example, 10 diesel unavailability of.61 percent.
I would argue that 11 any plant that is getting up into the ten percent 12 unavailability on its diesel engines is probably going to 13 wonder what's going on with its maintenance group.
I don't 14 think the manager of the facility is going to allow that.
15 Forget whether or not it's a regulatory threshold.
When we 16 first posed in the area of scrams that we set the green band 17 at the level of three, we had tremendous opposition on the 18 part of the industry.
They said, well, that's ridiculous.
19 We never have more than two.
Why don't we set it at two?
20 CHAIRMAN JACKSON:
The approach that is most 21 consistent with Reg Guide 1.174 is the delta CDF approach, a
22 MR. BEEDLE:
Right.
23 CHAIRMAN JACKSON:
At any rate, the delta approach j
24 is the approach in Reg Guide 1.174.
l 25 COMMISSIONER McGAFFIGAN:
But where he derived l
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 l
108 1
those numbers, I thought 2
CHAIRMAN JACKSON:
No.
If you put the viewgraph 3
back, it means that for that particular plant, in order to 4
have a delta of ten to the minus five, for that.particular
.5 plant _ theoretically it would require ten scrams, or diesel
~
6
-generator unavailability of about 30 percent, or HPSI 7
unavailability of about 14-1/2.
That's what that is saying.
8 COMMISSIONER McGAFFIGAN:
Therefore,- that's where 9
the threshold should be for the red zone.
10 CHAIRMAN JACKSON:
For that plant.
11 MR. BEEDLE:
That's where that bottom of the white 12 zone, start of the red zone would be.
13 COMMISSIONER McGAFFIGAN:
We are using delta CDF 14-there, but at the top, to decide where the green zone / white 15 zone interface is, the proposal is that we use the base CDF.
16 So its four for that plan for scrams and.61 percent and 17 1.81 percent, and then a different set of numbers for the 18-other plant.
So we are using the CDF itself as a mechanism 19 for deciding the green zone.
20 MR. BEEDLE:
I think in the case of the green zone 21 we are looking at perhaps CDF, but we are also looking at 22 some history of performance of the plants in the 1990 to 23 1994 range.
24 COMMISSIONER McGAFFIGAN:
Am I understanding you 25 for scrams, three across the industry?
We're not going to ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 L
(202) 842-0034
109 l
1 do three at one plant and four at another?
L 2
MR. BEEDLE:
In that case, we'll probably have 3
three across the industry.
4 COMMISSIONER McGAFFIGAN:
For diesels, rather than 5
11 percent versus 28 percent, it would be 10 percent across 6
the industry?
7 MR. BEEDLE:
I'm not sure, but I would guess that 8
the diesel would probably be somewhere in the one percent 9
range for the green band.
I can't imagine us putting it 10 down at 30 percent.
It may be 20 or something like that.
11 It demonstrates that there is a tremendous margin between an 12 operationally significant condition and a safety significant-13 one.
The risk insights that we have developed over the last 14 several years help us understand that every time you have a 15 wing nut out of position doesn't mean that the plant is-16 unsafe.
That's really what we need to focus on.
17 CHAIRMAN JACKSON:
Your point still remains about 18 having some consistency in approach.
What that consistency 19 in approach-translates into is a' fundamental question.
Is 20 it going to vary by plant, or do we want to just pick i
21 something and say that this is in fact the threshold?
I l
c-l 22 think that!s a regulatory decision.
23 MR. BEEDLE:
I think once we get all the data in 24 here, our task forces are going to look at that and say what 25 makes sense.
I think part of what makes sense also, you ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 u
Washington, D.C.
20036 (202) 842-0034 l
110 1
have to factor in your ability to regulate that and get some 2
consistency and standardization.
3 COMMISSIONER McGAFFIGAN:
I'm just trying to 4
anticipate.
Mr. Lochbaum at some point is going to pipe up 5
and talk about his views as to how good the IPEs are and all 6
that.
I guess he's waiting his turn patiently.
7 MR. BEEDLE:
I would also argue that as we look at 8
the spectrum of plants, when we see one that looks like it's 9
an outlier, we may have to take some special action in the 10 case of that one.
It will also point out difficulties 11 associated with some of the PRAs or IPEs that have been done 12 at the plants.
13 CHAIRMAN JACKSON:
Are you done?
14 MR. BEEDLE:
I'm finished.
15 CHAIRMAN JACKSON:
Mr. Lochbaum, 16 MR. LOCHBAUM:
Thank you for this opportunity to 17 comment on the NRC's initiatives in the area of inspection, 18 assessment and enforcement.
These important areas are the 19 foundation of the NRC's reactor safety oversight function.
20 It's vital that they be as effective as possible.
21 The staff mentioned the recent four day workshop.
22 I attended that workshop.
It wasn't as useful as it could 23 have been.
The structure of that workshop was such that it 24 would have been virtually impossible to result in anything 25 but alignment.
The breakout sessions and the cornerstones ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
. ~.
111 1
were determined well in advance of the workshop and really 2
could not have been changed by the attending stakeholders.
3 The workshop was, in my opinion, little more than 4
a dog and pony show that the staff could tell you today that 5
it had met with the stakeholders and had their endorsement.
6 In my opinion, those four days could have been better spent examining the pros and cons of NEI's proposed 7
8 assessment model and its regulatory scheme.
9 I have the following comments on the specific 10 items discussed by the staff today.
11 Commissioner Diaz already commented on one of the 12 concerns I had with respect to compelling cases.
13 I felt that the NRC staff in the past has had a 14 major flaw in its existing program and that there is a very 15 low threshold for compelling cases.
The staff, in my 16 opinion, should very rarely overturn indicator results.
I 37 agree with Mr. Gillespie that if you do overturn indicator 18 results, that also casts doubt on the validity of your 19 indicators.
That needs to be reexamined.
Basically, that 20 shouldn't happen very often.
21 In the same section, defining principles, the 4
22 staff said that the assessment process results might be used 23 to modulate enforcement actions.
We strongly feel that 24 enforcement actions should be based exclusively on the 25 severity of the offense.
Under no circumstances should ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
112 1
enforcement actions bn increased or decreased based on 2
assessment results.
3 We feel that a major flaw of the current senior 4
manage' ment meeting process, which doesn't seem to be 5_
addressed in the current plans of the staff, is that the 6
managers spend too much time deciding who's naughty and nice 7
and too little time figuring out what to do about the 8
naught-f ones.
9 The primary focus of the SMM process should be to 10 develop action plans to handle plants determined by an 11 objective assessment process to be performing badly.
12 The staff spoke about a risk-informed oversight 13 process to guide its inspections.
Virtually all of the 14 staff's efforts seem to be directed towards ensuring that 15 they look at the right areas.
The staff needs to spend more 16 effort on figuring out how to properly respond to their 17 inspection findings.
We remain baffled by the current 18 inspection process, which seldom triggers a scope expansion 19 either on the licensee's part of the NRC's part.
20 The staff conducts inspections of very small 21 samples.
The findings from those limited audits need to be 22 placed in context, but they are not.
We think that was the 23 problem at D.C. Cook.
The inspection that was done last 24 August and September revealed a problem that begged for 25 scope expansion that seemed late in coming.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
l 113 1
We are also disappointed that the staff hardly 2
ever asks the licensees to explain why they didn't find the 3
problems first.
After all, the licensees have the burden 4
for assuring that their facilities are maintained in 5
acce" dance with safety regulations.
When the staff has 6
evidence that a licensee may be shirking that burden, the 7
staff needs to find out why.
8 We hope that the revamped NRC assessment, 9
inspection and enforcement processes will at least obtain 10 one element, namely, the ability to occasionally call some 11 event or plant condition unsafe.
We feel that plants like 12 Millstone Unit 3 and D.C. Cook were operating unsafely for 13 years prior to their lengthy outages.
14 We are not asking the NRC to agree with us on 15 these cases, but it's crucial that the NRC have a line 16 between safe and unsafe practices and to occasionally 17 identify something as being unsafe.
Without such a line, 18 you can never really adopt a meaningful risk-informed 19 regulatory policy. Quite simply, if everything at every 20 plant is safe, you don't know where to focus resources and 21 attention.
Besides, it's very difficult for the public to a
22 understand why you could fine NU $2.1 million or AEP half a 23 million dollars for safe operation of their facilities.
24 Thank you for this opportunity to present our 25 views.
More importantly, we appreciate the fact that you've ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
114 1
undertaken these important initiatives.
2 CHAIRMAN JACKSON:
Thank you.
3-Let me ask you to kind of expand a little bit on 1l 4-what you mean when you say that so far all the efforts seem 5
to be looking at trying to decide what the right areas are 6
as opposed to more effort-on what to do.
7 MR. LOCHBAUM:
For example, Mr. Collins during the 8
early presentation talked about the annual meeting or what 9
conceptually that might anticipate.
In our view, that
'10
.should be talking briefly about plant performance or what 11 the NRC's assessment.is.
We agree that that's a backward 12 looking-thing.
There should be no new surprises in that 13 because'it's all based on available information.
But we 14 think that should be complemented by looking forward at what 15 the NRC is going to do the upcoming year to address any 16 weaknesses that have been identified.
17 We think the purpose of that meeting is twofold, i
18 One, to ensure that the public and all stakeholders know 19 what the NRC's current assessment of a plant.is, but also to 20 identify what the NRC is going to correct any deficiencies 21 or weaknesses.
If you have a series of these things every 22 year, or however periodic it is, if you keep identifying the 23
.same problem over and over again, that reflects on the 24 regulatory staff effectiveness as well as the licensee's 25 effectiveness.
It captures both.
ANN RILEY & ASSOCIATES, LTD.
Court-Reporters 1025 Connecticut-Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
115 1
We think too much effort is focused on grading the 2
plants and not responding to what those grades tell you or 3
what needs to be done to improve the low grading plants.
4 That's just an example.
We seem to see this in many cases.
5 COMMISSIONER DIAZ:
Have you done any work in 6
trying to define or bound what unsafe means?
Is that a 7
category of radioactive releases?
What are the things that 8
you would work with?
4 9
MR. LOCHBAUM:
I think the closest attempt that 10 I've made to addressing that question is the presentation 11 that I made at the NS meeting in Nashville this past summer.
12 What I advocated there was when a plant event or a 13 condition is found at a plant, the licensee should evaluate 14 the as-found condition, whether it's a single event or an 15 aggregate of many different problems, and look at that event 16 with all postulated design basis events, LOCA, loss of 17 offsite power, et cetera, and see whether the 10 CFR 100 18 limits would have been exceeded.
Starting from that point, 19 would the public have been jeopa'rdized had the events 20 occurred from that degraded point?
If not, then that event 21 poses relatively little safety risk.
It needs to be 22 corrected, but it's not a safety issue per se because the 23 public would have been protected even if the accident had 24 started from that point.
25 Occasionally you find that the 10 CFR 100 limit ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
116 1
might have been violated had the event occurred from that 2
degraded point.
That to me is a potentially unsafe 3
condition, and that is where everybody should be focusing 4
their attention.
Not the other ones, but the ones where the 5
public might have been harmed.
And also plant workers under 6
I think applying that standard to identification of 7
as-found conditions is the way to distinguish between safe 8
and unsafe.
9 COMMISSIONER DIAZ:
You say potentially unsafe.
10 So there is a potentially unsafe and there is an unsafe.
11 MR. LOCHBAUM:
I agree, but the public needs to be 12 protected even if the accident occurs.
So it is potentially 13 unsafe,'but I'm not sure in my mind that that's more than 14 just semantics or just a technical term, because had the 15 accident occurred at that moment, then the line would have 16 been crossed; the public would not have been protected, and 17 that can't happen.
18 COMMISSIONER DIAZ:
There is a difference.
19 MR. LOCHBAUM:
There is a. difference, right.
20 COMMISSIONER McGAFFIGAN:
What do you think the 21 prospects are for the success of this enterprise in terms of 22 defining performance indicators?
My recollection is that 23 you put out a report annually that uses a performance 24 indicator that is heavily focused on who identifies 25 problems.
My recollection is you gave Oyster Creek high ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
j 117 1
marks and other folks lower marks, and whatever.
That is l
2 your favorite indicator.
Or it's an indicator.
Have you 3
tried to insert that indicator into this process?
Does it 4
fit in any way?
Do you think we are missing things in this 5
NEI/NRC assessment process that is evolving?
l l
6 MR. LOCHBAUM:
We looked at performance 7
indicators.
When we provided the October 2nd comments on 8.
the IRAP that were submitted, we didn't include my favorite i
i l
9 indicator for the reasons that it didn't seem to be better 10 than the indicators that NEI was proposing.
I can't develop 11 NEI's indicators independently, so I didn't have access to 12 that information.
Had I had that, I probably wouldn't have 13 used the indicator I used.
14 I think the long-winded answer to your question is 15 I think NEI indicators are better than what I was using, and 16 I would prefer to continue using those.
With them being 17 public, I shouldn't have any problem doing that.
18 COMMISSIONER McGAF?IGAN:
Okay.
19 CHAIRMAN JACKSON:
If this process closes that gap 20 in terms of what the staff is going to do based on what it 21 finds, or what the NRC is going to do based on what it A
22 finds, would that address the major part of your criticism?
23 MR. LOCHBAUM:
I think so.
One of the things that 24 intrigues us about the NEI process is the trending.
When 25 you start getting into the white area, that's when the ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 7
118
~
~
1 regulator should get involved. 'The licensee will already be 2
trying to turn it around, but that's when the regulator 3
should provide whatever inducements'are necessary to ensure 4
-that it happens.
5 I am also encouraged by the fact that it doesn't 6
look like there is going to be a roll-up of however many 7
indicators there are into one global indicator of good or 8
bad.
I don't think that would have been entirely fruitful.
9 So it's good that it looks like it',s not going to happen.
10 I think the answer to the questio-is yes, that 11 concept seems to be the right way to address our concerns or 12 criticism.
13 CHAIRMAN-JACKSON:
Do you believe that the 14.
performance indicators and the risk-informed baseline 15 inspection program will cover the waterfront?
16 MR. LOCHBAUM:
No I think there will continue to 17 be surprises.
I don't think any process will ever eliminate 18 surprises, but I think we need to reduce the number of 19 surprises we have.
It looks like these initiatives will go 20 a long way toward reducing the number of surprises, and I 21 think that's positive from that standpoint.
?
22 CHAIRMAN JACKSON:
If I can paraphrase you -- you 23 can agree or disagree -- you're basically saying that the
~
24 missing element is what the regulator is going to do based 25 on what the regulator finds.
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
119 1
MR. LOCHBAUM:
That's right.
i-p 2
CHAIRMAN JACKSON:
And that somehow the work still I
3 hasn't, to your satisfaction, delineated safe from unsafe.
4 MR. LOCHBAUM:
I'm still wor':ied about the i
5 threshold about when the plants get shut down.
Mr. Beedle 6
pointed out that backward looking, some of the data showed 7
that the watch list plants, some of the indicators moved 8
into the white zone.
It didn't look like any of them moved 9
into the red zone, which would have necessitated a plant 10 shutdown.
Some of those watch list plants were shut down.
11' Was that a right decision or a wrong decision?
12 I need to go back and look at that.
I haven't 13 done that, so I don't know the answer to whether this 14 process would solve that.
15 The one thing I was concerned about in the staff's 16 presentation was about if you are operating in the green 17 zone and a violation comes up, do you not overlook that, but l
18 do you give the licensee credit for that?
We are kind of l
19 against that.
We think that process could lead to more 20 surprises because you tend to dismiss early indicators of 21 problems until it becomes so bad that several indicators go 22 into the white or things get so bad that you get the 23 regulatory bag brought out.
We are kind of concerned about 24 that.
We think all sanctions should be equal, depending on 25 the offense, no matter zone you are in at the time.
i T
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202)_842-0034
120 1
CHAIRMAN JACKSON:
Do you have a comment?
2 MR. BEEDLE:
We!ve talked a lot about what is the 3
regulatory response.
I would remind you that we still have 4
tech specs and rules and regulations to follow.
This 5
assessment process does not set any of those conditional 6
requirements aside.
7 The question, I guess, is really how s4.gnificant 8
is the violation.
I think we are headed trward a process 9
that would help us understand that, and that would then 10 determine what sort of reaction the regulator would take in 11 response to the violation.
I think that's really what the 12 whole point in this assessment process is all about.
13 CHAIRMAN JACKSON:
Are you operating from the 14 perspective that the assessment process in the end should 15 never lead to specific regult.cory action?
16 MR. BEEDLE:
No.
I'm saying that the assessment 17 process would help the regulator understand how to treat the 18 violation.
19 CHAIRMAN JACKSON:
No, no, no.
Let's leave aside 20 violations.
I'm talking about general performance.
I think the general performance would l
[
22 be dictated by the performance indicators.
I 23 COMMISSIONER McGAFFIGAN:
Do either of you have I
i 24 any concerns about the process whereby we are going to try 25 to integrate these objective performance indicators with I
ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034
m m-121 inspnction findings in the areas where the pe'rformance l
1 l
indicators are not going to provide useful information, and 2
3 then any other inspection findings that we come across?
Do L
you have any suggestions as to how to make that process more 4
5 scrutable or transparent?
6 MR. BEEDLE:
I think there are a number of areas 7
where these performance indicators are not going to tell you 8
about the compliance of the facility to the rules and 9
regulations, and I think those inspections and part of the 10 core inspections and baseline inspections that they 11 discussed earlier.
I think when you find problems as a 12 result of those inspections to supplement the performance 13 indicators, again the test of significance is whether or not 14 they have created problems from a safety point of view.
15 CHAIRMAN JACKSON:
Right, but if it's risk 16 informed in the first place, presumably one is looking at 17 the --
18 MR. 3EEDLE:
It would help you determine where 19 your inspection effort would be devoted.
20 MR. LOCHBAUM:
I would agree certain inspections 21 have to continue because they are not covered under 22 performance indicators, like human performance and training.
23 They have to have a fitness for duty that falls in that l-24 category as well.
25 I also think this whole process, I. don't know i
e ANN RILEY & ASSOC 1ATES, LTD.
Court Reporu.rs 1025 Cennecticut Avenue, NW, Suite 1014 Washington, D.C. 20036 (202) 842-0034
~
122 l'
whether it's allocated equally among assessm:nt, inspection, 2
and enforcement, but collectively there is a greater
-3 emphasis on corrective action programs of licensees.
I 4
think that greater emphasis will do more to ensuring there F
5 are no more-surprises than anything else.
The key 6
. difference between good and bad performance is the adequacy 7-of their corrective action process.
Everything I've heard 8
from'all three components is to ensure that that is a good 9
corrective' action process.
10 CHAIRMAN JACKSON:
Commissioner Diaz wanted to 11 make a comment.
'12 COMMISSIONER DIAZ:
I just want to'make a comment.
13 Without preempting the Chairman, I really want to express 14 how good I feel about what is going oh.
I think that the 15 staff has made a very valiant and a very intellectual effort 16 to get out of the box and think ahead and provide us with a 17 risk-informed framework that will serve this country better.
18 I want to thank also the industry and the stakeholders, and 19 Mr. Lochbaum.
This has been a fast and furious, but it has 20 been a very good process.
I am very encouraged by the 21 results and I look forward to see you all soon again to 22 finaiize it.
Thank you.
23 CHAIRMAN JACKSON:
Thank you.
24 Let me just say on behalf of the Commission, I do 25 commend the staff, NEI, all the stakeholders, for working ANN M.TLEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C.
20036 (202) 842-0034 f
123 1
together in developing improvements to the plant assessment 2
and oversight process.
I.would urge that if it is true that 3
others are provided with a fait accompli that there be more 4
real opportunity for participation and influence.on the 5
process, because in the end there are many stakeholders, and 6
we have to ensure that the public or the public surrogates 7
have an opportunity to be involved.
8 As the staff itself;has pointed out, although we 9
have made significant progress, and I am commending you for 10 that, much work remains to be done.
The decision 11 thresholds, for instance, for increasing regulatory 12 attention and regulatory action have to be well conceived; 13 they need to be benchmarked against historical experiences
)
14 and as easily implementable as possible.
So I would urge 15 the staff to stay focused, and all the stakeholders to stay 16 focused on those principles.
17 I would like to thank two of our stakeholders, Mr.
18 Beedle from NEI and Mr. Lochbaum from UCS,'for your 19 comments.
I appreciate the thoughtfulness that you put into 20 them.
They will be very halpful as we go forward.
21 Unless there are further comments, we are 22 adj ourned.
23
[Whereupon, at 5:00 p.m.,
the briefing was 24 concluded.]
25 ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1025 Connecticut Avenue, NW, Suite 1014 Washington, D.C. 20036
-(202) 842-0034 m
v-
CERTIFICATE This is to certify that the attached description of a meeting of the U.S. Nuclear Regulatory Commission entitled:
' TITLE OF MEETING:
BRIEFING ON REACTOR OVERSIGHT PROCESS s
E IMPROVEMENTS PUBLIC MEETING PLACE OF MEETING:
Rockville, Maryland DATE OF MEETING:
Monday, November 2, 1998 was held.as herein appears, is a true and accurate record of the meeting, and that this is the original transcript thereof taken stenographically by me, thereaftsr reduced to typewriting by me or under the direction of the court reporting company Transcriber:
Reporter:_ Mike Paulus l'
Industry Remarks at Commission Briefing on Plant Assessment Process November 2,1998 QEI OVERALL INDUSTRY VIEW
. Four-day assessment workshop was well run and was successful in reaching consensus on revised assessment process.
. Staff efforts to date appear consistent with workshop principlDs.
. Industry is encouraged that staff is heading towards a more objective and safety-focused oversight process.
'V' 1
INDUSTRY DATA ANALYSES
= Three-year trend curves being plotted for each plant for all proposed indicators.
a Histograms displaying worst value for each plant being plotted for each indicator.
= PRA sensitivity studies being run for Scrams and mitigation system indicators.
. 2 times CDF
-m INSIGHTS TO DATE FROM DATA ANALYSES
= Set ofindicators provide an overall perspective of safety performance.
l
= Barrier integrity indicators show strong performance for almost all plants.
e Initiating events and mitigation indicators i
exhibit the most detectable variation.
8 L
= Unplanned plant transient indicator appears most " leading".
= Indicators do not reveal design control problems.
QEI 2
INSIGHTS TO DATE FROM DATA ANALYSIS
= Indicators do distinguish levels of safety performance:
. Excellent performing plants have indicators high in the green band.
. Average performing plants have indicators in the green band with an occasional dip into the white band on one indicator.
. Declining trends show up in multiple indictors.
. Recent watch list plants had several QEI indicators in the whi.e zone.
4 Planta CDF Scrans _EDG HPCI Unavait Unavail Base CDF 1.47 E 5 4
0.61%
1.81%
Delta CDF 2.47 E 5 10 29.9 %
14.5 %
=1F,5 I
2X CDF i 2.94 E 5 i 14 i 40.3% i 18.1 %
Plant B e
Unavail. Unavail.
Base CDF 4.60 E-5 3
1.86%
0.59%
Delta CDF 5.6 E.5 10 11.2 %
18.3%
= IF 5 2 X CDF I 9.2 E.5 34 1 48.4 %
80.2%
EI
.I
SCHEDULE FOR PROVIDING DATA RESULTS TO NRC STAFF Trend curves and histograms will be
=
provided 11/4/98.
. Data available from 2/3 of plants PRA sensitivity results will be provided
=
11/12/98 for representative set of plants.
. Approximately 25% of plants QEI a
f S
f 4
i.
l d>R REa%.
?
p g
4 o
REACTOR OVERSIGHT PROCESS IMPROVEMENTS:
INSPECTION, ASSESSMENT, & ENFORCEMENT j
NOVEMBER 2,1998 FRANK P. GILLESPIE - NRR j
PATRICK W. BARANOWSKY-AEOD JOHN H. FLACK-RES BRUCE S. MALLETT-REGION ll MICHAEL R. JOHNSON - NRR JAMES LIEBERMAN - OE I
'1 1l1 l
l EN IL T
dn s
U u
p o
e O
r t
g s
s G
k u
2 t
c t
x a
N a
t e
B S
N IFE e
e e
IRB
g 9-t i
-BACKGROUND 1
Integrated Review of the NRC Assessment Processes for Operating Commercial Nuclear Reactors (IRAP) i Nuclear Energy Institute (NEI) white paper
)
i
- Challenges l
- preserve core values (safety, Principles of-Good Regulation)
- improve objectivity, scrutability, and predictability
- retain the ability to provide strong focus on j
licensees ~with significant performance problems
- single recommendation i
I l
APPROACH e Develop framework e Identify and address key issues e Develop oversight processes i
e Maintain link to changes in ongoing programs j
(SALP, PPR, SMM, enforcement, inspection, resident policy, etc.)
{
'l 4
L A
Y
. STAKEHOLDER INVOLVEMENT t
1 60-day public comment period on IRAP (pubhc, i
i l
States, public interest groups, licensees, NEI) l Public meetings with NEi with opportunities for all stakeholders to participate l
- Meetings with ACRS 1
Internal meetings (headquarters and regional personnel) i 4-day workshcp (public, States, foreign governments, public interest groups, licensees, industry, NEl, U.S. Senate rep., GAO, other government agencies, OlG, NRC) i 5
t i
\\
WORKSHOP ACCOMPLISHMENTS e Consensus on framework and objectives e Alignment on defining principles
- 2 significant issues remain: integration of data, nature of program: voluntary i
- these issues will be discussed later in the presentation e Progress on process development l
- performance indicators
- inspection areas t
- how to select thresholds i
i s
. _ _ _. -.. _ ~ _
u o
e 1
i
- NRC's PUBLIC HEALTH AND SAFETY Overen AS A RESULT OF CIVILIAN safety HUCLEAR REACTOR Mk*l" OPERATION i
Strategic REACTOR RADIATION Performance SAFEGUARDS SAFETY SAFETY Areas t
l net 1ATING IdlTMBATION SARfuER ERIERGENCY PHYSICAL Cornerstones N
OCCMTIONAL i
EVEm s SYSTEtts INTEGRITY PREPAREDNESS PROTECTION II i
- - - - - - - St#ETY CONSClOUS WORK - - - - - - - - - - - PROBLEM - - - - - - - - - - - - HUMA N - - - - - - -
ENVIRONMENT IDENTIFICATION AND PERFORMANCE RESOLUTION
- FERFORMANCEINDICATOR
- INSPECTION
- i
- OTHERINFORMATION SOURCES
- DECISION THRESHOLDS I
7
[
t I
DEFINING PRINCIPLES There will be a risk-informed baseline inspection program that establishes the minimum regulatory interaction for all licensees Thresholds can be set for licensee safety performance, below which l
e increased NRC interaction (including enforcement) would be warranted i
i Performance indicators (Pis) (supplemented with some inspection) will e
form the rebuttable presumption for licensee assessment The risk-informed baseline inspection program will cover those risk-l significant attributes of licensee performance not adequately covered i
by Pls j
l The risk-informed baseline inspection program will also verify the i
e adequacy of the Pls and provide for event response l
If risk-significant inspection findings and other information sources i
indicate that the PI results do not accurately portray licensee safety performance, the findin5s and information may be used to develop a j
compelling case to overturn the indicator results j
i 8tt l
l l
I l
c.
7 I
i 4
1 DEFINING PRINCIPLES (continued)
Enforcement actions taken (e.g., number of cited violations, amount of civil penalties) should not be an input into the assessment process. However, the issue itself that resulted in the enforcement action will continue to be an input.
Assessment process results might be used to modulate enforcement actions.
1 i
8b l
o m
[
SIGNIFICANT ISSUES
- Integration of Pls and inspection results
- No clear consensus on how (but, consensus that Pls and inspection results should not be artificially merged / integrated)
- Current PPR and SMM processes could perform this function, but objectivity and scrutability would continue to present a challenge e Voluntary reporting program is preferable to rulemaking
- Implementation issues (mechanism, frequency)
- Policy issues 9
l I
REGULATORY OVERSIGHT
. inspection
- Assessment e Enforcement 0
10 e
c
c L
i i
t LONGER TERM f
?
l e
e inspection documentation e Threshold for inspection findings and minor violations
- Allegations t
- Output to enforcement i
i
- Reporting l
5 l
l s
i 11 i
4 4
i
r SHORT TERM PROJECT STRUCTURE
- Project Activities r
- Technical Framework: Pat Baranowsky, AEOD
- Inspection: Bruce Mallett, Region 11
- Assessment (process framework):
l Mike Johnson, NRR j
i
- Enforcement: Jim Lieberman, OE i
e Integration
- Representatives from regions, NRR, OE, RES, and AEOD are participating j
12 j
e a
TECHNICAL FRAMEWORK GROUP
- Group leader: Pat Baranowsky
- Team composition I
- Charter / deliverables i
13 l
4
TECHNICAL FRAMEWORK GROUP -
CHARTER
- Comerstones
- PerformanceIndicators e inspection Bases
- Thresholds
- Enforcement Philosophy 14
~
~ KEY ISSUES RELATED TO RISK-INFORMED OVERSIGHT PROCESS l
- Generic v.ersus plant specific e Consideration of risk in inspection and decision making
- metrics l
- criteria I
e Treatment of items not modeled in PRA e inspection frequency and performance indicator updating i
15 I
J
1 FORMULATION OF A RISK-INFORMED OVERSIGHT PROCESS
- Identify and 'prioritize sources of risk and link to Cornerstones l
t
- generic PWR and BWR insights
- plant specific insights l
- Identify attributes associated with risk insights
- Link performance indicators and inspection j
activities to risk l
- Preliminary work completed f
16 I
=
e l
i i
INSPECTION GROUP i
i e Groupleader: Bruce Mallett i
e Team composition e Charter / deliverables e
t l
4 17 I
f
INSPECTION GROUP - CHARTER e Scope of risk-informed baseline program i
e Basis for inspection linked to NRC mission and risk e Address stakeholder issues e Process attributes
- Depth / frequency / resources
- Generic / site-specific
- Input to assessment / enforcement i
e Benchmark against other government agency inspection programs t
i i
18 l
a l
l O
V
~6 a..-
~m...--
--~..-*-+<--4..+_4~,mm~.
_.a..e&
4
+4.*u.+s.--%..-s us a4 a_Am_ _ _ _
.,..a m 4. a
.e4=.._.u4u-maa.4__4...
I j
i 4
s i
E L"
O ils W
E c
4 0
c E
i E
o g
M U) CL, x
2 c
l WD E
S o
e m
i OE
.B
~
o
=
a o
EG m
E 1
2 E
o o,
e o
v h
i z
-c i
W G
O F.
l<
E i
m is U) i W
M U) f 4
1
)
i e
i I
l ASSsSSMENT(PROCESS FRAMEWORK)
I t
GROUP-CHARTER i
l e Integration methodology i
l e Action i
l i
e Communication e Follow-up/ resolution l
l e Tie to inspection 1
e Tie to enforcement i
20 i
h l
t l
l t
i i
ENFORCEMENT ISSUES i
i t
e Enforcement philosophy l
- Other enforcement issues r
i h
i i
l l
i 21 i
i e