ML20127B100
| ML20127B100 | |
| Person / Time | |
|---|---|
| Issue date: | 01/08/1993 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | |
| References | |
| ACRS-T-1946, NUDOCS 9301120171 | |
| Download: ML20127B100 (64) | |
Text
_.
]
f, R,g z y " W
" 3;9l /,
PM,F $a6 @. '
m
's 9
.v. c th L
?
l j.
j*i T, Y. / ~
s i,, (,%,
e m
M, %m #x,
~ J ;. ; Q\\
\\
, q
() %
~
g m~
a v "
'm (v. '
r--)-M 4
2 2
g
~ m W
I
}
s
.c u:.
n-qs i> +'
vi ;:z-.
?
w 4
y 1
a i
'v>v g, ?.
,1 M -9:w a w Oj MT-i
" C+
TR04 (ACRS) e c
m'o $mC%
RETURN ORIGINAL TO
. (W i-B.J. WHITE, ACRS-P-315 f
y j'Qfy ik.
JW,-
WD &
. a
. 99 a
a:
'N THANKS!- barbara Jo 3' :c
~'.7W Ji i
u.
g w*
w - m g~ >
g,;
e
- 27288 os
'w e
(Q^
, va s w" a,
--m-
&fk;.j_
9a
.y 3 g DJ g,
M*
,f a[&
+
, qmx -
- k N_ ',
s 4
.w N'.n -
_],li
(,.,
((N
.[)[ f w
Agency:'
r
,n t u.s.swuclear neguietory;comission(
,1
%j ;;pl " ' >( p :a p
Q Advisory Comi.ttee On = Reactor? Safeguards,?
n
.n.,
1 w
4g 4
iQ e
~:Q m~ = ~k e l
t' w,0^j.
,+ i
/1 TitICI 393rd ACRS Meetingi
$ M M,N sg.
c
^
N,O p,
,.m lc, den %.+
a c U
~.
e
)
Q3 dx, Q g
. ;7 g
4 7/4
, iqL v,(:
36 ;
u s
..a.-.
n, -s m
..o 1
s
$ (.g %-
+
- w 3
DOCktt NO.
l lz
?r Op g9i WWW
. ',.. ',,' elfy 3 9 f. A
. M }.3
- 'qI;,t-O w.
.p._
3 y,, O,
3 4x
} '{ y
'..:- y::p;m.;.?;yk) 3 l
3 y
gg u,
s R ;,P
^-
i, e
<p:
.}{.
yL
+1A p
a g
i.
3N H
nv
.a. ~ n r.
,1 my,t bg g n
'i. >
n 1
y w4 pp
- t:
' J, i.- N 0,
'y w ! ' memy.
~"7 4
Y
'R.
. '; h,?
5 ".w&
~
r
%T
.y'..
m u !
ki w
d
,.s,
- e j
mL
-~
KO' p.?l t
8 y--
- w. '
,3 t
',yy eg&jk a
s 3
,y,,
Mfg y
, '4 '$
geg{74 Q
5
- y
--, g t
tocm0M fBethesdaD..,-Maryland? '
a,_ m e, %g.
i A
% % n-JG '
, f--"
9 i
y
- 1. r
{
y' gDi ?, t 9
b,,. M. ( hl'q[~ i
- ! d y D.K ',.
S [ ~ 9 '-[j )v 7 h ' '4 T>
A I.. (,1
.%y4 MC L
)
A
- 3 M.
1, 6P C
4-e s
s-ws a m
%e
- IXIC.
f.,
% M9 9:e
,t EFridsy JJanuaryj8C19933 >
m )PAGES,MI{7%y..m ]. < h~k..M 19 N t
y
- . ' ! t
[+
my mu
- if
.g.
dT t lJ t
.p /g, f d
g-.H '
tQ Qfg.fg 1~
+
p q-1 a--
de, ((
)
p'....
tr>
24 gge Q.d,M*4-Q s
,1, i.W 5, b
%/ /..
ic f*g q 4[ 3:.Jy s
/
4
-',%r g3
-w-
- . s ye,
,Jn t,
%' [.
e I i f
,W v 4 4....(h'a t,7..%.r, t
o 1
4L -
h
/
Si m"
/
g.
4 iS 2 s \\g.
+., g^ ;3:y
- l )
K,:;+s v,
.;p
,M
.ex e
[Q g
p y.
- u
_ Qg:
>j h f;y3 s
1
_ 30;7
-q c
y y:
i 4
g 33 QQ;K Q 4 x y n n Q;t &' t yM n
k
' y symyq.a $~argy " ~u< s~n
=1, W '.
W n.
nn M
9 ph 3 y y
A 2 p gf w)3 w
wwgnmy% m
- ~
w' wy; gm gy g.
- w,
%Qm y
gp, >.sN A y 6.y as 4
u
=
,m my 1
L4 :
g ;J M
aa gggg;a!g4 aq-
-mL+
gwmAuees,m.qqeg p_q e
+
y g + g_N
^]&$
- '^,'
? % ?QOY g;
- h;
- &
s i
N-Se=;
ANN MUiY&ASSOCWESED1
~b 9_d _J 11612 x st. nw.suke 3oo f '
f_ _i h[
he n
i l1 7 s f(202) 295-3950; 9301120171 93010s
"#
- W
- @L h
/ N 1MW
&.V >, #
%E gn T-1946 PDR Ma i m,,, w / >da; ' '-wwmn 7
n h.5 M,%,.
.n u,
^
o m. g: n
' w"M C
r ies og a
o.
a 11 -
y e
2
3%GW/L l O FFICIAL TRANSCRIFT OF PROCEEDINGS
&ll
/ TR04 (ACRS)
RETURN ORIGINAL TO B.J. WHITE, ACRS-P-315 THANKS!- barbara Jo
- 27288 Agency:
u.s. Nuclear negulatory Cottaission Advisory Connittee On Reactor Safeguards
Title:
393rd ACRS Meeting Docket No.
'O LOCATION:
Bethesda, Maryland dam Friday, January 8, 1993 PAGES:
177 - 219 3 n ~ m ty:p,,
35)
!,s l'ie ! ml~ m i ( ' 1 c.,.
4 P.
,;c, y*
c,.;. n,
[,, Us,-
t ;i 1 <. :. ':
idt lma, Mjg 9Q s
% ; ;,e,g ;;;;;.,.
. v
.s.
om 120023 ANN RILEY& ASSOCIATES, LTD.
(O 1612 x s
. s* 3*
Washington, D.C 20006 (202) 293-3950 9301120171 930108 PDR ACRS T-1946 PDR
= -
^
PUBLIC NOTICE BY THE UNITED STATES NUCLEAR REGULATORY COMMISSION ADVISORY COMMITTEE ON REACTOR SAFEGUARDS DATE:
Friday, January 8 1993 O
The contents of this transcript of the proceedings of the United States Nuclear Regulatory Commission's Advisory Committee on Reactor Safeguards, (date)
Friday, January 8, 1993
, as Reported herein, are a record of the discussions recorded at the meeting held on the above date.
This transcript has not been reviewed, corrected or edited, and it may contain inaccuracies.
ANN RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
177 b)
'l UNITED STATES OF AMERICA v
2 NUCLEAR REGULATORY COMMISSION 3
ADVISORY COMMITTEE ON REACTOR SAFEGUARDS 4
393rd Meeting 5
Nuclear Regulatory Commission 6
7 7920 Norfolk Avenue 8
Bethesda, Maryland 9
Friday, January 8, 1993 10 11 The meeting convened, pursuant to adjournment, at 12 8:30 a.m.,
P.
Showman, Chairman of the Committee, presiding.
13 Members Present:
O (f
14 P.
Showmon 15 J.
Carroll 16 I.
Catton 17 P. Davis 18 T.
Kress 19 H. Lewis 20 W. Lindblad 21
.C. Michelson 22 E.
Wilkins 23 C.
Wylie 24.
Designated Federal Official:
25 E.
Igne OO ANN-RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K.- Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950-
178 O
1 PROCEEDINGS
%J 2
MR. SHEWMON:
Good morning.
This is the second 3
day of the 393rd meeting of the ACRS.
During today's 4
meeting the Committee will discuss or hear reports on 5
proposed revision of NRC performance indicator reports, 6
subcommittee activities, which I guess means planning and 7
procedures, activities of ACRS working groups, proposed ACRS 8
reports.
9 The meeting is being conducted in accordance with 10 provisions of the Federal Advisory Committee Act.
Mr. Igne, 11 on my right, is the designated federal employee for the 12 initial portion of the meeting.
13 We have received no written statements or requests 14 for time to make oral statements from members of the public 15 regarding today's session.
16 A transcript of portions of the meeting is being 17 kept.
It is requested that each speaker use one of the 18 microphones, identify himself or herself, and speak with 19 sufficient clarity and volume so that he or she can be 20 readily heard.
21 The first item we have today is performance 22 indicators.
23 MR. BARANOWSKY:
Good morning.
My name is Pat 24 Baranowsky.
I'm chief of the Trends and Patterns Analysis 25 Branch.
We are responsible for conducting the performance b
ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
179 1
Indicator program.
I have with me today Tom Novak, who is 2
our division director, and Don Hickman, who is chief of the 3
performance indicator section and who will be giving the 4
presentation, and some other staffers from our branch who l
5 might assist in answering some questions.
6 On December 23 SECY-425 was sent to the Commission 7
with a recommendation to implement some enhancements to the 8
performance indicator analysis and display of information.
9 Pending approval by the Commission, we plan to implement 10 those enhancements for the first quarter 1993 performance 11 indicator report.
12 Today Don Hickman is going to give a summary of 13 the information provided in that SECY paper.
He'll include 14 a short background on what the program is about and its 15 history.
Primarily he will discuss proposed enhancements 16 and the changes in the displays in the reports that we plan 17 to implement.
18 With that, let me get Don Hickman to give the 19 presentation.
20
[ Slide.]
21 MR. HICKMAN:
Good morning, gentlemen.- As Pat 22 said, I'm Don Hickman, the chief of the performance 23-indicator section in AEOD.
I'm pleased to have this 24
-opportunity to'present to you the latest improvements to the l
25 performance indicator program that we have just recently
' O ANN RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950-
l 180 gQ 1
sent to the commission for approval.
2
[ Slide.)
3 MR. HICKMAN:
The performance indicator program is 4
a coordinated overall NRC program under the direction of 5
AEOD.
6 The program has been producing quarterly reports 7
since February of 1987.
The first report included two years 8
of data for cc.lendar years 1985 and 1986.
9 The addition of new indicators or major changes to 10 the program are subject to Commission approval.
13 Because of the potential for misuse of performance 12 indicators, there have been restrictions placed upon their 13 use.
These restrictions are contained in Announcement 200, 14 signed out by the Executive Director of Operations.
I've 15 included a copy of that in the attachment, the yellow pages 16 attached to your handout.
17 MR. LEWIS:
I hate to interrupt you so early in
~
18 the morning, because many of my friends aren't all here yet, 19 but you said this is a program which is coordinated; it's 20 throughout NRC; it's under the direction of AEOD.
Does that 21 mean AEOD commands some resources in the rest of NRC outside 22 of its own division?
23 MR. HICKMAN:
AEOD uses as a source of the 24 performance indicators the licensee event reports which are 25 submitted.
{~
x ANN RlLEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 l
Washington, D. C. 20006 (202) 293-3950
L 181 (y
)
1 MR. LEWIS:
But you said it was a program that nj 2
extended throughoot all of NRC and it was coordinated.
I'm 3
trying to find out how the rest of NRC fits into it.
Is it 4
just done by AEOD?
5 MR. HICKMAN:
The official NRC performance 6
indicator program is that program done by AEOD.
7 MR. LEWIS:
I'm trying to understand what you 8
meant by coordinated throughout NRC.
9 MR. HICKMAN:
That means that the reports that 10 AEOD publishes are the only official performance indicators 11 used by the NRC.
12 MR. LEWIS:
But It's being done in AEOD.
13 MR. HICKMAN:
It's being done in AEOD and it's (D
V 14 distributed throughout the NRC.
15 MR. LINDBLAD:
Distributed.
16 MR. LEWIS:
I don't think that's the word you used 17 at the beginning.
Maybe it is.
Distributed, in fact, 18 outside NRC too.
19 MR. HICKMAN:
Outside of NRC.
We send them to 20 licensees.
21 MR. LEWIS:
I see.
It's simply an AEOD program.
22 That's what I'm trying to understand.
Okay.
23 MR. HICKMAN:
From the Announce1 rent 200 which you 24 see there I will briefly summarize some of the major 25 restrictions on the use of the performance indicators.
OL)
ANN RlLEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
182
(
l' First of all, they are intended as a tool for 2
senior NRC managers to monitor trends and overall 3
performance at individual power plants.
4 They are to be viewed as a set to provide 5
additional information on performance level.
6
-They are one of several tools.
They are not to be
'7 overemphasized in relation to any of the other-tools that 8
the NRC uses.
9.
Finally, they are not to be used to rank plants.
10 MR. SHEWMON:
Just to see what the words mean, are 11 the SALP ratings used to rank plants?
12 MR. HICKMAN:
No.
We'do not attempt to rank 13 plants.
b1
\\_/
14 MR. SHEWMON:
Fine.
By your parlance, you-never 15 rank plants; they just happen to be divided'into good, bad 16 and indifferent.
17 MR. HICKMAN:
That's true.
18 (Slide.]
19 MR. HICKMAN:
These are the eight performance 20 indicators that are currently published quarterly.
21 The newest-indicator is the.cause codes indicator..
22 It was added to the program in 1989.
Cause codes capture-23 programmatic causes of every reportable event in six areas.-
24 Collectively they are' treated as one indicator.
25
[ Slide.)
' O.
.V ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K.. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
183
-1 MR. HICKMAN:
These are the six programmatic areas 2
in which the cause codes identify each LER submitted-to the 3
NRC.
The cause code program was approved by the Commission 4
in 1989.
5
[ Slide.)
6.
MR. HICKMAN:
As I said, they identify 7
programmatic causes of every licensee event report submitted 8
to the NRC.
At the time we submitted this program to the 9
Commission for approval the Commission approved the use of 10 cause code trends, that is, comparison of each-plant's 11 performance in the cause code indicator to its previous past 12 performance.
13-The Staff recommended that the. Commission _ approve 14 comparison of cause codes to the average-of the NSSS vendor 15 group for each individual plant.
At that time.the 16 Commission was concerned that this would result in 17 comparison of a plant such as Ginna to a group which 18 included a plant such as South Texas.
~ 19 MR. MICHEISON:
Excuse me.
Are the cause codes.
20 that INPO uses about the same, or are they'using their-own 21 set?
'22 MR. HICKMAN:
I'm not aware of any cause codes 23 that-INPO uses.
24 MR. MICHEISON:
In their own LER tracking system 25 they have_causes.
I don't know if they relate to these same b
w/
ANN RILEY & ASSOCIATES, Ltd.
Court. Reporters -
1612 :K. Street, N.W., Suite 300 Washington, D. C. 20006 (202). 293-3950
1 causes or not.
They have their own database for LERs.
2 MR. BARANOWSKY:
We give them LER information.
3 They get that from us.
And they have access to sequence 4
coding and search systems.
To the best of my knowledge, 5
they do not have an independent LER cause coding, although 6
they review each LER and have some sort of a significance 7
ranking.
I forget the name of their program.
8 MR. MICHELSON:
It's the CN program.
9 MR. BARANoWSKY:
It's not the equivalent of cause 10 codes, though.
11 MR. MICHELSON:
When they set that up in the early 12 1980s they also had cause codes.
I just wondered whether 13 there has been any attempt to make a standard so when they 14 talk about a cause everybody is thinking the same thing.
15 MR. NOVAK:
This is Tom Novak.
The causes that we 16 identify are what we draw from the event.
The licensee has 17 his own cause.
In many cases they are similar, but as part 18 of the development of this indicator we decided that from 19 the basis of our own reading we would identify a cause.
20 Generally we attactr about two causes to the complete event.
21 There is not a single cause for the event.
Generally you 22 have about two causes per licensee event report.
23 MR. MICHELSON:
My only reason for the inquiry was 24 I was wondering if we had become sophisticated enough to 25 where everybody had a common cause compilation.
You could 3
(O ANN RILEY & ASSOCIATES, Ltd.
Court Reporters i
1612 K. Street, N.W., Suite 300 I
Washington, D. C. 20006 (202) 293-3950
185
-r, 1
have a universal standard for causes.
2 MR. LEWIS:
Maybe you defined it but I'm not quite 3
clear.
What is a programmatic-cause as distinguished from a 4
real cause?
5 MR. HICKMAN:
As opposed to root causes of events.
6 We are looking at programmatic areas, programmatic 7
weaknesses that were involved in the events.
8 MR. LEWIS:
What does programmatic mean in-this 9
context?
10 MR. HICKMAN:
We had the list of the cause codes 11 here.
Maintenance, for example.
One of the cause codes is 12 maintenance.
If we believe that there was some sort of 13 maintenance' deficiency involved in the event, such as a poor p..
\\-
14 preventive maintenance program.
15 MR. LEWIS:
I begin to see it.
Would.I be unfair 16 if I said what you mean by programmatic cause is a cause 17 that can be attributed to something that can be corrected by 18 NRC regulation?
Is that what is meant by a programmatic-19 cause?
20 MR. HICKHAN:
No.
21 MR. LEWIS:
Then what is it?
If something breaks, 22 say a shaft breaks on a motor, then the cause wouldn't be a 23 broken shaft; the cause would be whoever manufactured it or-24 somebody who didn't look at it, or something like-that?
25 MR. HICKMAN:
That could conceivably be a design O
ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950-
186 J )_ l' weakness.
It'could be an installation weakness. -There are 2
various things that it could be, depending on what the LER 3
says.
4 MR. LEWIS:
When lightning hits something and 5
destroys it we don't blame it on God.
That's because 6
insufficient lightning protection was_ built in, _ You're 7
trying to pin it on somebody.
That's'the sense I have.
I 8
may be wrong about that.
9 MR. HICKMAN:
External events are captured under 10 the miscellaneous cause code, such as lightning, flood.
11 MR. LEWIS:
I'm trying to find out whether you are 12 looking for a technical cause or for a punishable cause.
13 MR. HICKMAN:
Actually-neither.
14 MR. NOVAK:
We are trying'to identify if from the 15 events and from the equipment failures or the operator 16-errors that occur can we see some recurring programmatic 17 area, whether it's maintenance, whether it's the_ operations 18 side of the house.
I think if you look at our trends you 19 will see what we are trying to do.
This is just one.other
- M) way of looking at the data other than counting the. number of 21 times an RHR pump failed.
We are interested in more or less 22 is this plant having more maintenance problems.
23 MR. LEWIS:
I'm just trying to find out whether-24 you are slipping something by me with that word 25
" programmatic" or whether you are looking for the wallet nU ANN RlLEY & ASSOCIATES, Ltd.
Coud Reponers 1612 K. Street, N.W., Suite 300 Washington, D. C 20006 (202) 293-3950
-~
187 b-
-1 under the lamppost.
But-you're telling me I'll-find out'as
.V 2
you go along if I'let you go along.
3 MR. WILKINS:
Hal, I have inferred from looking at 4
the list of cause codes that the first bullet would be just 5
as meaningful if the word " programmatic" were deleted.
6 MR. HICKMAN:
The reason that we included that 7
word was because these'are not intended to be root causes of 8
events.
We don't believe we can get those for every event 9
out of the LERs.
10 MR. WILKINS:
Nevertheless, you say that each 11 reportable event has a cause code.
12 MR. HICKMAN:
That's correct.
13 MR. LEWIS:
That's what it says.
14 MR. WILKINS:
One or more cause codes.
The list 15 of possible cause codes was on the preceding slide and there 16 are six of those.
So you assign each reportable-event one 17 vr more of those cause codes, and the word " programmatic" in 18 that context is meaningless.
It's one of these'six.
19 MR. HICKMAN:
I'11 see to that.
20-MR. LEWIS:
I'm delighted that you understand.
21 MR. WILKINS:
I'm not sure I understand,.but I 22 have stated my understanding.
23-MR. SHEWMcN:
We've all agreed on this.
Why don't 24 you go on.
25 MR. HICKMAN: lokay.-
p
%.)
ANN - RILEY & ASSOCIATES, Ltd.
Court Reporters 1612.K. Street, N.W., Suite 300 Washington, D. C. 20006
-(202)- 293-3950
188 1_ '
MR. LEWIS:
Paul, forgive me.
I think it's 2
important at.the very_beginning.
3 MR. SHEWMON:
Would you.like to have him scratch
.4 it out?
Would that make you allow him to go on?
5 MR. LEWIS:
No.
Actually having him explain it-6 clearly enough that it could be understood would help a 7
great deal.
8 MR. SHEWMON:
Do you think he hasn't explained it-9 yet?
10 MR. LEWIS:
No.
11 MR. SHEWMON:
Well, I'm not sure he can to_your 12 satisfaction.
13 MR. LEWIS:
That's entirely possible.
That's what 14 I'm trying to find out.
15 MR. HICKMAN:
Maybe as we go on one of the things:
16 that I could do is explain to you how we use the performance-17 indicators, and that may help to clear it up.
18 MR. CARROLL:
Let me ask a related question.- Have 19 you seen any indication that people'ars less forthcoming in 20 writing LERs as a result of having this program?
21 MR. HICKMAN:
No, we have not seen that.
There is-a great deal of int' rest in'the industry about cause codes 22 e
23
_and.wo. provide them information freely.
They in fact 24 attempt to do their own coding of events to see how-it 25 matches ours.
But I've not seen any impact upon reporting Q
ANN-RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
189 1
as a result of that.
2 MR. CARROLL:
Having written LERs personally.
3 myself, I know that you can make judgments and say we don't 4
really need to send one in on this event.
There are some 5
very gray areas there.
Okay.
6 MR. HICKMAN:
To finish up this slide, the 7
Commission did not approve the use of cause codes to compare 8
an individual plant against an NSSS vendor.because of the 9
wide variation in design within the vendor groups.
So the 10 Commission then directed the Staff to develop appropriate 11 peer groups to be used for the purpose of comparing cailse 12 codes from an individual plant to some appropriate peer 13 group.
\\
14
[ Slide.]
15 MR. HICKMAN:
As we got into that study we learned 16 that cause code data are cyclic with an interval that 17 corresponded to the time interval between refueling outages.
18 We-looked into this-further and found that in fact 19 there is an effect of the mode of operation throughout the 20-cycle upon the cause codes.
21 We also found that the types of events which 12 2 occurred throughout the cycle were a function of the mode ofL 23 aperation of the plant.
The types of modes that we are 24
. talking about are things such as startup, power operations, 25 refueling outages, non-refueling outages, that type of LO ANN RILEY & ASSOCIATES, Ltd.
Coud Reponers 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
-190
! )
1-thing.
2
[ Slide.]
3 MR. HICKMAN:
We then-developed techniques to 4
incorporate peer groups and these operating cycle phase 5
effects into the PI program.
The major benefits of these 6
techniques are the three shown on this slide.
7 We have indicators now for both the operational 8
period and the shutdown period for each plant.
We compare 9
the' indicators for an individual plant to its peer group, 10 and we have indicated the statistical significance of the 11 trends and comparisons that we make.
12 I want to add at this point that we developed 13 these methods for the purpose of comparing cause codes.
We 14 found, however, that these are suitable peer groups and 15 suitable techniques to use for all of the performance 16 indicators.
17 MR. WILKINS:
Are you going to tell us later what 18 you do when you discover that the' peer group-for Plant A is 19 significantly statistically worse than Plant B?
20 MR. HICKMAN:
I can get into that-,
21
[ Slide.]
22 MR. HICKMAN:
A brief1 history of the development 23 program.
We utilized two national labs and the NRC's 24 Interoffice Task Group on performance indicators.
25 We developed the peer groups at the Idaho National l
O L
ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
191
(
1 Engineering Laboratory.
2 The Oak Ridge Laboratcry worked on the operating i
3 cycle phases.
4 We used both to help develop the calculational and
?
5 display methods that we use.
j 6
Then we ran a trial program which used the 7
Interoffice Task Group and their feedback to help' improve 8
the human factors aspects of the displays.
l 9
We also provided draft copies of the new format to 10 senior managers at the most recent screening meetings that-11 were held in October and November.
The commentsLfrom the i
12 senior managers ranged from "no objection" to "quite good."
i 13 The Interoffice Task Group felt that the 14 presentations by the new format were an improvement over the 15 previous methods and provided more information.
16 MR. CARROLL:
Who makes up this Interoffice Task 17 Group?
-18 MR. HICKMAN: -The Interoffice Task Group-is 19 composed of representatives from each of the regions, 20 several from NRR, Research, and AEOD.
It's a working group-21 level to develop the techniques that the workers can use.
22
[ Slide.).
23 MR. HICKMAN:
Let me get into some of the issues 24 that we looked at for developing the peer groups and the 25 operating cycle. effects.
O ANN RILEY &- ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300-Washington, D. C. 20006 (202) 293-3950
192
'(
1 We looked at more than 80 factors which were 2
broadly categorized into three major areas,.that is, design, 3
regulatory issues, and organization.-
4 We decided that the design factors were the most 5
important, that the regulatory issues were next.
We didn't 6
look at the organizational factors because that's the effect
.i 7
that we wanted to measure through the performance 8
indicators, how individual licensees performed given a 9
particular plant.
10 A couple of other.important issues.
We needed to 11 determine the minimum group size to have. valid statistics 12 for a peer group, and we also sought input from the owner's 13
- groups, tO
(_/
14 The minimum group size effort determined that on 15
'the order of six would be the minimum group size we could 16 have to have valid statistics.
17 With that input, then~we used the design and 18-regulatory issues in order to achieve that minimum number.
19 These are the factors that we have considered and-have now 20 incorporated into our peer groups.
21 MR. MICHELSON:
Would you refresh my memory again 22 as to what-you meant by-group?
23 MR. HICKMAN:
A peer. group is a group of similar 24 plants that we are using to compare the performance of an 25' individual plant.against.
i ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3350
193' 1
MR. MICHELSON:
If it's a multi-unit plant it's 2
still one member of the peer group and you evaluate both 3
units of that plant or all three?
4 MR. HICKMAN:
We consider each unit separately.
5 MR. MICHELSON:
It's a group of units that you are 6
looking at; is that right?
7 MR. NOVAK:
That's correct.
Oconee would 8
represent three units.
9 MR. MICHELSON :
You are going to do it on a per-10 plant basis?
11 MR. NOVAK:
Per reactor.
12 MR. CARROLL:
Per unit.
13 MR. MICHELSON:
Well, I hear two stories.
He said 14 Oconee was three units, and that's quite right, but is that 15 one member of the group?
16 MR. WILKINS:
It's three units of the group.
17 MR. HICKMAN:
It's three members of the group.
18 MR. MICHELSON:
I could just have two plants each 19 with three units and have a group.
20 MR. CARROLL:
Why don't you show the next slide.
21 I think that will clear it up.
22 MR. NOVAK:
In the performance indicator we have 23 never averaged out a dual unit site and said here's the 24 performance of that site.
We've always represented each 25 reactor individually.
O ANN RILEY & ASSOCIATES, Ltd.
COud RepOders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
0 194 h
1 MR. MICHELSON:
The problem is, of course, a lot 2
of LERs may be a plant LER and not a unit LER.
They report 3
it as a unit LER but it's a common problem they have.
4 MR. BARANOWSKY:
Most of them are unit-specific 5
LERs.
For instance, we wouldn't call TMI 1 and TMI 2 an 6
average plant.
I don't know how you would g7t that average.
7 So we have to go with individual units.
8 MR. MICHELSON:
I just wanted to make sure I 9
understood what your group would be.
Thank you.
~
10 MR. HICKMAN:
LERs that are applicable to more 11 than one unit are assigned to all units that are applicable.
12 MR. MICilELSON:
All units that you may have in 13 your group.
You cited Oconee as a case.
If Oconee is a 14 member of the group it counts as three, anything on any of 15 the three units?
16 MR. NOVAK:
No.
I'm sorry, Carl.
If you look at 17 the B&W plants, there are seven operating B&W reactor 18 designs.
That met our minimum peer group.
In other words, 19 we had to have at least six operating units based on the 20 work we did.
Each Oconee unit reports its own LERs.
There 21 is some commonality.
I recognize that.
But on occasion 22 that LER will be applied to two units.
It may be 23 represented on more than one unit.
24 MR. MICHELSON:
In that_ case your group was nearly 25 all the B&W units, because there aren't many.
How-about the llk ANN - RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
195 I
_1 case of boiling or pressurized?
2 MR. NOVAK:
We'll show you that.
3 (Slide.]
4 MR. HICKMAN:
Let me finish up this slide.
Then 5
we can get into that.
6 We talked to the owners' groups, the Westinghouse 7
and GE owners' groups and got their input.
Their input 8
primarily focused upon the licensing date relative to TMI 2.
9 They have developed their own peer groups for their own 10 purposes.
They find that that is an important factor.
11
[ Slide.]
12' MR. HICKMAN:
These are the peer groups.
13 MR. CARROLL:
These are the new peer groups.
14 MR. HICKMAN:
These are the new peer groups.
15 MR. CARROLL:
The old ones were what?
16 MR. HICKMAN:
In the past the only peer groups we 17 really used was a differentiation between PWRs and BWRs.
18 That's as far as we got 19 MR. CARROLL:
Thank you.
20 MR..HICKMAN:
The GE plants are divided into two 21 groups, basically along the lines of the licensing date pre-22
'and post-TMI.
The Westinghouse. plants are divided into four 23 groups.
24 MR. SHEWMON:
Sir, you did have trend lines, 25 though, and for trend lines you would~use all BWRs or all ON.)
ANN RILEY & ASSOCIATES, Ltd.
COud Repoders l:
1612 K. Street, N.W., Suite 300 L
Washington, D. C. 20006 (202) 293-3950
i I
196
()
1 PWRs?
2 MR. HICKMAN:
There were only two performance 3
indicators for which we broke up the industry at all-into 4
B's and P's, and that was for the safety system failure 5
indicator and the collective radiation exposure indicator.
6 MR. SHEWMON:
Fine.
7 Carl, go ahead.
8 MR. MICHELSON:
Now you are getting back to plants 9
again.
Which of the plants are units?
10 MR. HICKMAN:
There are seven B&W units.
We count 11 them all separately.
12 MR. SHEWMON:
Your slide says plants.
13 MR. WILKINS:
Yes, but it says all B&W plants and 14 some B&W plants have more than one unit.
15 MR. NOVAK:
Let me give you an example.
Beaver 16 Valley has one unit in the old 3 loop and one unit in the 17 new 3 loop.
That was the way we thought we could-best 18 represent those two units.
So they don't appear in the same 19 peer groups.
20 MR. HICKMAN:
Substitute the word " units" for 21 plants in that top bullet and maybe that will help.
22 MR. MICHELSON:
I think on all the bullets you've 23 got to do that, not just the top one.
24 MR. CARROLL:
Or Millstone, the three units.
25 MR. HICKMAN:
Millstone is in three separate peer ANN RILEY & ASSOCIATES, Ltd.
Coud Reponers 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
197 1
groups.
2 MR. SEALE:
I can't help but ask this.
Have you 3
considered a peer group comparison where the comparison-is 4
between regions?
5 MR. HICKMAN:
We do look at that.
We look at that-6 when we are evaluating-plants, largely because the screening 7
meetings that Dr. Murley holds are by region.
So when a 8
region comes in, we look to see within that region which-9 plants are the poor performance and which plants are the 10 better performance, but we really don't do much with that.
11 MR. SEALE:
I'm saying between regions.
12 MR. HICKMAN:
That has been done.
We don't use it 13 at all.
14 MR. SEALE:
I can see where that might be an 15 internally useful thing.
16 MR. HICKMAN:
Yes.
17
[ Slide.]
18
-MR. HICKMAN:
To determine the effects of the 19 operating cycle, we determine that these effects were 20 largely due_to changer.
First'of all, it was based upon 21 this observed effect that the rate of cause codes varied 22-with the phase in the cycle.
We determined that that-was 23 due to the change in activities between a refueling outage 24 and initial plant startup, power operations and that sort of 25 thing. --We-looked to identify those phases of an operating
(%)
ANN RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
198 1-cycle in which plant activities changed significantly.
2 (Slide.)-
3 MR.-HICKMAN:
The result was the five phases shown 4
on the top: startup, power operations, pre-refueling, 5
refueling outage, and non-refueling outages.
6 We track this data and we tabulate it and we 7
publish it in the draft PI report.
For purposes of display, 8
however, we found that it was confusing and really not 9
necessary to attempt to display the indicators in five 10 separate phases.
Rather what we did was to combine those 11 into the two general phase types of operations and shutdown, 12 as shown on this slide.
This is what you will see in-the 13 new report.
14 MR. MICHELSON:
What is pre-refueling?
Is that 15 just before you open the head or is that as soon as you 16 start coming down'in power?
17 MR. HICKMAN:
Pre-refueling is the last 25 days.of 18 operations prior to the start of the refueling: outage.
19 There are some plants where an attempt is made to get ahead 20 of the refueling outage by starting. activities early.
Wo.
.21 felt that we might monitor that and see if we see-any 22 problems because of that.
There aren't-many plants doing 23 that but there are some.
24-MR. MICHELSON:. 'The' folklore in the business is 25 that things don't happen when you are running at a nice h
ANN. RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 -K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
199
.1 steady state; they happen when you are coming up._or coming 2
down.
3 MR. CARROLL:
What he's saying, Carl, is sometimes 4
you want to get a jump on the refueling outage and you do 5
some things that do have a risk of causing a trip.
If you G
are going to overhaul all-your condensate pumps, you may do-7 one of them before the outage.
8 MR. MICHELSON:
That part didn't bother me.
I 9
just wanted to make sure I understood what pre-refueling 10 meant.
11 Isn't it important to kind of look at if there is 12 any difference between changing the state of the reactor and 13 running it steady state?
You don't have any category here-14 where you are changing the state, cooling down and so-forth.
IS Pre-refueling might be deferred to cover that but not in the 16 same way, because it covers a 25-day period and not the time 17 during which you are cooling down.
Cooldown is'only a 18 couple of days.
19 MR. HICKMAN:
One.of the constraints that we have 20 is being able to identify the phases.
21 MR. MICHELSON:
Yes.-
You may not have enough
'22 information to be that finite.
Usually-in an LER they will 23 tell you if they were cn1 the way down when this happened.
24-If they were cooling down, for instance.
-A lot of things i
25 happen when you cool down.
I don't know'if that's folklore.
LO ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202)-293-3950
200 1
When I read LERs I find a lot of_ things are happening, just 2
like transients.-
3 MR. CARROLL:
Cooling down would be enveloped by 4
the refueling outage.
5 MR. MICHELSON:
That's true, but it's a 25-day-6 period and you're cooling down only a couple days of that.
7 MR. CARROLL:
No.
8 MR. HICKMAN:
The refueling outage is the length 9
of the outage.
10 MR. MICHELSON:
But you are losing the data during 11 the change in thermal conditions.
12 MR. HICKMAN:
The one phase in here that I feel 13 would be desirable to divide further if we could would be 14 the refueling phase, because they go through a lot of-15 different activities during that time.
The problem right 16 now is we don't have the data in order to do that.
17 MR. DAVIS:
What is your source of_ data during 18 shutdown?
Is it only LERs?
19 MR. HICKMAN:
Right now we primarily use the
~20 morning status report that the op center gets.
It calls 21 each licensee and gets their status.
That gives_us within 22 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> of when they have shut down.
23 MR. DAVIS:
Is-that a consistent reporting 24 procedure across all plants?
25 MR. HICKMAN:
Yes.
The op center calls every-DU ANN - RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
-l 201 4(f 11 plant.
It starts at four o' clock in the morning and.gets 2
the status of each plant.
3 MR. DAVIS: 'Thank you.
4 MR. MICHELSON:
Are you storing those in a 5
computer and reading them back later?
6 MR. HICKMAN:
Yes.
7 MR. MICHELSON:
They would tell you if they are 8
cooling down.
9 MR. CARROLL:
If you would like to see that, Pete, 10 it is Balletin 4 on the ACRS bulletin board.
11 MR. DAVIS:
I don't have access to that.
12
[ Slide.)
13 MR. HICKMAN:
These are someaof the new
-]
N )-
14 calculational methods that we use.
15 We determine trends for each individual plant 16 using a linear regression.
17-For-comparison of a plant to'its peer group, which 18 we have called deviations, we use the median;of the peer 19 group for comparison purposes rather than the mean.
ThisLis 20 because the data are not normally distributed, and therefore 21-the median is more representative of the group value than 22_
the mean is.
23 MR. LEWIS:
I'm sorry.
That was a comment.I.can't 24
'let go by.
25 MR. WILKINS:
That's all right.
You go ahead and O
l l
ANN RILEY _&. ASSOCIATES, Ltd.
L
.Coud Repoders l
1812 K. Street, N.W., Suite 300 i
Washington, D. C. 20006 i
(202) 293-3950
202
)
1 say-it.
v
-2 MR. LEWIS:
I will stipulate a dissent from that 3
generalization.
It's wrong.
4 MR. WILKINS:
Your word "therefore" is quite 5
inappropriate.
But please continue.
-6 MR. LEWIS:
If as you go on you really depend on 7
that, then we will come back to this.
8 MR. HICKMAN:
We determined the statistical 9
significance of the calculated trends and deviations using a 10 Monte Carlo simulation technique.
11 It's important to point out that we did not expect 12 when we undertook this effort that would see major changes 13 in many plants.
We recognize that this is not going to be 14 some whole new revelation that we've never seen before.
We 15 expect, however, to see improved representation of plant 16 performance by using these methods in approximately 15 to 20 17 percent of the plants, primarily those that are involved 18 with changing from operations to a refueling or some type of-19 extended outage.
20 MR. LEWIS:
Do you mean something technical by the 21 words " statistical significance" or is this an eyeballing 22 operation?
What do you calculate?
23 MR. HICKMAN:
We calculate the probability _that an 24 observed sequence of-events, an observed pattern, if you 25 will, is random, o
ANN RILEY & ASSOCIATES, Ltd.
Coud Reponers 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
203
(,,)
1 MR. LEWIS:
How can you do that if you are using 2
the median as the norm?
You don't have an a priori 3
statistical distribution based on the median if the data.are 4
not normally distributed.
So what formula can you use?
5 MR. HICKMAN:
For the median we use a binomial 6
distribution.
7 MR. LEWIS:
I understand that, but what formula do 8
you use to calculate the probability that the observed data 9
might have occurred as a random occurrence from a median 10 unless you have presupposed a distribution?
You are 11 presupposing the binomial distribution, not the normal 12 distribution.
Is that what is going on?
13 MS. GRADY:
I'm Bennett Grady with AEOD staff.
()N
(_
14 I've also served as a statistical consultant.
15 As shown on the last bullet there, we used Monte 16 Carlo simulation to develop tables that would compute these 17 P values.
18 MR. LEWIS:
For a Monte Carlo simulation you have 19 to insert a model.
What model do you use?
20 MS. GRADY:
The model that we use, as Don told 21 ycu, ;s a binomial distribution.
22 MR. LEWIS:
The binomial distribution requires 23 that you specify in advance a mean.
24 MS. GRADY:
It requires a probability of observing 25 a particular event.
ANN RlLEY & ASSOCIATES, Ltd.
Coud Reponers 1612 K. Street, N.W., Suite 300 i
l Washington, D. C. 20006 (202) 293-3950
204
/1 MR. LEWIS:
That's correct. -How do you find that-2 probability.to use?-
3 MS. GRADY:
We have two different simulations, 4
depending on whether we are computing the deviations or --
5 MR. LEWIS:
No.
When you use the word " random" 6
-you have to have a model for your random. distribution.
7 MS. GRADY:
Yes.
8 MR. LEWIS:
I'm trying to understand.
The model 9
you are using is the binomial distribution.
10 MS. GRADY:
Yes.
11 MR. LEWIS:
I have no problem with that.
But in 12 the binomial distribution there is a parameter which you can 13 call P or whatever it is.
Where does that come from?
s) 14 MS. GRADY:
Let's take the deviations from the 15 peer group.
We assume that the events in the peer group all 16 have the same probability of occurring.
If we observe the 17 event, then the probability of P for the binomial 18 distribution is one over the number of plants within.the 19 peer group.
20 MR. LEWIS:
Fine.
If you use the' peer group, you
~
21 are using the median of the peer group and inserting that' 22 for the P in the binomial distribution?
D23 MS. GRADY:
Yes.
-24 MR. LEWIS:
That's wrong, of course, because the P 25 in-the binomial distribution should be-the mean of'the peer h
ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202)_293-3950
205 A.
(j 1
group.
2 MS. GRADY:
-It's the probability of an event.
3 We've observed a certain number of events during this 4
period.
5 MR. LEWIS:
I understand.
In order to infer the P
~
6 that you put into the binomial distribution the formally 7
correct way is to infer it from the mean of the peer group 8
if you are using it as sort of an-empirical binomial
-9 distribution simulation.
I don't understand how the median 10 gives you that information.
11 MR. HICKMAN:
Bennett, we don't use the median in 12 that determination, do we?
13 MS. GRADY:
No.
We are looking at rates and we 14 observed a number of events over a number of times.
In that 15 sense.it is a mean.
16 MR. LEWIS:
Then I don't know where the median 17 comes from, ~because it says here deviations from peer group 18 median.
19 MS. GRADY:
The median is used in calculating what-20 is on the display.
21 MR. LEWIS:
The simulations you are running.are 22 binomial based on the mean?
23 MS. GRADY:
Right.
24 MR. LEWIS:
Okay.
That's correct.
Then the 25 statistical significance-'is based on the distribution of O
~ ANN RILEY '& ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W.
Suite 300 Washington, D. C. 20006
.(202) 293-3950
1 l
206
/3 1
actual events around the mean as determined using the
-(g 2
binomial distribution to get the a priori probabilities of 3
the numbers of events.
4 MS. GRADY:
' hat is correct.
5 MR. LEWIS:
That's a legitimate procedure if you 6
have enough data.
Thank you.
I understand.
So the median 7
is simply a red herring up there.
8 MR. HICKMAN:
There aro two different things we 9
are talking about here.
One is to get the value for the 10 plant's deviation from the peer group.
For that calculation 11 we are looking at the median of the peer group and comparing 12 it to median value for the peer group compared to the value 13 that the plant had, and that will determine its deviation 14 from the peer group.
When we calculate the statistical 15 significance we use the binomial distribution, as was just 16 explained.
17 MR. LEWIS:
But then you are calculating the 18 statistical significance of something that is-different from 19 what appears earlier on the slide.
You are doing two 20 completely independent things and they are not related.
21 MR. WILKINS:
They're related a little bit.
If 22 you can trust the binomial distribution, then they are 23 related.
24 MR. LEWIS:
That's a different level of 25 questioning.
I'm stipulating that just to try to get on O
ANN RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
207 l'-
with the show.
2' MR. WILKINS:
It obviously is your judgment that
-3 seven units is large enough so that you get meaningful 4
statistics.
5 MR. LEWIS:
It's not my judgment.
6-MR. WILKINS:
I mean it is their judgment.
7 MR. HICKMAN:
Yes.
8 MR. LEWIS:
I'm trying to understand what they 9
have done.
10 MR. SHEWMON:
Are you addressing this to Hal or 11 the speaker?
12 MR. WILKINS:
I'm actually addressing it to the 13 speaker.
14 MR. LEWIS:
But he's doing it past me.
15 MR. WILKINS:
I can't look at the speaker without 16 Hal seeing me.
17 MR. HICKMAN:
Seven is the minimum group size that 18 we want to use.
We are kind of constrained to do that 19 because of the B&W group.
It really doesnt go much with 20 any other plants.
While there may be problems with it, we.
- 21.
feel that in the_ evaluation process we can sort out those 22 problems.
p
_23 MR. WILKINS:
In my judgment, the probability that 24 what you see is due to random events rather-than to something real_will be higher in the smaller peer groups
.h ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite 300
. Washington, D C. 20006 (202) 293-3950
208
()
1 than.t will be in the larger peer groups, and you must be 2
aware of that.
I assume that you are.
The last speaker 3
certainly indicated some awareness of statistical theory.
4 MR. LEWIS:
This is addressed to Ernest, not to 5
the speaker, although you can cavesdrop.
When they 6
calculate statistical significance, presumably -- correct 7
me if I'm wrong -- they are going to calculate a priori 8
probabilities based on the binomial distribution, multiply 9
them together for all the events that occurred, and for a 10 small enough number that's a very shaky way to find what is 11 called statistical significance.
12 MR. WILKINS:
I think so.
13 MR. LEWIS:
We agree on that.
But that's another
(}
14 level of conversation.
15
[ Slide.]
16 MR. HICKMAN:
This shows the quarterly data from 17 the current performance indicator report.
18 Some things to point out here.
You notice that we 19 display eight quarters of data for each performance 20 indicator.
This dotted line represents the critical hours 21 for the quarter.
22 In this case we can see that zero critical hours 23 in the third quarter of 1992 indicates that the plant was 24 shut down the entire quarter.
25 These values in here indicate that it was shut n.
l
[-.)
l ANN RILEY & ASSOCIATES, U3.
I Coud Repoders 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
1 209
)
1 down for some period of time, and we don't really know how 2
long.
3 You will see that we have for each plant the dark 4
line which represents a six quarter moving average of the 5
value of the individual indicator, and the cause codes we 6
show only those six quarter moving averages.
We do not show 7
the value of the cause code itself because of the concern of.
8 comparing plants inappropriately.
9
[ Slide.)
10 MR. HICKMAN:
This is the same page from the draft 11 proposed report.
In this case we no longer have that 12 critical hour line.
We show a time lino which indicates 13 time in operations, time in shutdown, and the letter "R" l
14 indicates a refueling outage.
15 MR. MICHELSON:
Excuse me for a moment.
Back on 16 your previous slide you showed equipment failure as 17 essentially unchanging.
I wasn't quite sure what that block 18 in the lover left-hand corner of the previous slide was 19 really telling me.
What is that telling me?
20-(Slide.)
21-MR. HICKMAN:
Equipment failures is kind of like a 22 catch-all category.
It's only used for random failures of 23 electronic equipment and external events,'and typically the 24 value is zero.
25 MR. MICHELSON:
Those values are low.
The ANN RILEY & ASSOCIATES, Ltd.-
Coud Reponers 1612-K. - Street, N.W.; Suite - 300 -
Washington, D. C. 20006 (202):293 3950
210 1
unvarying is the part that puzzled me a little bit.
2 MR. IIICKMAN:
That means that they have been zero 3
for quite while.
4 MR. MICl!ELSON:
I don't know if that was zero or 5
not.
Is that a zero there?
6 MR. IIICKMAN:
By the old method you couldn't tell.
7 MR. MICilELSON:
It looks like a finite value.
8 It's not at the bottom.
9 MR. HICKMAN:
The performance indicator report is 10 divided into two parts.
These graphs are in part 1 and this 11 was done this way to make it more difficult for people to 12 flip pages and look at two plants and try to compare their 13 cause codes.
N 14 MR. MICHELSON:
I was only puzzled about why it's 15 a flat line.
Is there some simple explanation?
16 MR. BARANOWSKY:
It's probably zero and this 17 presentation is an obfuscation of the information.
18 MR. MICHELSON:
Is that a zero in the middle of 19 the graph?
20 MR. HICKMAN:
No.
In part 2 the data are all 21 tabulated.
You can go see if they are actually zero.
In 22-this case the plant has had'zero equipment fallures for many 23 quarters and therefore the six quarter moving average has 24 stayed at zero.
25 MR. MICHELSON:
I-just-would have intuitively O
-ANN RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
I 211 1
expected it to be at the bottom, a black line, just like you 2
put zeros up in that second block.
3 MR. IIICKMAN :
Right.
This was intentionally dono.
4 MR. MICllELSON:
There is a finito number of 5
failures but they are apparently very constant.
6 MR. BARANOWSKY:
Wo were told not to be able to 7
compara plants, and so we had to present it this way.
8 MR. WI LKINS :
I am impressed by the willingness of 9
the Staff to concede that they are deliberately obfuscating 10 something in response to orders.
I understand.
11 MR. CARROLL:
But not anymore.
Flip the page.
12 MR. IIICKMAN :
I want to clarify one thing just so 13 there is not any misunderstanding.
Equipment failures are 14 not capturing overy piece of equipment that fails.
There 15 are many of thoso.
Those are going to have some other 16 cause.
17 MR. MICllELSON :
That wasn't even the point.
18 (Slide.)
19 MR. HICKMAN:
You'll see here that we show stack 20 bara now shaded to indicate the phase in which the event 21 occurred.
While we calculate only for operations and 22 shutdown, we also indicato events that occurred in startup 23 because those are of special concern to us.
24 The forced outage rate, the equipment forced 25 outages and the radiation exposure are not differentiated by O
ANN RlLEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
l 212 1
phase type because they are not a function of phase type.
2 How we show the cause code actual numbers here.
3 MR. MICHELSON:
Does that mean there were no 4
miscellaneous causes during that period?
5 MR. HICKMAN:
None.
We changed equipment failure 6
to miscellaneous because it was misleading, and there vera 7
no failures during that entire three-year period.
8 Another point to bring out.
We show 12 quarters 9
of data rather than just eight.
10 MR. LINDBLAD:
Excuse me.
I recognize that this 11 will be reviewed mostly by people who are knowledgeable and 12 the like, but it kind of troubles me to see a reactor vendor 13 identified as the peer group and then the equipment forced b
q 14 outages down below, which I suspect are generally not 15 reactor equipment so much as balance of plant equipment and 16 the like.
The grouping before was talking about plant 17 generally, but so many of the other numbers are associated 18 directly with reactor equipment.
I think that the reactor 19 vendor may be getting a bad rap if the balance of plant 20 equipment is shown to be performing poorly.
Was there any 21 other alternative as to how to describe those plants?
22 MR. HICKMAN:
We looked at a lot'of other ways to 23 group plants.
24 MR. DAVIS:
You could do it by A-E, for example.
25 MR. HICKMAN:
That was one consideration.
If you OO ANN RlLEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
- _ = _.. - _
213
()
1 are looking at scrams, you might want to look at types of 2
feedwater systems.
The problems with those are that there 3
are so many variations on that that the peer groups wind up 4
being very small.
5 MR. LINDBLAD:
How about in equipment forced 6
outages if you were discriminating in what kind of equipment 7
it was?
Can you discriminate between reactor equipment and 8
balance of plant equipment?
9 MR. HICKMAN:
The indicator does not, but in 10 monthly operating reports we get information on outages, and 11 in LERs we will find out if it was forced shutdown.
12 MR. LINDBLAD:
Thank you.
13 MR. CARROLL:
The industry average line is ror all 14 BWRs and PWRs?
15 MR. HICKMAN:
The entire industry, yes, sir.
16 There is also-the peer group-line on there.
Those are the 17 linear regressions.
18
[ Slide.)
19 MR. HICKMAN:
I have provided an expanded view, a 20 blowup of just one of.the indicators to show the difference 21 between the two.
By the old method'we had eight quarters of 22 data.
You will see there one safety system actuation in the 23 third quarter of 1992.
24 When we como down to the new method, we_can see 25 that that safety system actuation occurred during shutdown.
l ANN RILEY & ASSOCIATES, Ltd.
Coud Repoders 1612 K. Street, N.W., Suite. 300 Washington, D. C. 20006 (202) 293 3950-
l 1
214
()
1 If you look at the LER, sure enough, that was associated 2
with surveillance activities being done during the l
3 shutdown.
4 By showing 12 quarters of data we also go back a 5
little bit further and show a second safety system actuation 6
immediately prior to the time period shown on the top curve.
7 We can use this time line to see the plant mode 8
when some of these events occurred.
So it provides a better 9
picture for us of exactly what is going on at the plant.
10
[ Slide.)
11 MR. HICKMAN:
In part 1 of the report there are 12 two pages.
The quarterly data we just talked about.
This 13 is the trends and deviations page and this shows the 14 currently approved method of displaying trends and 15 deviations.
16 We noted that by the old method we would have 17 looked at that safety system actuation and made no 18 distinction between the phase of operation.
So we would 19 have taken that actuation, compared it with the performance 20 immediately prior to that where there were no actuations, 21 and we would have shown a declining. trend.
I want to show
-22 you how that is different by the new method.
23
[ Slide.]
24 MR. HICKMAN:
Here are the trends and deviations 25 calculated by our proposed method.
Since that safety system
()
ANN RlLEY & ASSOCIATES, Ltd.
Coud Repoders..
L 1812 K. Street, N.W., Suite 300 L
Washington, D. C. 20006
_ (202) 293-3950
l 215 1
actuation occurred during shutdown, it will not appear in 2
the operations section and will show no trend in safety 3
system actuations.
They were zero for the period of time 4
that we looked at them.
5 Instead, that safety system actuation will be 6
shown in the shutdown section.
What we find is that by 7
comparing that outage period with the previous outage period 8
in fact there is an improvement there.
If you look back to 9
the previous outage, they had more safety system actuations.
10 So in outage performance they have improved; in operations 11 there is no trend.
12 We also indicate on here the statistical 13 significance.
Black is a high probability that the pattern 14 of events we've seen is not random.
Gray is medium and the 15 white is low.
We can look at this graph and use those 16 shadings to help us evaluate licensee performance.
17 MR. CARROLL:
What does high, low and medium mean?
18 MR. HICKMAN:
They are cutoffs in the value of the 19 probability.
20 MR. CARROLL:
And the report states what those 21 cutoffs are?
22 MR. HICKMAN:
Yes.
23 This allows us to discount those bars that are 24 shaded white, to apply more weighting to those that are gray 25 and more weighting to those that are black.
O ANN RILEY & ASSOCIATES, Ltd.
Court Reporters 1G12 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
216 1
There was some mention earlier about programmatic 2
cause codes and how we use this.
I can get into that if the 3
Committee has time to do that.
This concludes what I have 4
prepared.
5 MR. SEALE:
Could I make a qualitative comparison?
6 MR. IIICKMAN:
Yes.
Please do.
7 MR. SEALE:
In looking at your two thermometers 8
here, the one on the left, it's good to be to the right.
9 MR. }{ICKMAN:
Yes.
10 MR. SEALE:
The one on the right, it's good to be 11 to the left.
12 MR. IIICKMAN:
We are still working on the r
13 terminology.
In overy case to the right is good and to the 14 left is bad.
We tried some different terms.
What we mean 15 here is that this is below average performance as opposed to 16 below the median.
17 MR. SEALE:
So it's performance deviation.
18 MR. HICKMAN:
Yes.
19 MR. WILKINS:
The fact that a man as capable and 20 shu.d as Mr. Scale could fall into that trap means you 21 really ought to give some thought to the language you are 22 using.
23 MR. IIICKMAN :
You're right.. We've tried five 24 different things, sir.
You're correct in that observation.
i 25 If there are no other questions, that concludes' O
ANN RILEY & ASSOCIATES, Ltd.
Court Reporters
-1612 K. Street, N.W., Suite 300 Washington, D. C. 20006
- (202) 293-3950;
217 h
I what I have prepared.
2 MR. SilEWMON:
We are just about on schedule.
3 Thank you very much.
4 MR. DAVIS:
Do we have time for one comment?
5 MM. SilEWMON:
Sure.
6 MR. DAVIS:
As almost everyone knows by now, there 7
is a considerable interest in shutdown risk evaluations.
To 8
follow up a little on what Carl Micholson brought up 9
carlier, there are two rather significant efforts under way 10 at Brookhaven and Sandia to look at shutdown risks.
Those 11 efforts have come to the conclusion that it's appropriate to 12 divide refueling outages into a rather large number of 13 operating states which were selected on the basis that the (j'
14 plant goes through several configurations during the 15 refueling outage procedure and that in each configuration 16 the plant has a different vulnerability to certain failures.
17 I think for the BWR they are looking at it is like 13 18 operating states and for the PWR it's ten or 11.
I don't 19 remember the numbers exactly but it's on that order.
20 The problem when you do that is that you need to 21 have a lot of data for each of these states in order to 22 estimate the probability of a core damage accident.
23 It might be worthwhile to consider taking a look 24 at what they have done in these studies and see if there is 25 some way you can cut your data to be consistent with the G
ANN RILEY & ASSOCIATES, Ltd.
Court Reporters i
1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
218
(
1 needs for each of these operating states.
I realize it's a 2
difficult proposition, but I think eventually there is going 3
to be a lot more done in the way of calculating risk during 4
shutdown, and this kind of data is going to be desperately 5
needed to handle that.
6 Just a suggestion.
7 MR. CARROLL:
Why is this data particularly useful 8
for that purpose?
I don't see why you feel this kind of 9
information is that useful for these shutdown risk PRAs.
10 MR. DAVIS:
I think it should be if they are 11 looking at what equipment is out during certain phases of 12 the shutdown process.
13 MR. CARROLL:
That's available from other things 14 besides this.
15 MR. DAVIS:
It was difficult to get in these two 16 programs.
17 MR. NOVAK:
Let me make two comments.
One, even 18 to do what we have done has taken quite a bit of time.
If 19 you were to take, for example, a Westinghouse designed 20 plant, and let's say there was a period during shutdown when 21 they were in mid-loop operation.
To track that kind of data 22 so that you could go back to say, well, in the previous 23 refueling when they were at mid-loop operation did they have 24 an event, we would be really stretching our ability to track 25 all that data.
This is just a step improvement.
O ANN RILEY & ASSOCIATES, Ltd.
1 Coud Reponers I
l 1612 K. Street, N.W., Suite 300 Washington, D. C. 20006 (202) 293-3950
219 1
But to go back to your point, remember, we also 2
use the accident sequence precursor program.
So when we do 3
have an event where during mid-loop operation for this plant 4
they run into trouble, we will analyze that event.
So kind 5
of on a separate track we do look at the risk that we might 6
perceive from some of these events.
7
-This is a screening tool to look at the dozen 8
plants out of a hundred that we might want to begin to look 9
at more carefully.
That's the kind of thing this has 10 attempted to do.
It's one tool.
11 MR. DAVIS:
I appreciate that, Tom.
Of course 12 it's more than just events.
You also need to have the 13 availability of equipment during these shutdown operating 14 states, because you are going to postulate the occurrence of 15 an event.
Just a thought.
16 MR. CARROLL:
One comment I might make is I know a 17 number of us are on the distribution for these quarterly 18 reports.
They are available if others want to get on the 19 distribution.
I'm sure Mr. Igne can arrange for that.
20 Thank you.
Good presentation.
21.
MR. SHEWMON:
Gentlemen, I would like to now go on 22 to Charlie's letter on Generic Safety Issue 120.
23 We will go off the record.
24
[Whereupon at 9:30 a.m. the recorded portion of 25 the meeting was concluded.)
O ANN RILEY & ASSOCIATES, Ltd.
Court Reporters 1612 K. Street, N.W., Suito 300 Washington, D. C. 20006 (202)_293-3950
REPORTER'S CERTIFICATE O
This is to certify that the attached proceedings before the United States Nuclear Regulatory Commission In the Matter of t NAME OF PROCEEDING:
393rd ACRS Meeting DOCKET NUMBER:
PLACE OF PROCEEDING:
Itethesda, Maryland were held as herein appears, and that this is the original transcript thereof for the file of the United States Nuclear Regulatory Commission taken by me and thereafter reduced to typewriting by me or under the direction of the court reporting company, and that the O
transcript is a true and accurate record of the foregoing proceedings.
Abd nf
/
official Reporter Ann Riley & Associates, Ltd.
O l
i
_ _.. -.. - - - -. - _ _ _ _ _. - -. - - - - -. -. - - - -. -. - - - - -. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ~ - - - - - - - - - ' - ' - - - - - ' - ' - ' - - - - -
' - ' ^ ^ - - - ' ' ^ ^ ^ - ' ~ ^ ^ ^ ~ ~ ^ ^ - ~ ~ ^ ~ ^ ^ '
4' s
~
4
SUBJECT:
PROPOSED ENIIANCEMENTS TO PERFORMANCE INDICATOR REPORTS.
t DATE:
JANUARY 8, 1993 O
PRESENTERS:
PAT BARANOWSKY, BRANCII CIIIEF DON IIICKMAN,- SECTION CIIIEF h
I I
t BR/DIV:
TRENDS AND PATTERNS BRANCH DIVISION OF' SAFETY PROGRAMS:
u TELEPIIONE:
492-4480.
492-4431' li :
n l.
u L'.-
- 0?
'O.
O PERFORMANCE INDICATOR PRCGRAM.
INTRODUCTION
- SINGLE PROGRAM UNDER,AEOD
.. QUARTERLY. REPORTS SINCE 1987
- COMMISSION APPROVES INDICATORS
- RESTRICTIONS ON THEIR USE i
4
-..d.a-
l
- r[!!!i
- l ;!l!!
- ll!!!
lii
~
o S
L R
A C
E O
T R
I T
U IR S
A C
S SE O.
C N
E G
P L
O S
AS X
ID I
H T
E TR E
I A
R UU N
W U
U E
OO O
N.
I T
S L
T H
S C
T A
A D
T I
I E
M A
N F
R EL A
O C A
E CA ID N
R M
M E
RC V
A C
E E
G OT R.
S.
I E
A S
T T
A FI S
T S
T R
E.
E M
C Y
N Y
U TC V..
O.
D R
T S
A.
S O
N I
I E 0 T
C.
A Y
C Y
D 0
C.
O M
T F
T E
M I
0 E
E P1 F
. O E
N E
C L
S.
I I
T F
F R
UR L.
U
.R G
U A
A O
QE O
A I
E A
S S
S F
EP C
C.
P-o
! ;:!il:lj i
L
O O
O t
CAUSE CODES 1
}
o u
i-
- ADMINISTRATIVE CONTROL PROBLEMS
- LICENSED OPERATOR ERRORS
= OTHER PERSONNEL ERRORS l
- MAINTENANCE PROBLEMS L
- DE' SIGN,fCONSTRUCTION, INSTALLATION l
OR' FABRICATION PROBLEMS 1
P
= MISCELLANEOUS FAILURES i
p ll i
[,
u 5
OL O
O 1
I BACKGROUND 2
THE USE OF PEER GROUPS l,
t i
i f
- "CAUSE CODES" IDENTIFY PROGRAMMATIC i
.CAUSES.OF EACH REPORTABLE EVENT i
- COMMISSION APPROVED TRENDING l
OF "CAUSE ' CODES" FOR.EACH PLANT r
t i
o, DIRECTED DEVELOPMENT OF PEER' GROUPS 1
L FOR4 COMPARISON OF "CAUSE CODES" o
[
i l
l
~
o o
.l o
BACKGROUND l
THE EFFECT OF OPERATING CYCLES
]
= "CAUSE CODE" DATA CYCLIC l
WITH REFUELING INTERVAL
= OPERATING MODE. AFFECTS RATEiOF REPORTABLE EVENTS
- EVENT TYPES VARY. WITH
. OPERATING MODE I
a 10 O
O ENHANCEMENT
SUMMARY
- l i
l
- INDICATORS FOR BOTH OPS AND SHUTDOWN 4
- INDICATORS RELATIVE TO SIMILAR PLANTS
- STATISTICAL SIGNIFICANCE OF INDICATORS i
i
OL O
O s
DEVELOPMENT PROGRAM 1
e o
p, 1
i g
p.
i:
l l'
a L,
[;
- PEER GROUPS - INEL 1
i I '.
- OPERATING CYCLE. PHASES - ORNL 1
l I
i
- - CALCULATIONAL AND DISPLAY METHODS - BOTH L
i-
- :TRIALL PERIOD - INTEROFFICE TASK GROUP l
i1 l
t t
U
[
e 1
~
- 01 0
O I'
PEER GROUP FACTORS g
a i
L i
I
= DESIGN i
NSSS VENDOR
- 4 PRODUCT LINE i
GENERATING CAPACITY l
AGE-L L'
L
- REGULATORY ISSUES I
- cTYPE OF LTECH SPECS L
. MINIMUM GROUPLSIZE 4
L
- EINPUT FROM OWNERS' GROUPS 1.
[
i i
..... ~
l l;!!t !
i!I il; 1 !l!
iii !
S T
N A
L u.
P S
S T
T E
N N
S a
A A
U S
S L
L O
P T
P P
H N
E E
G U
A S
S N
O I
L U
U T
P O
O S
R E
H H
E G
C S
S G
G W
P S
T U
N N
P R
C C
T N
O I
I T
T O
S P
N A
H S
S E
T O
C A
L G
E E
O L
E N
/
/
L P
N W
W A
W W
P E
T 4
I P
. L E
G S
P P
D
. P S
S T
T G
E O
O N
I M
W O
O A
W N
N I
L L
A A
M T
. B L
L T
L 4'.
3 3
P P
T L
L E
S A
D D
W L
E E
R O
M L
L E
. A C
C P
P S
O O.
N
.,.L il:i; a
l L
jb
.i: tI
o o
o OPERATIhG 34ASE 3 ASIS
- CHANGES IN RATE OF REPORTABLE l
EVENTS WERE OBSERVED
. REPORTABLE EVENTS VARY WITH PLANT ACTIVITIES
- PLANT ACTIVITIES CHANGE WITH OPERATING PHASE
- IDENTIFY PHASES IN WHICH PLANT ACTIVITIES VARY SIGNIFICANTLY
l
!t lIt !
!1! f :ii l
l i,
t o.
).
S E
G
)
S G
A.
N T
E U
I L
S E
O A
U G
F H
E N
I R
L P
E.
E U
E R.
F S
P E
L E
R.
C G
S A
P N
Y T
O O
_ oC EU N
S N
R O
GO E
D A
S G
TGTG E
W N
I c
A N UN P
O A
N RI OL YS P I
E L IT E
TN NG A
S P E G U OPWIN O U N F STUOL.
EI E P FI R
S U RELE A ATD E A T ER ER H R U-H RRT U.
E EAU F P
PA WEF N P
PT H E.
TOREO O O (S S (R.
O E
SPPRN V
W T-IF
- o 4
1:
L L
!i; i
i
- 4
.LOL
. O O-CALCULATIONAL METHODS L
L l
p' i
Q
.* TRENDS FROM LINEAR REGRESSIONS j
l s
[
- DEVIATIONS.FROM PEER GROUP MEDIAN i
i SCALEDL FOR ' COMPARABILITY-i
'
- STATISTICAL SIGNIFICANCE FROM y
MONTE CARLO SIMULATION 7
T-i' j.
'j I
t
Q kdColor Legend: - Critical Hours N-et6otr*-9^-w eusses IVt (i Qurs ter M3vng Average 90-4 to 92-3 OUARTERLY DATA R *w t -m '"'*)
- 1. Automatic Scrams While Critical
- 2. Safety System Actuations 4_
7t00 4
2500,
- 27!c f
{
7230 y }, *
- \\*-
/',f.
1750 N
'\\
2000 yG ~
. 2XD
.J
~
too e
- 4
\\
1750 $
g gg
\\
. U%
%g2
(
k F.00 e
% *p ? _
1250 i
./
\\
too
~f
.,V.
\\
110 o h" ;f, -
\\/
\\
t
\\g 3
. 7$0 y
( 3,-
\\,
- N y
_ soo 3 a
_ soo 3 zz
- 2%
0 M4 W
D 0
- i 0
Wb4 95 1 9d2 9$3 9E4 92'-1 92+2 92-3 90b4
-t 9 2 tb3 954 92' 9 92'-2 92'-3 Y eor - Quar t er Y ear - OJy t er J.
Significant Events
- 4. Safety System Failures C
l ' 3_
s 223.
a %,
F 4
t
.J". cr 2-1 1
.2.
- 1. L t
?j i_
t.
4 es*h
[
$l o
a 10 -4 96-1 9,.-2 91-3 91+4 92-1 92-2 92-3 90-4 9%1 95-2 943 91-4 92 -1 92-2 92-3 y ear - Quy t er i ear - Dunrter
- 6. Equipment Forced Outages /
- 5. Forced Outage Rate (s) 1000 Commercial Critical liours to
- e L
U h [* * -
E" 6 -
,, i
~
pm a
e i
p 3
y ao _
m
~~
v*
/-
s 2.
I[
s w
i.{. 7 7
..g o
o.
t.
_7. 9,., 9.7.
7,- 9 92., 92., 9,3
..g.
g q>
.. 9-,9
,9,.
9,....,922...
29 ieor - Quar ter Y eor - Quarter
- 7. Collective Radiation Exposure two,
- 8. Cause Codes T
- 2* f ADMIN L1C OPER OTi!ER PER Y. to _
14 9
~ ** g f.g v
. nso -
}
f g.
' q r
M m>oo =q m.
J,?
N,,
too 3 a-Q
~ 73o M MAINT DESIGN EQUIP FAIL B
50 -
Sj
- 500 u 39 w
s G
o o
m-. 9 s, 9,_2 t o 9,_4 m., m-2 u-3 Y ear - Quarter
Third Quorier 1992 Quorterly Doto Legend Plani iretv3 StariUp t 1
Feer Avg Trend Cperotion L
,,,,,J Peer Group. Comt>ustico Engineering withou't CPC h);stry Ayg Trend
% tDoon L T Aefvenrg R
Hot Shown Using 0p. Croe GZG Li.
a.
R R
R R
s/o s/o y y g p
.. y 7 g,
.,7 g,
T eor - Osor ter Year - Qwter Automatic Scroms Whie Critied Safety System Actuaticro 4-4 F
3' 3.
e3-o u*
"2-
%2.
1 g
h i'
{ i~
N fy' R1 2
idI St.ymIf~i"12fr 0 - T "rJ"fs*rd'ly'",w v w s" Ahi
,,_ 3
- c.,
0;.i 1
Year - Ou ter y ear - Quar ter Sipficant Events Sofety System fcikses
?
k'-
0
" 10.-
A 2.
J L
3 m-1' m-=
c+_lesA 00 hs h>- J ia W W 6 6- >
w.s U-J i
w.y w_3 igwis aw}
6fg. :
r*
i o
Yeor - Outs ter Year - Quarter Eggrnent Forced Outoges/
Fcreed Outoge Rote (*)
p 000 Commercia tburs 5
go E PJ.
0 7'-
hw
,7 p
"3 49,
f ta?
r 6mme-of sm;u 9
l l
rME_N hMMMMSM455)MS) o O
h92 I'2"I" ~ ' -- # [';" *!
^5 J t.
I WI ki-J is h k Yb}
% -I E2-J i
UM 90-.J iss6 bw}
Na 3-' 'D2-3 I
Year - Ouarter Year - Quarter I
l Cause Codes t
Rcdotion Expostre o Adrrun
- b. Lie Oper
- c. Other Per 5
e a
- co
_.Q m
s m
l
(
gn e,
e.
e e
h**
[
u3 fg 8'
8 8-g WE L
m adh rp,
!I m_
Mi s
nl s{::
n.,i m
.g 1
@m.
- t' 4 N....,. G
- d Leint.
- e. Design
- f. use.
g w-3.
. 7
--- E M. s 0.,
- e. i
-1
.~
m, t
Ytar - Qucrter Sie Average Ro$ation {sDOSJO O P,.,. r 0
t T
- - - * +
-w-ie i
a t
k e--
a r
e-ru
~ '
e.t*
6 Indcolor n.
Legend:
Oioer Piont 6 Otr uovia9 Avero9e Critical Hours
- ()'
summme Plant 6 Ouarter Moving Average (Long Term Trends)
- 2. Safety System Actuations E
22503 o
a.
av 2000 18 8' 88's.
tb
/\\
./..
l '...\\
_ t250$
4$
j
\\.
1500 a
/
% e2_
\\
..,...,.i w1 e-S
_ 750 8 s 1.
\\3 -
z mg 250 0_
0 90-4 9b1 ' 9b2 9b3 S he 92-1 92-2 92 Yeor - Quorter 0
m-Legend:
Ptont Trena StortUp 1 i
Peer Avg. Trend Operation tMM3 industry Avg. Trend Shutdown l355553
. Ref ueling R-NDt Shown Using Op Cycle ex<4 ups.
R R
S/D i
ago-s -
90- 4 9wu 19 5-y 92-i 92-3 Year - Ouorter Sofety System Actuations 8.
=83_
y 4
o2.
d.
.a -
1 1
430- V 00-2 t
89hU 19 b33 19 2 - 1 U2-;i Year - Quorter
x (3
. w/;-
Deviations Short Term -
from Older Trends-Plant Long TRENDS & DEVIATIONS Term Means Declined improved Bebw Avg. Abc<e Avg.
t Automotic Scroms While Critical.
o 14 Ik
- 2. Safety System Actuotions.
og f@g 02 ab
-"?
- 3. Significont Events -
o U
o.5
\\
=
G FitM hl 4 Sofety System Fodore5 -
eo.5
-10 n()-
[g sh ssi emow t. ew %.
~
- 5. Forced Outoge Ro*e -
.t2
-10 fi Cok rNoN krN* Hours -
12~
-0.9
- 7. Cause Codes (AR LERs) 7 Aa.m trotw c.*w Praer
-2.5 ' I.5 - -0.5 O!5 l'$
2.5 7s.tk-.m c wain ere r..
Meu from O*r Ptmt Long 4.
Term Means 7e.on P-wwtar.
(u.a.w.e w stone 14 o waw) -
7d Maintewze Prdem -
. (6 Ott Avg end 92-3).
47
- 7. o,v wt.*si.vra mouon ere w.
7f. Es emert Fonse s
s
-4.:>
I 5 -d 5 05 t'5 2.5 Owiatow from Cwrent 6 oir Phrt Mars (Measwed h Stanord Devaew)
(2 Otr Avg end 92-3)
I
Third Ouorter 1992 Trends ond Deviations
,n
(
j' legend Statisticar S4gnificance ns MME LJ.
~
"'# N Peer Group Combuston Engineering w/o CPC Los F-~1 Plant Deviations From Self-Trend Peer Group Median OPERATIONS Dechned impr oved Bebw Above Automatic Scroms While Critico!
o 0.2 Sofety System Actuotions -
o o
Significant Events -
o o
Sofety System foikres -
os
-02 Cause Codes (ALL LERs) 3p~
-o2 L
. m si.- ce,an =
m
,a...~~.
4r
- c. Otrer Pesortw4 (rror _
g,
-c,
_. ~ ~ _
3.,
. 3g nV 3,,
-.c
, e,i-,--r.r-Pe
.. ~. ~.
SHtJTDOWN
' hsy
0 Sofety System Actuations 09 Significmt Events -
o o
Sofety System foikses -
03
-o.2 Couse Codes (ALL LERs) a Aene.strotw comre Pronom-
] a.2 8
7]
.o i{
b Leecues 4=otar Protem..
en
- c. Ouw Fesw trrar _
g e
If hk*Midftrice Preh.
g4 O
e cesar/kmi invaa/Fencokwg Prm 63 O
t hanceamma _.
o 0
FORCED OUTAGES
"]
-o 4 g Forced Outoge Rote * -
os om r io s -
f o5
-o 2 d 5,,,monc,vo !5 00 O
10
-10 d5 00 0'S 1.0
-10 e tot Ccicdated for Operot.oruf Cycle p,,
Pcforrnonce ex1es g"N,
g"%g?s ne V
D'2 U NITED. STATES Q L j/!
NUCLEAR REGULATORY COMMISSION g..v ANNOUNCEMENT NO.
200 DATE: November 28, 1989 All NRC EMPLOYEES TO:
SUBJECT:
REVISED GUIDANCE ON THE USE OF PERFORMANCE INDICATORS This announcement revises the earlier guidance of HRC Announcement 30, dated February 5, 1988, regarding the use of the results of the NRC Performance Indicator Program. All NRC employees shall adhere to the following guidance.
The Performance Indicator Program provides an additional view of operational performance and enhances our ability to recognize areas of (d
poor and/or declining safety performance of operating plants. However, G
it is only a tool and is to be used in conjunction with other tools, such as the results of toutine and special inspections and the systematic assessment of licensee performance (SALP) program, for providing input to NRC management decisions regarding the need to adjust plant-specific regulatory programs.
It should be recognized that performance indicators have limitations and are subject to misinterpretation. Therefore,' caution is warranted in the interpretation and use of the data. The application of performance indicators for purp3ses and in manners other than those stated above will be counter to the NRC objective of ensuring operational safety. To avoid such situations, the following specific directives are provided:
l.
The Performance Indicator Program for operating reactors is a single, coordinated, overall NRC program under the direction of AE0D. NRC offices other than AEOD should not deviate from the HRC program without written permission of the EDO or the Director, AE00.
l w.._,,, o.,
j j
L Performance indicators are intended as a tool for senior NRC 2.
management to monitor trends in overall performance for a given plant. The performance indicators for a given plant should be viewed as a set. When viewed as a set, the performance indicators provide an additional measure of plant. operational performance.
However, they should not be used in communications with licensees as a measure of performance level.
Performance indicators are intended to be one of several tools 3.
for use by senior NRC management in decision-making regarding plant-specific regulatory programs. Senior management in each NRC office should have access to performance indicators for their assignedunit(s). Performance indicators are not to be overemphasized in relation _to other measures of safety performance.
For this reason, no regulatory acticn should be taken on the basis of Performance Indicator Program results alone.
4 Performance indicators do not provide a valid basis for ranking individual nuclear power plants and should not be presented in such a way as to imply " problem. facility" status for individual plants.
The Performance Indicator Program is separate and distinct from 5.
the SALP program, although it is recognized that the indicators qV have relationships in varying degrees to SALP functional areas.
Indicators, such as failures of a plant's safety systems or frequent forced outages due to equipment failures, may be symptomatic of safety problems. Thus,~the staff may recognize events and failures captured by certain indicators in SALP-discussions and reports, but these SALP references are to be based on the underlying causes of-poor performance and not on the results of the Performance Indicator Program, either individually
=
or as a set.
Regional Administrators should ensure that our decision-making process adheres to this guidance, especially in SALP discussions and documentation.
u 6.
NRC senior management should-bear in mind when evaluating performance indicator results that the indicators are assessment tools that aid in identification of unanticipated performance, and that the underlying causes should be carefully assessed, evaluated, and understood (factoring in other available information).
7.
Quarterly compilations of Performance Indicator Program results should be placed in the Public Document Room following dissemination to NRC management and the Commission.
O v
, (~\\
' () -
It should be recognized that in conducting reviews, inspections, and evaluations of plants, it is often necessary to rely on plant data.
Such information has been routinely used in our SALP, safety evaluation reports, and technical evaluation reports. The foregoing policy is-not intended to change this process.
NRC staff must be sensitive to inappropriate pressure from any source which causes licensee personnel at individual nuclear power plants to
" manage the indicators" or to take any actions that are contrary to plant safety because of performance indicators, individually or as a set (such as inhibiting reactor trips). Any such instances should be promptly communicated to appropriate licensee management and brought to NRC management attention.
- /
W<
J,es M. Tay r cting Executive Director for Operations O
9 O
_