ML20215A071
| ML20215A071 | |
| Person / Time | |
|---|---|
| Issue date: | 06/09/1987 |
| From: | NRC COMMISSION (OCM) |
| To: | |
| References | |
| REF-10CFR9.7 NUDOCS 8706160490 | |
| Download: ML20215A071 (66) | |
Text
N8WN8WWWWWWWW WWWWWAWWWWWWWWWWgggggggggg ij' TRAt!SMITTAL T0:
X, Document Control Desk, 016 Phillips Vi ADVANCED COPY TO:
The Public Document Room b,!//,!h7 DATE:
SECY Correspondence & Records Branch 5
FROM:
)
I l
Attached are copies of a Commission meeting transcript and related meeting 1
ii, document (s). They are being forwarded for entry on the Daily Accession List and
- j placement in the Public Document Room. No other distribution is requested or j
required.
Myr me.e M dr Meeting
Title:
D t S
- u_%
0 9/om p S
[
li Meeting Date:
G.\\%k87 Open y
Closed l
i ll k
f!
i, i:
Copies
[
+
ij Item Description *:
Advanced DCS E
llj
'8 E
to POR Cg jl 5
E l
1 1
lf
- 1. TRANSCRIPT-E
{A)] d ta-%fcL9 hip E
i 2
E I
E I
3 :
E 2-
- 2. 94 %. I9-RD E
t 2.,
[
Sf
- 3. S e N d 'l - L L 1 C.hrr M h 2
3 kh kC 8-2:
4-I M 3
!!i 5-t ai' 5:
$ g 6.
2 3.
3: :
- POR is advanced one copy of each document, two of each SECY paper.
3i C&R Branch files the original transcript, with attachments, withcut SECY g
- E 3
$ i 8706160490 870609 _.
3 papers.
$$$$$$$$$$$$$$$0 YSYNNNNbYNNI lYNlYNlYNhNNAW@lY p
"0RIGINA!.
UNITED STATES OF AMERICA NUCLEAR RIGULAT,ORY COMMISSION
Title:
Discussion of Performance Indicator Programs (Public Meeting)
Location: Washington, D.
C.
Date:
Tuesday, June 9, 1987 Pages:
1 - ss Ann Riley & Associates l
Court Reporters 1625 i Street, N.W., Suite 921 Washington, D.C. 20006 (202) 293-3950
.. ~
.o 1
'D 1 SC LA I MER 2
3-4 5
6 This is an unofficial transeript of a meeting of the-7 United States Nuclear Regulatory Commission h eld on a
6/09/87, In the CommissioIn's office at 1717 H Street, 9
' N. tJ., (Ja s h i n g t on, D.C.
The meeting was
'1 open to public 10 attendance and observation.
This transcript has not been-11 reviewed, corrected, or edited, and it may contain 12 inaccuracies.
13 The transcript is intended solely for general 14-Informational purposes.
As provided by 10 CFR 9.105,-it is 15 not part of the formal or informal record of decision of the 16 matters discussed.
Expressions of opinion in this transcrip.t 17 do not necessarily reflect final determination or beliefs.
No 18 pleading or other paper may be l
filed with the Commission in 19 any proceeding as the result of or addressed to any statement 12 0 or argument contained herein, except as the Commission may 21 authorize.
22 25 J
24 1
25
4 1
6 l
IL UNITED STATES OF AMERICA P
2 NUCLEAR REGULATORY COMMISSION 3
Discussion of Performance 4
Indicator Programs 5
6 Public Meeting 7
8_
1717 H Street, N.W.
,9 Room 1130 10-Washington, D.C.
11 12 Tuesday, June 9, 1987 13 14 The Commission met in open session, pursuant to 15 notice, at 2:02 o' clock p.m., the Honorable Lando W.
- Zech, 16 Jr., [ Chairman of the Commission] presiding.
17 j
fl8 COMMISSIONERS PRESENT:
l
.19 Lando W.
Zech, Jr., Chairman l
20 James K. Asselstine, Commissioner 1
21 Frederick M. Bernthal, Commissioner
.22 Thomas M. Roberts, Commissioner 23 Kenneth M. Carr, Commissioner 24 25
2 1
STAFF PRESENT AND SEATED AT TABLE:
2 V. Stallo, EDO 3
J. Rosenthal 4-E. Jordan 5
R. Singh 6
S. Chilk, SECY
-7 W. Parler, OGC 8
9 10 11-12 13 14 15 16 17 l
18 19 20 21 22 23 24 25
3 1
PROCEEDINGS 2
CHAIRMAN ZECH:
Good afternoon, ladies and gentlemen.
3 The purpose of today's meeting is for the Staff to 4
brief the Commission concerning the current status of NRC's 5
performance indicator program, and to discuss the development 6
of a coordination plan for NRC and INPO to use these 7
performance indicators, and the discussion, I understand, will include the issue of the disclosure of INPO performance 8
indicator data which I understand has been satisfactorily 9
10 resolved.
11 During the presentation, Mr. Stello, I would like you 12
.to specifically address the status of development of a g
maintenance performance indicator, and perhaps other 13 14 indicators.
15 I understand that copies of the slides to be used 16 during the presentation and the NRC-INPO coordination plan 17 should be available in the rear of the room.
18 Do any of my fellow Commissioners have any opening 19 comments to make?
20 COMMISSIONER ASSELSTINE:
No.
i 21 CHAIRMAN ZECH:
If not, Mr. Stello, you may proceed.
22 MR. STELLO:
Okay.
I will turn to Mr. Jordan in a 23 moment.
With respect to the INpo coordination plan, we sent 24 down the revisions to the Commission dated May 7th.
The change 25 that was made in the original proposal that was sent to the
, _ _. _.. _. _ _ _ _.. ~. _.
.c 4
s-1-
Commission was to. identify the performance indicators that we 2
did receive from INPO, together with the indicators we 3
developed ourselves,.that we would combine those and place the 4
entire performance indicator package-in the Public Document 5
Room.
6 This was a particular area of some debate with'INPO.
7 They have now agreed to go forward on that basis, at least 8
discussions with them have indicated they are prepared now to 9
sign and enter into that agreement, and I understand we worked 10 with OGC to assure that it in fact -- there is no legal 11
' objection, and it is my understanding there is not.
And with 12 the Commission's approval, we would move forward, execute that 13 agreement.-
14 There is one more piece of paper that we wish to 15 generate that I think is very, very important, and that is a 16 policy paper to put on top of the performance indicator that 17 cautions the proper and improper uses of them, because there is 18 some controversy about them being used for purposes other than 19 for (a) what they were intended and (b) what they really can 20 do..
21 Once the Commission agrees to move forward, we would 22 develop that policy paper and then execute the program as 23 outlined in the paper before the Commission.
24 CHAIRMAN ZECH:
All right.
Before you proceed, I 25 understand we have SECY 87-117 before the Commission, and after
5 1
.the meeting today,-I would hope that if we haven't acted on 2
that, we could act on that paper.
3 COMMISSIONER ASSELSTINE:
Sure.
4 CHAIRMAN ZECH:
That's what the meeting is for, in 5
great part, anyway.
6 All right, proceed.
7 COMMISSIONER ROBERTS:
You mean act on the SECY paper 8
at this meeting?
9 CHAIRMAN ZECH:
After the meeting.
After the meeting 10 is over with today, and we hear the briefing --
11 COMMISSIONER ASSELSTINE:
Then we vote.
12 CHAIRMAN ZECH:
-- then we vote.
I don't intend to 13 vote at this meeting today; kind of reflect on what we hear and 14 vote in the next few days if possible.
15 Any other comments?
16 All right.
Pr'cceed.
17 COMMISSIONER ASSELSTINE:
But that would include the 18 modifications that we subsequently received as a result of the 19 General Counsel's good actions?
i j
20 CHAIRMAN ZECH:
Absolutely, i
21 MR. STELLO:
Those are the ones I have described.
It 22 is the modification to the paper.
23 CHAIRMAN ZECH:
I think it satisfactorily resolves 24 it.
It seems to me --
25 COMMISSIONER ASSELSTINE:
The June 2nd corrections?
l
6 1
MR. STELLO:
Yes.
i.
2 MR. PARLER:
That is, if I may just. speak to the 3
legal point, Mr. Chairman, to-get that out of the way.
-4 CHAIRMAN ZECH:
Please do.
5
. ER. PARLER:
It's a problem that was' raised by'an 6
earlier paper which was a problem created by the lack of 7
internal OGC coordination, for which I accept full 8
responsibility.
It was resolved by the June 2nd correction, 9
and it was resolved by taking out a paragraph on page 3 of the 10 old paper at a paragraph number 5 in the attachment to the old 11 paper, both of which, although they were offered by the OGC 12 folks with good intentions, created a problem because they 13 indicated to me, at least, that information that should not be
- 14 withheld from NRC people as well as the public, at least in i
15 theory could be withheld by being labeled as proprietary.
16 What the folks, the legal folks, intended was that 17 withholding would be in accord with the old agreement that we 18' had entered into with INPO, which had been litigated, which we 19 have won at the federal District Court level, which is on 20 appeal now, and which has been argued, and which we are 21 awaiting a decision on.
l l
22 But that problem has vanished because the troublesome i
23 words have been taken out, and no troublesome words, at least 24 from my standpoint, have been substituted for them.
l 25 CHAIRMAN ZECH:
All right.
Fine.
Thank you very i
.g
,,ey gy,ym wyy---r--w
---rvw-m'"
"S"'-t'"rT-
- ""N"lW T#
s.
7 1
much.
2 You may proceed.
3 MR. STELLO:
After the introduction, Mr. Chairman, I 4
will turn to --
5 CHAIRMAN ZECH:
And we appreciate your support for 6
that, Mr. General Counsel.
I think you have clarified the issue and made it a more professional product, and I thank you 7
8 for that.
J 9
Proceed, Mr. Stallo.
'10 MR. STELLO:
I will now turn to Mr. Jordan to give 11 you a very quick summary of our performance indicator program.
12 MR. JORDAN:
Okay.
I brought with me Jack Rosenthal,,
13 the branch chief responsible, and Robbie Singh, who is the 14 acting section chief, and so they can respond to more detailed 15 questions.
16 I would like to have the first slide.
17 The things we will go through rather quickly are the 18 status of the program, our plans for development, coordination 19 with industry and INPO.
20 I would make an opening statement indicating that 21 thus far I believe that the performance indicators are serving 22 the purpose that was intended and as briefed to the Commission i
23 in November, in being able to promptly go through a large group 24 of plants and identify outliers, and then look deeper in other 25 areas to try to support that finding.
es n,
-,,.,-w,,-.,e----,,,-------,--,-
-,,,,.r.--,,-.
n
,.---w_n-,-m----
8 e
1 Let's-have the first - ~that's it.
2
.Okay.
The actions that were required, based on the 3
commission meeting,-were Staff policy.
Mr. Stallo has
-4 mentioned that we a.re going to develop that as soon as we have 5
the agreement with INPO formalized.
6
'The deletion of the enforcement action index was 7
recommended by the Commission.
It was an action that we have 8
taken.
It's no longer a part of the performance indicator set.
9 We have taken into account the INPO concerns in our
~
10 communications of-the packages to the personnel that are using 11 them.
12 We are. continuing to explore indicators in several 1-13 areas.
14 Next slide, please.
15 I would just simply recount quickly the indicators 16 that we currently have in the program.
Total automatic scrams, 17 and that is an indicator that is in common with INPO, identical 18 definition.
Safety system actuations, also identical to the 19 INPO definition.
Significant events are events that are 20 screened by the Staff, 7t '.s is through review by NRR against 21 criteria they develvped asdntly with I&E before the L
22 reorganization, and this is done.in a real time basis.
Plants l
23 are then identified as this particular event becoming a 24 significant event.
So that tally is carried forward, is not an L
25 INPO item.
l
9 1
COMMISSIONER ASSELSTINE:
Could you give just a 2'
couple of examples of those kinds of events?
3 MR. JORDAN:
Jack, I will ask you to give a couple of 4
examples.
5 MR. ROSENTHAL:
Sure.
Let's say a reactor trip due 6
to loss of feedwater, in which you lost one of three trains of
.7 auxiliary'feedwater, we would consider significant.
That same 8
trip, where all engineered safety features worked as designed, 9
we would not count as a significant event.
10 COMMISSIONER ASSELSTINE:
Okay.
11-CHAIRMAN ZECH:
All right.
12 MR. JORDAN:
Safety system failures.
These are cases 13 in which the entire system was unavailable, failed to operate.
('
14 And that is not in common with INPO.
15 Forced outage rate is the type of information that is 16 also consistent with the INPO definition.
17 The equipment forced outages per 1000 critical hours, 18 that is the closest thing we have to a maintenance indicator in 19 the SIT.
Otherwise, they are clearly secondary, and that's not 20 one that we have in common with INPO.
21 COMMISSIONER ASSELSTINE:
Does that actually look at i
22 equipment failures?
Is that what you do basically, look at the H23 number of failures that forced an outage, so that you actually 24 have a failure?
25 MR. JORDAN:
Yes.
I
10 l'
COMMISSIONER ASSELSTINE:
Okay.
And then average it 2
out over a number of hours.
3 MR. JORDAN:
That's correct.
4 COMMISSIONER ASSELSTINE:
So you get an equal 5
balance.
Okay.
6 MR. JORDAN:
And the last one is the one the 7
Commission added, collective radiation exposures.
We currently 8
have annual data.
INPO collects that same data on a monthly or 9
quarterly basis, so this would be a substantial benefit to 10 obtain that information from INPO.
11 The other six indicators, the Staff currently obtains 12 from information that is reported to the NRC.
No additional e
13 reporting has been required for use of thos.e items.
It does 4
14 require some additional Staff review or contract review to 15 extract information from the utility report.
16 Thus far, we have provided two quarterly reports to 17 senior management, one in February and one in May, compiling 18 the performance indicator data, and those reports will be 19 coming to the Commission shortly.
20 COMMISSIONER ASSELSTINE:
Do you know when we will 21 get them?
22 MR. STELLO:
Let me -- what I would like to do is 23 before I send them -- I could send them to the Commission 24 tomorrow; that's no problem.
But before I make further dissemination, I would like to have this policy statement put 25
11 1
on top of it to guide for the use, so you don't have them 2
, separated, with the actual data not having that policy 3-statement, which is of concern, I think, to all of us, as well 4
as it's'a major concern to INPO, in terms of misuse as well.
5-So I would like to have that, if I could.
If there 6
is any sense of urgency, I certainly could just reproduce them 7
tomorrow, if there's a need.
8 COMMISSIONER ASSELSTINE:
I'd kind of like to see 9
them.
The last one I saw was the one from last year.
It would 10 be interesting to_me to look at --
11 MR. STELLO:
Well, is it acceptable to wait until we 12 have the policy statement?
13 COMMISSIONER ASSELSTINE:
Sure, as long as it would 14 just be within a week or so.
My window is --
15 (Laughter.)
16 MR. STELLO:
I've got to get some sense of agreement 17 from the commission on the memorandum, and then develop the i
18 policy --
l 19 CHAIRMAN ZECH:
Well, we will try to vote on the SECY 20 paper within the next week, if we can.
I think we should be 21 able to do that, and from there you ought to take -- from there 22 you shouldn't take an awful lot of time beyond that, it seems i
23 to me, to be able to --
l 24 MR. STELLO:
A week after you give it to us.
25 CHAIRMAN ZECH:
Okay.
All right, let's do that.
12 1
- All right, proceed.
2 MR. JORDAN:. Okay, enxt slide, please.
3 The continued development - _the interoffice task' 4
group that we established last year, we are maintaining.
There
, is some change in membership based on organizational changes, 6
but we are meeting this week'to review the development activities, to get their feedback into this process once again.,
7 8
The things that we are doing are refining the 9
program, trying to improve the displays.
The packages are 10 still weighty, and we would like to try to resolve just the 11 bulk of the material.
12 We are, of course, devoted towards getting closer to 13 risk-based performance indicators and including some f
14 programmatic indicators.
15 We expect to make a full report to the commission in 16 December as we forecast in-the last Commission paper.
We have 17 looked at plant type and age in our review of the data, and I 18 guess somewhat s.orprising to us, the plant type has not made a 19 lot of difference, whether it's a BWR, a PWR, or within the PWR f
20 set, different NMSSS's.
There's no great sensitivity.
21 COMMISSIONER ROBERTS:
Is that true on your 22 collective radiation exposures also?
23 MR. JORDAN:
No, not true.
Clearly the BWRs and the 24 PWRs have a different overall exposure, but in terms of plant 25 trips or safety system failures or forced outage rates, those T
w-,
c---,,e--
--.w,
-.q,
---+-,-g.,-..n e-
--n-,
-,--,..r--e---
-w-.,
v------
---,m-.-
,---e-
13.
I
. things'are not -- don't seem to be strongly correlated with 2
1 plant type.
3 Age.
Certainly the younger plants go through a 4-shakedown, a learning curve, and so we are comparing presently 5
plants in the first two years of operation as one population, 6
and that helps our understanding quite a bit.
7 We continue to have statistical issues in terms of 8
these population groups and the statistics of small numbers and 9
skewed distributions, and if you would like to have further 10 discussion of that, we could provide it, but it's almost 11 academic.
12 The risk-based PIs, as I indicated, are something we 13 feel very strongly about.
We are looking at schemes to
(
14 risk-weight some of the existing PIs even more strongly.
15 We are very interested in the train unavailability.
16 We have several tests running with utilities on train 17 availability type data, and combining integrating transient 18 arrival with unavailability.
19 COMMISSIONER ASSELSTINE:
On risk-weighting, could 20 you give me an example or two of some of the indicators that 21 you think fall in the heavy and of the risk scale?
22 MR. JORDAN:
I will ask Jack Rosenthal.
23 MR. ROSENTHAL:
We think that we should risk-weight 24 significant events and safety system failures.
25 COMMISSIONER ASSELSTINE:
Okay.
l 1
4
.-.--,,,,--,-..,1,-.,,
,---,n,n---,
-,--w,-e a,.,
.m,
. - - -..-n.,----m.__,
---,c,-----.
n-
4 14
?1
.ER. ROSENTHAL:
It just doesn't pay to do every trip.
2 The advantage of being quantitative is it takes out some of the 3
~ subjective judgment of one group or another about what was 4
important or what wasn't, so wa would definitely do the 5
significant events, which has a subjective aspect to it now.
6' On safety system failures, it is just clear that a 7
diesel failure, aux feedwater failures, high pressure safety 8
injection failures, are more important than a control room 9
chlorine monitor type thing.
10 COMMISSIONER ASSELSTINE:
Right.
11 MR. ROSENTHAL:
And in turn, we can get a sense of is 12 the utility putting its resources into those items that are 13
'truly important.
14 MR. JORDAN:
I would like to go to the next page, 15 please, and the first item there is maintenance items out of 16~
service.
17 COMMISSIONER ASSELSTINE:
One last question on the i
l 18 risk base.
19 MR. JORDAN:
Yes.
20 COMMISSIONER ASSELSTINE:
Do you also have a way to 21 look at repeats?
Can you get a historical look to find out not 22 only is there one of these fairly significant problems, but 23 it's the same problem over and over and over again, if you 24 looked back historically, have they had repeat problems with 25 say --
= -.
15 1
MR. ROSENTHAL:
We could do that, yes, sir, except 2
that we have other indicators that we are exploring, for 3
example, safety system rework, which has a lot of face validity, which would point directly to that issue that you 4
5 have raised.
i 6
. COMMISSIONER ASSELSTINE:
Great.
Good.
7 MR. JORDAN:
And that safety system rework is one of 8
the maintenance indicator schemes that we are quite interested 9
in.
4 10 COMMISSIONER CARR:
Why do you restrict that only to 11 safety systems?
12 MR. JORDAN:
It's a good question.
The presenting 13 reporting requirements of the utilities are safety systems.
14
.The NPRDS system would be where you would have to go for other 15 than safety systems, for data that is presently in a system, 16 and there's not a requirement for reporting utilities.
It's a 17 voluntary scheme.
So there is not a strong consistency between 18 the utilities.
So it certainly would be comparable, subject to 19 20 COMMISSIONER CARR:
Maybe we ought to ask them to 21 report it.
1 22 MR. JORDAN:
That is an old issue.
We'd like to, I 23 think, decide what kind of data we need and then decide on the 24 way to report it.
So we are working now on a more theoretical 25 basis in terms of what kind of data do we need, and then if we
16 1
need'to come to you with a change in reporting requirements, 2
then we would do that.
3 MR. STELLO:
I'd give, I guess, maybe a more 4
simplistic answer.
The first area of maintenance that we are 5
most-interested in is where the maintenance can have the 6
largest impact on safety.
That's the safety system.
We are 7
not so sure how to do that yet.
But if we get that working, f
8 then clearly you look at how to expand it.
9 I think the NPRDS system may hava enottgh information 10 in it to help us, but it depends on what the program is.
There 11 is a reporting system for other.than the safety systems that is t
12 now in place in NPRDS.
So it is captured.
How useful this 4
13 will be, I don't know that we know yet.
14 MR. JORDAN:
We are looking at training the operator 15 licensing exam results, operator error and so on.
We 16 discussed, I think at the last commission meeting, cause codes.
17 We believe that is still an amplifier of each of the indicators 18 that we have and possibly others that would give us an added 19 dimension.
If we could extract personnel error, equipment 20 failure, design or construction as a basic cause, then you 21 could trend that particular cause with a given plant or across 22 the family of plants.
23 Staffing, we are continuing to look at.
Looking at 24 the generic issues backlog is also, we feel, very important and 25 that's through the SIMS program.
17 1
And I should indicate that this is a case where the 2
offices have a very cooperative effort in developing these
-3 indicators.
Research is supporting us with two contracts and 4
staff.
NRR is supporting us with their comments and their 5
development'of the significant events.
And so it is a very 6
cooperative effort, and very satisfying in that regard.
7 COMMISSIONER CARR:
On the maintenance items, are you 8
looking for the safety system reworks'mean time to repair, 9
i.e., how long it takes from the time it's reported until it's 10 fixed?-
11 MR. JORDAN:
I will ask Robbia to respond to that.
12 MR. SINGH:
Yes, we are looking into that.
We do not 13 have enough data yet to make any findings, but we are looking 14 at it.
15 MR. ROSENT!!AL:
We are also very interested in 16 tracking LCO hours, the time that trains of equipment are 17 declared inoperable, and that would be the item as well as the 18 duration.
19 COMMISSIONER ASSELSTINE:
That should be pretty easy 20 to do, shouldn't it?
{
21 COMMISSIONER CARR:
Well, sometimes they exit those i
22 without repair.
- l 23 COMMISSIONER ASSELSTINE
Well, yes.
24 MR. ROSENTHAL:
And it's not currently reported.
l 25 COMMISSIONE11 ASSELSTINE:
I think Ken's point is a i
!j'
i i
.~
18 1
good one, though.
What you want to do is pick up where you've 2
got one train that's out for a long period of time, or a 3
component that is ou-for a long period of time and just never 4
gets fixed.
5 MR. STELLO:
It's going to be a lot of work to do, I 6
think, because --
7 COMMISSIONER CARR:
Well, are they living with their 8
problems or are they doing something about them?
9 MR. STELLO:
A lot of plants take trains out of 10 service to do surveillance and maintenance on them, too.
That 11 isn't any repair at all, they just have to take them out of 12 service in order to do the surveillance and maintenance, so 13 it's -- when we devised a reporting system, we didn't have it 14 set up to get all of what you need.
We'll look at what we've 15 got that we can use and we might have to change some things.
16 CHAIRMAN ZECH:
But you could have -- on the items 17 out of service thing, you could have -- if you wanted to, to 18 get a little bit more sophisticated, you could give weights to 19 certain equipment.
How long has it been out.
If certain 20 equipment has been out over a period of time, that should weigh 21 heavily, as to something that just went out recently.
If it's 22 been out a long time, it's important.
I think the point is 23 well taken, and it could be factored in, I think, in 24 maintenance.
25 MR. STELLO:
A good example is a pump that you take v-.
-,,,-_..-,-__.,-,,_.-v.,--_._--,---...--,r.....
..., - -....,, - - - +, - -
19 4
1-
. apart and have all the parts on the-floor.
That's entirely 2
different than something you just got to run an instrument 3
' check on it.
4 CHAIRMAN ZECH:
Yes.
But also the pump you take out 5
and you don't have the'right parts and you don't have the right 4
6 parts and you don't get the right parts --
7 COMMISSIONER ASSELSTINE:
It sits there broken for a
~
8 year, that's right.
9 CHAIRMAN ZECH:
And, you know,.you say, well, you get 10 used to it.
And that is the kind of thing that ought to some 11 way or another jump out at us as a maintenance performance 12 indicator.
i 13 MR. SINGH:
We are looking at a number of situations.
I 14 CHAIRMAN ZECH:
Good.
Fine.
15 COMMISSIONER ASSELSTINE:
On the training on the 16 operator license exam results, are you looking only at our 17 exams, or are you looking at utility-administered requals as 18 well?
At this point we are looking at our 20 requals.
21 j
COMMISSIONER ASSELSTINE:
Our requals?
22 MR. ROSENTHAL:
Yes.
We also took a look at initial 23 exams, whether you could make a better safety argument on 24 requal exams.
We have a problem that the -- ideally you could i
25 correlate pass / fail on requal exams with how operators worked i
20 1
at the boards, and if you saw a-good positive correlation, then 2
.you know that that was a good indicator.
It isn't the case.
,3 And the reason, in part,'is that what goes on at the boards is
'4-
~ normal, relatively low stress operation.
What the requals tend 5
to do is test how well the operator behaved under transients 6
that'are anticipated to occur, and what you really, from a 7
' safety standpoint, are interested in is how will that operator 8
behave on a super high stress situation when half'.the plant is 9
out of service for some reason.
10 And so we would like to be able to reconcile and-11
. fully understand those differences before we adopt any one 12 specific training indicator.
~13 COMMISSIONER CARR:
But you are also very interested 14 in how he operates normally so he doesn't cause one of those 15 high stress situations?
16 MR. ROSENTHAL:
Right.
And that we will pick up with 17 cause codes, where for each event that happens, a simple trip i.
18 or more complicated thing, we will say was it due to personnel, 19 communications, maintenance, procedures, et cetera.
So we will pick them up.
But we are also looking for those correlations.
20 1
21 COMMISSIONER ASSELSTINE:
I take it right now in that
'l 22 area you do something, but it's really on an ad hoc basis.
I 23 mean if we go in and give a requal exam, and we see an 24 extraordinarily high failure rate, on the order of 40, 50, 60 25 percent or more failures on the requals that we give, that is a I
._. _.... _. ~. _. _.... _.., _.. _ _. _ _... _
. ~. -
21 1
pretty good sign that there's something wrong, and apart from 2
taking those guys out of service, it usually results in a 3'
- special team going down to find out what the problem is with-4 the requal training program, or with the operators themselves.
5 What you are talking about now, I gather, is a little 6
bit more of a formal-look.
7 MR. ROSENTHAL:
We think that we owe a formal look in 4
8 order to have an auditable, defensible program; sure.
And so 9
we will grind through the data.
It is interesting to note that 10 the requal exams, at least from what we've looked at so far, 11 don't corelate well with how the operators do at the boards.
I 12 Now we have to understand why.
13 COMMISSIONER ASSELSTINE:
That's why.
But it's more 14 a function then really of how well their training programs 15 prepare them to deal with transient and complex accident 16 situations.
That's not the kind of thing they routinely face 17 on a day-in and day-out basis on the boards.
18 MR. JORDAN:
I think I would like to summarize the 19 development by saying that the Staff is looking at a very large 20 variety of possible indicators, and we are also looking at the 21 large variety of utility indicators that are in present use.
I 22 Each utility has unique programs that they've developed to help t
23 begin a goal-setting process as well as looking at their 24 management performance.
So we are examining those, and the 25 results of them in cooperation with the regional offices to try i
4
C 22 1
to pull out of that the small number of indicators we'd like to 2
add to the existing set that would be meaningful in a safety 3
sense.
-4 So it is a distillation process.
5 COMMISSIONER CARR:
In that licensing area, can you 6
-track operator -- licensed operator turnover?
7 3GR. JORDAN:
We talked about it and looked at it 8
during our trial program last year, and we didn't find a 9
correlation in that, at that time.
There is a logical 10 correlation in one's mind, but we didn't find one that we could 11 make with respect to any operator plant performance.
12 (Commissioner Roberts left the room at 2:24 p.m.)
13 COMMISSIONER BERNTHAL:
I think the question was can 14 you track that?
15 MR. JORDAN:
Yes, we can track it, yes.
16 COMMISSIONER BERNTHAL:
And you are tracking it?
l 17 MR. JORDAN:
We have looked at it, but we didn't find 1
- 18 a correlation.
We looked at it for this particular purpose.
19 The data is in hand, certainly.
20 COMMISSIONER BERNTHAL:
I don't think that we ought l
21 to be throwing out data that in a qualitative sense, judgmental i
22 sense, seems like it would be relevant.
And a perfect example 23 l
24 COMMISSIONER CARR:
It's bound to be relevant.
Not i
25 much of it occurs, but --
9 3
_ -...,.., _. - _ _ - _ -. - - _ -. _ _. _. ~.,
=.
23
'l-COMMISSIONER BERNTHAL:
Whether or not it correlates 2
with something:shouldn't really matter, because what we are 3
finding is an awful lot of these performance-indicators-_ don't 4
necessarily correlate.
5 COMMISSIONER CARR:
It's probably got a small data
-6 base, I assume.
7 MR. JORDAN:
Yes.
We are trying to find a set or 8
sets of indicators that would have a reasonable correlation, 9
and not be just a scatter pattern.
Because you can overload 10 with information as well.
4
.11 (Commissioner Roberts entered the room at 2:27 p.m.)
12 COMMISSIONER BERNTHAL:
That has a dangerous sound to 13 it.
We shouldn't be looking for indicators that correlate, and 14 the ones that don't, we don't use.
If they seem to be j
15 reasonable things to be using, that's sort of -- as you very 16 well know, we could and up laundering the data to make sure he 17 got things correlated.
If they don't correlate, they don't t
18 correlate, but they may still be of interest to our regulatory 19 mission.
20 MR. JORDAN:
And I think the difference is whether 21 they are selectad as a specific tool that the management uses in review of plant performance against the regions using and 22 1
23 NRR using in individual plant reviews, because they do that.
24 CHAIRMAN ZECH:
Before you go off this slide, I'd 25 just like to make a comment on the management indicator.
I i
-,- ~ ~.,..
24 a
-1
. recognize'that is probably the most difficult one you have up 4
2 there to make any real objective or subjective analysis of.
3' But there are other things you can kind of get a little bit
.4 more specific when you -- I see you are trying to get generic 4
5 issues backlog, that's fine.
But there are some specific other-6
. things, and the more specifics you can get, the more 7
quantifiable you can get things done, the easier it is to make-8 that final judgment, in my view.
But some things like outage 9
planning, for example.
Now I submit that's not the easiest 10 thing at all to make a judgment on, but some plants do a lot of 11 outage planning, they do it quite well, they do it in a short 12 timeframe, and it's extremely well done.
Others seem to take a 13 little longer and they don't perhaps do it as well.
14 Now that's something I think you might want to at 15 least look at for the long haul.
I recognize you can't make 15 any short range thing on that one, but also how they coordinate 17 between the divisions or between departments.
Now we know how 18 hard that is, but also we know that some of the plants we have i
19 followed seem to not do that coordination as well as others, 20 and they are the ones who have the problems.
21 So this is the kind of thing that's very hard to look l
22 at, very hard to quantify, very subjective, very subject to 23 criticism and debate and all that, but it's valuable if we can i
24 do it right.
25 It takes time to put this kind of an indicator i
J 25 1
together, but it's so valuable, it's worth doing, in my 2
- judgment.
But if you can think of as many -- I've just given 3
you a few examples, but if you think of other management 4
specifics that perhaps can help you to quantify, as best you 5
can, some of the management things that are quantifiable, at 6
least to a degree, you may be able to come down with an 7
assessment of how well the management group is doing as a 8
whole.
9 So I submit that is something you ought to keep in 10 mind, but don't try to develop too quickly.
It's not going to 11 be very good at first, but it could be very valuable over a 12 period of time, and the more you're thinking about this, you 13 might be able to think of other more quantifiable management 14 issues and tools and objectives that could be useful.
Because 15 the whole subject of management and management coordination, 16 management as it relates to safety is, in my judgment, very 17 important'to the operational safety of these plants.
18 Management involvement is the key.
How do you 19 measure that?
Very difficult.
But if you think about a lot, 20 perhaps there are some things you can -- say, weekly meetings, 21 how many weekly meetings, who attends them.
I don't know.
But 22 if you think about it carefully, perhaps we can come up with 23 some kind of -- and it may not even be a finite thing, but it 24 might be a showing, certain plants or certain groups of plants, 25 that really seem to have a very strong management emphasis, and
26 1
they perhaps do reasonably well; others that don't, and perhaps 2
we can relate'it.
But it's such a valuable tool, I submit, I 3
would hope you could give it some continuing attention.
And 4
you're not going to develop that one overnight, but it's worth 5
the effort.
6 COMMISSIONER BERNTHAL:
Lando, let me just comment.
7 As you know, we haven't entirely agreed on this issue, and your 8
comments here just feed the concern that I had when we started 9
down this path.
10 I cannot, sitting here today, and I'm willing to look 11 a month or six months from here, even imagine a way that would 12 be remotely fair whereby one can quantify these human factors 13 based elements as they relate to management, as they are part 14 of management, and get the kind of numerical quantitative 15 analysis that we are really talking about as performance 16 indicators.
17 Every single one of those I can think of -- surely we 18 don't mean numbers of meetings per week.
How easy it is to 19 trump up an indicator in that category, or how you cross-20 communicate, all of those things, I couldn't agree with you, 21 are terribly important, but I was very concerned from the 22 beginning that this program not burgeon into a Trojan horse for 23 general operating criteria, and boy, I've got to say the 24 concerns are reappearing.
25 I always felt that this should be probably a lower
9 27 1
profile program where we do our job the way we ought to be 2
doing it and quantifying as many of these things as we can to 3
make use of the data that we have access to.
If we start 4.
_ struggling to quantify things that relate to human factors, for 5
example, I have really got to worry about the kind of data and
~
6 the quality of data we are going to get.
J 7
I know of one plant, the Clinton Plant, which I think 8
I have mentioned a time or two, where Admiral Hall was doing 9
what I thought was a really very imaginative thing, to try and 10 quantify his own management operation in a kind of performance
~
11 based quantitative performance indicator program.
I don't know 12 how well that has turned out, but it was surely extremely 13 unique to that organization, and the kinds of things that he
(
14 picked to try and do, the manager of the plant can try and do.
15 But I sure can't see the NRC trying to do anything that t
16 approaches an evenhanded, objective way.
17 On management, I just really worry that we'd be 18
-getting into something that doesn't fit.
It's hard to quantify 19 human factors.
20 COMMISSIONER ROBERTS:
I share your concern.
21 COMMISSIONER ASSELSTINE:
Fred, it does seem to me 22 that although it's difficult to come up with indicators of 1
23 management performance, there may be a few that are worth 24 looking at, and that may even be quantifiable.
And I guess I 25 would look at that list up there and say there are a couple
~
28 1
under staffing that to.me are very good indicators of 2
management performance.
t 3
For example, vacancies.
That --
4
. COMMISSIONER BERNTHAL:
That's fine.
That's fine.
5 COMMISSIONER ASSELSTINE:
Well, if you have got a 6
plant where.the operations manager, the operations 7
superintendent, the maintenance superintendent, the I&C 8
superintendent, the rad chem superintendent, that position is 9
vacant for several months --
10 COMMISSIONER BERNTHAL:
Straw man, Jim.
I mean I 11 concede the point.
j 12 COMMISSIONER ASSELSTINE:
But we've had plants where 13 that's been a problem, and tracking that kind of an indicator might have tipped us off earlier on that things weren't working 14 15 right.
16 overtime, the same thing.
If you have got operators 17 working 80, 90 hours0.00104 days <br />0.025 hours <br />1.488095e-4 weeks <br />3.4245e-5 months <br /> a week, you know you've got a problem, and 18 you know it's a management problem, because management isn't 19 sufficiently involved to'make sure they have enough operators 20 to run the plant safely, and those are two examples where if we 21 had had indicators, we might have learned about some of the 22 weaknesses at Davis Besse or the weaknesses at Pilgrim even 23 earlier than we did, and perhaps under somewhat happier 24 circumstances.
25 COMMISSIONER BERNTHAL:
But those are not human 4
.i
_..,-- ~
29
.1
-factors related elements.
Those are simple hard numbers, and I
'2 think one can mention turnover rate.
Of course, turnover rate, if it's very high, obviously indicates that there is something 3
4 you don't like going on here.
5 COMMISSIONER ASSELSTINE:
But that's a quantifiable 6
7 COMMISSIONER BERNTHAL:
But there aren't many.
8 COMMISSIONER ASSELSTINE:
Well, there may not be 9
many, but those are at least some that strike me as really 10 focusing more on management performance and management 11 involvement, and I guess my sense was those are the kinds of 12 things, some of them at least, that Lando had in mind.
13 7-CHAIRMAN ZECH:
Those are exactly the things I have 14 in mind.
You cannot quantify everything, and you have to be 15 very careful because you are going to eventually have to make a i
16 judgment, it's going to be a subjective judgment.
17 On the other hand, if you have enough you can i
18 quantify overtime, you can quantify the vacancies, you can even 19 quantify, in my judgment, outage planning, because some plants 20 do the same work in a shorter period of time and do it better 21 than others.
l 22 But there are other things you can quantify, and you 23 have to be careful about it.
That's what I tried to emphasize, 24 but I think it is worth the effort, because of the importance, 25 overall importance, of management to the whole scheme of i
i i
30 6
1.
safety.-
2 COMMISSIONER BERNTHAL:
Well,-we certainly all agree
. on that, and I agree there are a few areas, and I would trust 4
we already have data available for turnover rate, vacancies are 5
important.
We surely can get data, surely have data for that L
6 sort of thing, but --
7 COMMISSIONER ASSELSTINE:
Overtime.
Yes, that's 8,
right.
9 COMMISSIONER BERNTHAL:
-- but all I can say is we T
t 10 ought to be very careful about these things that are not 11 quantifiable, and that --
12 MR. STELLO:
Let me say, let's remember that we have 13 a lot of other programs other than the performance indicator..
14 A lot of the discussion that has just gone on is an area for 15 which there is very heavy emphasis, in the SALP program.
Those i
16 are the areas where we identify all the issues commissioner 17 Asselstine was talking about.-
18 If we do see key positions that have been vacant for 19 a long time, it is an indicator.
If we do seu a lot of -- so 20 we have got to be very careful about trying to cause a 21 performance indicator from reaching out and replacing an awful 22 lot of -- where those judgments have to be made, and there are 23 difficult judgments.
But again, also recognizing that if we 24 can devise any particular kind of parameter which causes us to 25 ask the questions, say, look, we're interested, but we've got
1
'g g
31 1
to make sure that we don't let it grow to where we become too
'2 burdened-with.the data.
And it's a judgment.
That's what 3
they're doing, and we are exploring all these areas, and 4
together.over a period of time I am convinced that we are going 5
-to do better.
But it takes time.
6 COMMISSIONER BERNTHAL:
How many new indicators are 7
you projecting over the next year or two?
8 MR. JORDAN:
Not really a projection.
We are going-9 to bring you a package in December that we would have, you 10 know, a strong support for, for any new indicators.
11 MR. STELLO: 'He's looking for just an estimate.
Are 12 we talking a half dozen more?
13 MR. JORDAN:
I would say of the order of three or 14 four would be the kind of numbers that we would be looking at.
15 There may be some compounding through cause codes that gives 16.
you, you know, another dimension, so that for each of the 17 indicators we have,'if there were a cause code attached for 18 each of the events that got tallied, then that would be a 19 substantial larger dimension to add to it.
20 MR. STELLO:
An order of a half a dozen between now 21 and December.
The longer term, though, is the chairman's point 22 on the organization management effectiveness.
That's precisely 23 the point we're driving at.
That is a long term goal., to try 24 to be able to do it, and I think it is a combination of --
25 CHAIRMAN ZECH:
It is a longer term, but it's so
r.
32 1
important that we appreciate your putting it down, focusing on 2'
the fact that it is a long term, because it is, but it's a 3
valuable, very valuable tool, if we can develop it.
4 MR. STELLO:
In fact,-if you really could do that 5
well, you could almost let the rest of it go, if we could get 6
it.
f 7
MR. JORDAN:
Well, there's the inverse, that the sum 8
of what we have now really is a measure of management 9
effectiveness, but it's a lagging indicator, rather than a 10 leading indicator, and so that's the idea in the longer term, 11 that we would like to focus on more predictive, get further 12 ahead of this thing.
13 CHAIRMAN ZECH:
Okay.
14 COMMISSIONER BERNTHAL:
Let me ask an underlying 15 question again.
I probably asked it before.
Why wouldn't all i
16 of these things, and perhaps many more, simply be something 17 that would provide supporting data for -- I'm almost afraid to l
18 utter the words -- for the SALP analyses and evaluation of the l
19 SALP program?
I mean why does one set this off as a separate i
20 element in what is recognized and understood as our general 21 evaluation process for the overall performance of the nuclear I
22 power plant?
Why wouldn't you just use these things as 23 underlying supporting data to assist you in reaching judgments 24 on SALP?
25 MR. STELLO:
You do.
33 1
COMMISSIONER CARR:
I thought tha problem was the 1
2 SALPs didn't give us enough heads-up, or at least we didn't 3
take. action on what we got.
4 COMMISSIONER BERNTHAL:
The SALPs are whatever you 5
make of them.
I mean they are --
6 MR. STELLO:
The amount of resources that go into.the 7
SALP program are already so great and some of them are between 8
a year and 18 months before you can get them down, and trying 9
to snrink those into doing them any faster is an enormous 10 increase in resources, where here the performance indicator can 11 give you a pulse to ask questions at a frequency --
12 CHAIRMAN ZECH:
And it's information we generally 13 have.
All we have to do is tap the system to get it.
14 COMMISSIONER ASSELSTINE:
I agree.
15 MR. STELLO:
But as part of that SALP -- but whatever 16 you do with SALP, you need to keep in mind that invariably you 17 are looking at history which is sometimes two or more years 18 old.
And that's too long to wait.
You really want to be 19 asking questions and catching things at a frequency well in I
(
20 advance of that timeframe.
I 21 COMMISSIONER CARR Well, at best, these should 22 highlight a plant you ought to visit.
l 23 MR. STELLO:
Yes.
j i
24 COMMISSIONER ASSELSTINE:
That's right.
25 COMMISSIONER BERNTHAL:
Well, I would have hoped I
l
---,,,,.---...._-,,--,,-e-,..n..,
.,.,,..r..,en,-.-..,,e.
,a-,,,----,n,-..-.,_,,,,,,,---.-_,,,,._.,.--_..--,n-,,-.-mme--.
34 1
these. wore things that we were monitoring all along, but we can 2
certainly monitor them now.
3 COMMISSIONER ASSELSTINE:
I suspect some of them, 4
they were, it was just more on an ad hoc basis.
5 CHAIRMAN ZECH:
But not on a formal basis.
6 COMMISSIONER ASSELSTINE:
If you had a plant that was 7
having 15 trisp in a quarter, that ought to raise a flag in the 8
regional administrator's mind that he better get somebody out 9
there and find out what the heck is going on.
10 CHAIRMAN ZECH:
All we are doing is disciplining the 11 system, getting more formal program, and using the data we 12 already have.
Essentially that's it.
13 COMMISSIONER CARR:
We were monitoring them in i
14 retrospect after some significant event.
l 15 CHAIRMAN ZECH:
That's right.
And we are trying to i
16 get ahead of the problem.
If you recall, the whole purpose of 17 this whole business, at least in my mind, is if -- and I know 18 it's very difficult, but if we have enough data here from this 19 to help us to possibly predict plants that are showing downward 20 trends and heading towards trouble, we are trying to prevent 21 the problem, rather than just wait and say, well, what 1
22 happened, and let's go back and see.
We are trying to use that 23 hindsight and project it forward.
It's very difficult, but I 24 think the effort is worth it.
If we can only be successful in 25 several cases, I think it will be well worth it.
35 1
MR. STELLO:
What we need is the time to develop the 2
system and learn how to.use it without it being misused, in the 3 ~
meantime.
That is a bigger concern.
4 CHAIRMAN ZECH:
It is.
It's important, though.
5 MR. STELLO:
One other aspect of this that has not
-6 come out, performance indicators, while you are looking for them to alert you of problems, we also want to be looking at 7
8 where you have a plant that's performing really well.
Also, as 9
the Commission pointed out the last time, then we're going to 10 understand why, and see if we could glean from those --
11 CHAIRMAN ZECH:
Learn the lessons.
12 MR. STELLO:
Get the lessons, and --
13 CHAIRMAN ZECH:
Exactly.
. (.
14 MR. STELLO:
-- then we could suggest to INPO some
'15 additional things that they could do by striving for that kind 16 of excellence throughout tho industry.
It's both sides that t
17 we're trying to --
i 18 COMMISSIONER ASSELSTINE:
That's absolutely right.
j i
19 MR. STELLO:
We haven't spent a lot of time on that i
20 latter point, but I think it is an important one.
21 COMMISSIONER ASSELSTINE:
In fact, I think that's l
l 22 probably the area where you might have the biggest gains on 23 management effectiveness.
If you can understand what makes a 24 place like Farley good, why they're so effective at outage i
25 management, why they have achieved the level of success that
36 1
they have, and understanding what it is about the management 2
system that operates so effectively, get that lesson and pass 3
it on to the others, then we would get a real benefit.
4 COMMISSIONER CARR:
We already know that, it's 5
people.
6 COMMISSIONER ASSELSTINE:
Well, that's right.
7 COMMISSIONER BERNTHAL:
That's exactly what I was a
going to say.
Go down and meet the people, and you'll have 9
your* answer.
10 MR. STELLO:
If we could devise a pill for that, then 11 we'd make a lot of money.
12 MR. JORDAN:
If I could have the next slide.
I won't 13 g-dwell on this, but since we met with you last, we did have 14 workshops with licensees in each of the regional areas in 15 conjunction with the incident investigation workshops, and 16 those were interactive with the utilities, and quite positive.
17 We got good responses from utilities in terms of their 18 understanding and acceptance of this type of a program.
19 With regards to INPO, we have had continuing discussions meeting with them..We are planning a meeting late 20 21 this month to go over our common development goals, and to 22 finalize then the method for transfer of data.
23 Next slide, please.
24 The objectives of this coordination plan that we have 25 with INPO is to clarify the purposes of our respective roles in
r 37
/.
1 using performance indicators to once again recognize the 2
limitations, and as an aside, during one of the meetings with 3
the utilities in the regional office area, I made it a point of setting my watch so that every five minutes during the speech, 4
5 I would alert them that performance indicators were only a tool 6
and it was amusing, but they liked the idea that we were so 7
sensitive to that consideration, and I got several different 8.
remarks afterwards.
9 So the limitation is quite important to provide this 10 framework for our coordination, so we don't duplicate efforts 11 either in collection or development.
12 The benefits of the plan:
INPO is expected to 13 provide us data on those common indicators, and we will simply 14-spot-check that data to assure that their process and ours 15 gives approximately the same answers.
Certainly there are 16 differences.
We have done some checking already with them, and 17 numerically there are small number differences that we can 18 resolve quite easily.
19 The reports that we issue will identify those four 20 indicators as being from INPO-provided data, and then those 21 packages will be provided to the PDR.
We are quite satisfied 22 with that combination, and we feel that this program would then 23 improve our effectiveness by being able to communicate more 24 directly with industry.
25 That's the end of the presentation.
.6.-
1 MR. STELLO:
One last simple point that I wanted to 2
bring up.
There was a concern that as you start performance 3
indicator programs, the utilities would tend to restrict their 4
reporting and find a way to make their performance indicators 5
look more favorable.
I am no longer at all concerned that we 6
need to worry about that at all, because there's so much else 7
that we rely on to make those judgments, that if that does go 8
on, it's not going to mislead us.
I'm convinced it is not a 9
problem at all.
10 I am, however, in agreement with some of the points I 11 keep hearing from INPO, as undue emphasis on something like i
12 scrams, automatic or manual, no matter what, can have an 13 untoward effect on safety, too..
So.there's this need to be
(
14 balanced about how to use them and question people on what they 15 can and can't do.
16 I am convinced it's a program worth continuing, and I 17 strongly urge the Commission to continue supporting it.
18 CHAIRMAN ZECH:
All right.
Thank you very much.
19 Questions from my fellow Commissioners?
l l
20 Mr. Roberts?
21 Mr. Asselstina?
22 COMMISSIONER ASSELSTINE:
Just a couple on the 23 coordination plan, just to make sure that that is consistent 24 with what we have been talking about today.
25 Vic, you mentioned scrams.
Scrams and forced outage l
39
. rates are. highlighted in this as being something that could 2
lead to nonconservative safety decisions.
Those statements are 3
kind of strong.
Could you give me a couple of examples of how 4-those things are particular problems?
5 MR. STELLO:
Well, I think it is self-evident.
If 6
you have a performance indicator which makes you look poor in 7~
performance by having --
8 COMMISSIONER ASSELSTINE:
9 MR. STELLO:
-- a lot of automatic scrams come out, 10 then you are going to try to find a way around the problem, not 11 try to avoid the scrams when you ought not to avoid them.
f 12 I think anything that you try to do to manipulate 13 those data on safety equipment per se would have an untoward
- 14 effect on safety.
If its purpose is to manipulate either the 15' plant or the equipment to look more favorable.
That by itself 16 is a concern.
It has to be.
l 17 COMMISSIONER ASSELSTINE:
How could you do that other 18 than manually scramming the plant?
)
l 19 COMMISSIONER BERNTHAL:
That's exactly what may be 20 happening.
21 COMMISSIONER ASSELSTINE:
Is that it?
Is that the 22 concern, that the operators will manually scram the plant in 23 order to avoid taking an automatic scram?
24 MR. STELLO:
That's one way.
25 COMMISSIONER ASSELSTINE:
But if the plant was going
40 o
1 to scram' automatically, anyway --
i 2
MR. STELLO:
Well, it may not.
3 COMMISSIONER ASSELSTINE:
You're not~ talking about 4
changing setpoints and things like that?
5 MR. STELEC:
They do that, too.
-6 COMMISSIONER BERNTHAL:
They do that, too, that's 7
right.
8 COMMISSIONER ASSELSTINE:
Within tech specs?
9 MR. STELLO:
Sure.
Change the tech. specs.
10 COMMISSIONER ASSELSTINE:
Okay.
But then you get a 11 safety review by us before you can --
12 MR. STELLO:
It doesn't make any difference.
Its 13 whole purpose is being to -- you know, to correct a problem 14.
through a way other than the right way, that is the intent.
15 And whether you are introducing a' manual scram because you 16 avoid it, if the plant were working correctly, you have made a 17 bad call, you may have had a scram you didn't need.
18 COMMISSIONER BERNTHAL:
Jim, if I may interject here, 19 it is interesting that the average for manual scrams since we 20 initiated this program -- I guess I'm talking about 1987 so far 21
-- is 50 percent greater than it was in the preceding two years 22 on average, and meanwhile automatic scrams have dropped.
23 COMMISSIONER ASSELSTINE:
Is that just since we've
-24 instituted our program, or since the industry has started 25 tracking manual scrams, or is that the EOD guys?
41 1
COMMISSIONER BERNTHAL:
1987 versua 1985 and
'6, 50 2
percent increase in manual scrams.
That's my understanding.
3 2GR. - STELLO:
Does anyone have the number?
4 MR. JORDAN:
I don't have the statistic.
We will 5
look at it and provide it.
6 COMMISSIONER ASSELSTINE:
It would be interesting to 7
know what the Staff thinks.
8 MR. STELLO:
But it is a concern that is shared by 9
both the Staff and INPO, that you start to manipulate things
.10 just for the purpose of making performance indicators look 11
'different, and it's not the correct way to do it.
12 COMMISSIONER ASSELSTINE:
Yes.
13 MR. STELIO:
The right process is improving
,f i
14 performance, overall, and making those kinds of changes 15 necessary to do it.
16 COMMISSIONER BERNTHAL:
Let me -- I'm sorry.
On the 17 same subject, if I may, I also worry about radiation exposures.
18 Suppose that radiation exposure looks a bit high, you aren't 19 looking so good in the performance indicator, and you start 20 keeping your people out of high radiation areas where 21 maintenance needs to be done.
That's one way to get your 22 performance indicator to look better.
I would hope that you 23 don't do that.
24 COMMISSIONER ASSELSTINE:
Presumably, though, you may 25 a consequent penalty, though, on other performance indicators
7 42
~1 2
COMMISSIONER CARR:
It's pay me now or pay me later.
3_
COMMISSIONER ASSELSTINE:
That's right.
If you have 4
got the right set of indicators, you can't get away with that 5
very long.
6 COMMISSIONER BERNTHAL:
Well, let me point out that 7
if you caught that valve in your not of --
~
8-COMMISSIONER ASSELSTINE:
No, forced outage rate 9
catches you.
10 COMMISSIONER BERNTHAL:
Maybe; maybe not.
Maybe not.
11 You're playing the odds.
12 MR.'ROSENTHAL:
When you look at the -- what one 13 would perceive as some of our best performers, and you look.at 14 the PIs, they are above average in every category --
15 COMMISSIONER ASSELSTINE:
Every category, right.
16 MR. ROSENTHAL:
-- and improving.
17 COMMISSIONER ASSELSTINE:
That's right.
18 MR. ROSENTHAL:
And then you look at some of the 19 plants that you might be concerned over, and you see that they 20 are less than average and declining in several, and then you H21 ask yourself some questions about why, and you march on from 22 there.
23 We have tried to display the data in such a means 24 that one could give that overall picture, and it's that overall 25 picture rather than any one indicater that we key on.
43 1
COMMISSIONER ASSELSTINE:
And you are seeing that 2
kind of correlation in plants that we have typically tended to 3
look at and the industry tends to look at as truly outstanding 4
performers do good across the'coard, and the ones that we are 5
identifying or have focused on as problems, fall in the bottom 6
category?-
7 MR. ROSENTHAL:
Generally.
8 COMMISSIONER CARR:-
That's the way you pick.the
'9 correlating indicators.
10-(Laughter.]
11 MR. STELLO:
All generalizations, including this one, 12 are -- you get -- looking at performers by indicators are not 7 -
13 necessarily good performers.
14 On the other hand, you get some performance 15 indicators in some plants'looking poor, and it's just the way 16 they report things and they do things, and they look really 17 good, so you've got to be careful.
18 MR. JORDAN:
My favorite comparison is looking at i
19 identical units on the same site, and seeing the differences 20 between those, because you really have to look behind why are 21 they different, and there are some differences clearly in the 22 data.
23 MR. STELLO:
And you see both.
You see plants at the 24 same site sitting next to each other, both of them look good.
25 Another example, one looks very good and one doesn't.
And you
44 1
really -- you have got to be very careful.
I think that's the 2
bottom line.
3 COMMISSIONER BERNTHAL:
Oh, I think it's important 4
that we get.the latest set of data here, I hope before Jim 5
leaves, but we certainly should get it here soon.
Who knows, 6
he might be able to use it when he leaves.
7 COMMISSIONER ASSELSTINE:
I certainly would be 8-interested in looking at it now in light of the results from last year and sort of track what's happened, what's changed.
9 10 COMMISSIONER BERNTHAL:
And do more than just have 11
.the numbers tossed down in front of us.
I would like -- this-12 is going to be qualitative now, but I would like to see some 13 attempt to do it however you like, poll the Staff on good and
-14 bad -- well, you've got a list of average, above average, below 15 avera@e performers, and without fiddling the data, just see on 16 an unbiased sample of some' kind how well things do correlate.
17 That would be interesting to know.
18 MR. STELLO:
We will provide both those to the 19 Commission..
20 CHAIRMAN ZECH:
Anything else, Mr. Asselstina?
21 COMMISSIONER ASSELSTINE:
Two quick questions.
There 22 are two paragraphs of this B-4, the first bullet being one, and 23 I guess it's C-4 being the second, that talk about the NRC use 24 of the indicators as a tool.
I guess what I'm wondering in 25 light of the discussion we had today, does the Staff view this
45 1
as in any~way too restrictive?
2 For exampla, the first one says NRC uses the indicators as one tool for assessing and trending plant safety 3
4 performance as an input to' adjust NRC programs.
[
5 I take it you are reading that broad enough to say 6
it's not ju t our programs, but where we focus attention on, s
7 whether we go out -- we use this as a tool to inquire further?
8 Is that --
9 MR. STELLO:
That's what these words are intended to 10 say.
11 COMMISSIONER ASSELSTINE:
Okay.
12 MR. STELLO:
And we both do that.
We both do that.
13 COMMISSIONER ASSELSTINE:
The other one had to do 14 with -- I understand not writing deviations or violations based 15 upon performance indicators or SALP findings based specifically 16 on performance indicators, but I take it that is broad enough 17 to include or use these again as a tool in pursuing items that 18 may get reflected in SALPs; is that -- that's consistent with 4
19 what you are thinking as well, and what we have been talking
- 20 about?
21 (Commissioner Roberts left the room at 3:00 p.m.]
22 MR. STELLO:
Yes.
Yes.
23 COMMISSIONER ASSELSTINE:
Okay.
24 MR. STELLO:
It's avoiding that undue emphasis that 25 we were talking about just a moment ago.
P g
y s.
,-,.-._-,--.--..-.7
,,.w,--
.e.
,,.-,..-,---,,,,e-..,
- -- -,~,.,-
.-.-._,--,--...m e ~
-,v, e.,,,-
46 w
1 COMMISSIONER ASSELSTINE:
Good.
The only other 2
question I'had was on C-3, where it says NRC and INPO 3
interactions with utilities concerning performance-indicators 4-will be conducted at the senior management level to avoid 5
perceived or real pressure at the working level.
6 The only concern I have is if you saw an indicator 7
that indicated a plant has had 15 or 17 trips over the past 8
couple of months, that pops out of the indicator package, you 9
aren't going to wait for some senior level discussion, are you?
10-I mean you are going to send somebody out there to find out 11 what's going on, and that's going to be at the working level, 12 first and foremost and pretty quick.
' 13 I don't want this to be so restrictive that it's 14 going to limit our ability to follow up on these things when ta.
15
'see something that deserves attention.
16 MR. STELLO:
Well, be careful, now.
You don't want 17' every inspector having these indicators and then deciding that 18 he's going to go and do all these things based on what he sees.
19 You want the management to look at it and make the decision you 20 just said needed to be made.
Yes.
21 COMMISSIONER ASSELSTINE:
Okay.
Good.
But I want a regional administrator to be able to say I have got a question 22 23 about what's going on out there --
24 MR. STELLO:
Well, I consider --
25 COMMISSIONER ASSELSTINE:
-- you go find out what's
47 1
happening.'
2 MR. STELLO:
I consider a regional administrator to
~3 he certainly a senior. management level.
4 COMMISSIONER ASSELSTINE:.But he doesn't have to say, 5
well, I've got to go discuss this at the vice president level, 6
at least --
7 MR. STELLO:
No.
No, no, no, no.
8 COMMISSIONER ASSELSTINE:
All right.
9 MR. STELLO:
It was to avoid having everyone on the 10 Staff go out and -- because=I could see everyone in their 11 specialty deciding this is the most important thing in the 12 world.
13 COMMISSIONER ASSELSTINE:
That's fine.
But I do want 14 him to be able to inquire further fairly quickly and easily.
15 Good.
Those are ths only questions I had.
16 CHAIRMAN ZECH:
Mr. Bernthal?
17 COMMISSIONER BERNTHAL:
I think I've covered most of 18 the things that I was concerned about.
I am going to make one I
19
- further comment.
You know, I am cast in an unaccustomed role 20 here, because I'm a physicist.
Lord knows, I believe in 21 numbers and believe in proper analysis of numbers.
It's just 22 that I'm skeptical about bad numbers, and that's all that I'm
- 23 trying to make sure that we don't end up with here.
24 I have one other broad policy concern that I think 25 I'd better just state very succinctly.
The more I see of this s
v-
. m-rs
,__--,-we_,.,
--,-r----v,.,.,- - -, - -
,-,.--.-.,y--.----,,,-,.m
48
.1 business of trying to " coordinate with INPO," and to reach 2
various understandings and memoranda of understandings with 3
.INPO, the more convinced I am that we really ought to both just 4
go our own way and do our own job in this area.
5 It is not, as I understand it, even though it may be 6
duplicative of effort, it is not going to require that much 7
additional by way of agency resources for us to do those 8
overlapping performance indicators.
If we are going to do 9
those things at all, it seems to me we ought to do them 10 ourselves.
11 I just don't like the appearance of getting into a 12 situation where we are having to rely on INPO or compare with 13 INPO,'and I am not convinced that it is in the best interest of 14 the public or of INPO, and the long term integrity of INPO's 15 programs for us to do that.
16 INPO has a job to do; we have a job to do.
And I l
17 would strongly urge that we reconsider th'is coordination 18
. effort.
I think we are going to and up in trouble with that.
19 I would much prefer to see us do our thing, let INPO do its l
l 20 job.
I'm sure INPO will comment if they disagree with some set l
21 of indicators that we might have.
I suspect we will comment if 22 INPO should choose to make public statements about their 23 indicators, and that's probably the way it ought to be.
There is supposed to be that differing point of view and differing 24 25 responsibilities.
49 1
That is the only remaining concern I have here, and I 2
look forward to receiving more information and more data as 3
they come out.
4 COMMISSIONER ASSELSTINE:
In the coordination aspect, 5
is it more than just avoiding asking the utilities for the same 6
data twice?
7 My sense was that's basically what we were doing 8
here.
This MOU sort of provides at least some statements of 9
what INPO uses this stuff for and what statements we want to 10 use it for.
But my impression was this was kind of an effort 11 to avoid asking the utilities for the same information twice, 12 and then getting into an arguing match about if there were any 13 differences in the number, whose number was right and whose was 14 wrong.
Am I wrong about that, or --
15 MR. STELLO:
Some of that.
Well, clearly the 16 performance indicator on radiation monitoring is one that we 17 get it annually and they get it more often, so we'd have to get 18 into that.
I respect Commissioner Bernthal's point, and it's a 19 valid one, but I think both we and INPO are still scratching 20 the surface, and I think as you learn, I think we -- because 21 there's pitfalls in doing it, I see it as a program that's 22 worth trying to learn together.
(
23 They have a different view in mind of what they are 24 doing than we do, but I don't see that the both of them can't l
25 go together, because I think many of the pitfalls are in fact l
l
-y--,p w
v-n
-m
50 1
the same.
And they have seen them.
They have seen plants that 2
when they had certain performance indicators being developed' 3
for maintenance, just redefining on how, you know, you write 4
out a ticket for an item that needed maintenance or repair.
5 Instead of one ticket for each item, you'd just have one ticket 6
with 30 subparts and count it as one item, and your performance 7
indicator now looks 30 times better.
8 So I think the need to proceed is not one that is 9
inconsistent with both of our objectives, but if the Commission 10 desires, we certainly can go forward and do this alone.
11 COMMISSIONER BERNTHAL:
It just seems like we are 12 struggling with it.
We are struggling over the question of 13 proprietary data.
I'm sitting here --
14 COMMISSIONER ASSELSTINE:
We've solved that, I think.
15 COMMISSIONER BERNTHAL:
I think we have solved that.
16 It wasn't proprietary.
I've got to ask myself if the utilities 17 realize that there is a communication coordination between the i
18 NRC and INPO.
Does that weaken INPO's ability to deal in the 19 most candid possible way with its utilities or not?
Are we 20 unable to do our job without INPO's assistance in some of these 21 areas?
I think not.
22 I'm just concerned.
When things get this hard and 23 this complicated, I usually think twice about going ahead and L
24' doing --
25 MR. JORDAN:
Commissioner Bernthal, I think the
51
.e 1_
greatest benefit thus far in the coordination is that we have aligned our definitions so that they are consistent where they 2
were quite similar, so that the utilities won't be confused by 3
4 somewhat different definitions, where we cut scrams --
5 COMMISSIONER BERNTHAL:
Well, that's fine.
That 6
could be achieved across a conference table.
7 MR. JORDAN:
And that was done early.
Now that we 8
have common definitions, then the numerical differences are not 9
as'important, but it seems simpler to have identical data, or 10 to confirm data across the two organizations.
But as far as 11 being constrained from further development, we will go our own 12 ways and coordinate periodically on those development 13 activities.
(
l 14 MR. S'TELLO :
And they will do likewise.
But it's a l
l 15 Commission decision.
We can go either way.
16 COMMISSIONER BERNTHAL:
Well, I guess for my part, 17 you have heard my view.
I think we ought to decide what our 18 program is going to be and we ought to do it, and I think 19 they'll be a stronger organization in the long run for deciding 20 what their program is and taking care of their business.
21 CHAIRMAN ZECH:
Yes, please.
22 MR. PARLER:
Mr. Chairman, I have two brief questions 23 that I would like to state and have somebody answer in order to 24 assist me in carrying out my responsibilities to the agency.
25 The first question is, is the purpose of this NRC-
52
+
1 INPO coordination plan, which is attached to SECY 87-117, only 2
for the purpose of the exchange of information and. data 3
generally. relating to the safety of nuclear power plants?
4 MR. STELLO:
I probably want to consult my lawyer to 5-make sure I understand all of that.
6
[ Laughter.]
7 MR. STELLO:
But I think if I understand them, the 8
answer is yes.
f 9
MR. PARLER:
It's very plain, simple English words.
10-Is'it for data and information, or is it for -- or maybe the 11 second question will help.
12 Does this coordination plan provide in any way for r-13 obtaining INPO's advice and recommendations on regulatory 14 issues or policies which are within the scope of the NRC's 15 responsibilities?
16 MR. STELLO:
No.
As I just told the Commission, we 17 -
are prepared to go our own way, if the Commission makes that 18 decision.
19 MR. PARLER:
Well, even though you might be uncertain 20 about the answer to the first question, your answer to the 21 second question satisfies my objective.
So thank you very 22 much.
23 COMMISSIONER ASSELSTINE:
I mean in essence what you 24 have is we are going along the same road in terms of making 25 sure we have got a consistent set of indicators.
We are not l
.---,-.--e,
-..-,,,.w.-w w -
,,,,y-,,-.------_,,,-r,-------~,-,w.,y,-y,y
---,e---,
53 1
acking for duplicative.information, we are not creating a lot 2
of new confusion in gathering the information.
But then we get 3
to a fork in the road once we have got the information.
.goes its way in terms of'doing its thing to improve utility 4
s 5
performance; we go our way in terms of carrying out our 6
regulatory responsibilities.
And how we use the information is 3
7 really separate from the INPO involvement.
8 COMMISSIONER BERNTHAL:
But why would the question of
. proprietary information, of INPO supplying information, why 9
10 would that ever arise?
11 COMMISSIONER ASSELSTINE:
It shouldn't have.
12 COMMISSIONER BERNTHAL:
If we are simply going to sit 13 down across a table nd decide yes, these are the right things 14 to request.of utilities, we all agree, and of communication on 15 that subject.
Because at some point, those are public data, 16 and to be sure, INPO does its business and we do ours.
But --
17 COMMISSIONER ASSELSTINE:
It shouldn't have arisen.
18 It's a red herring.
19 COMMISSIONER BERNTHAL:
That should not be a 20 complicated subject, and it seems to be.
21 COMMISSIONER ASSELSTINE:
Well, that issue wasn't 4
22 dealt with quite as effectively as it could have been, I think.
23 Bill noted that early on.
I don't think that is a problem.
24 CHAIRMAN ZECH:
Well, let me just say, too, that INPO 25 has been involved in this performance indicator program for i
c
--e.
r e
v,:
e-
,-.r,.-
.m,,,. -.,.,, -. - - - - - -. -,... - -
,m.-.
,e,,,n--..-.-
,..,+ n -,.. - - -
54 1
five or six years, as I understand it.
'2 MR. STELLO:
Longer than we; that is correct.
3 CHAIRMAN ZECH:
They have got a fairly well developed 4
program.
It's a program that it would seem to me would be 5
useful for us to learn from, if we want to put one in effect 6
just to see what pitfalls they've had.
Whether we use the same 7-information -- we can gather it ourselves -- whether we get it 8
from them or not, I don't think that's the point at all.
The 9
point is it's a very useful tool, and I think we should go 10
. ahead and develop it.
We use INPO's data if we want to; if we 11 have any reason to doubt it, we don't use it.
We have all 12 kinds of optiens.
We have regulatory responsibilities, whereas
'I 13 they do not.
They encourage excellence and improvements, and 14 that's good; we certainly encourage that, too.
But we have f
15 other responsibilities, and we are developing our program, as 16 far as I am concerned, in order to help us in the long haul to
- 17 prevent incidences, prevent casualties.
And it's a program 18 that can be useful.
It seems to me we should not feel 19 reluctant to use the type of learning that has gone into this 20 program that's been developed for five or six years.
21 But there shouldn't be any question but the data we 22 use is reliable.
We can verify it ourselves, through INPO, 23 through the utilities, and there shouldn't be any question 24 about that.
25 To me, the important point is developing some kind of l
l 4
55 1
a progrca thst will parhaps help us to better monitor the 2
performance of the plants.
That's what we're trying to do.
3 Mr. Carr?
4 COMMISSIONER CARR:
Amen.
5
[ Laughter.]
6 CHAIRMAN ZECH:
All right.
Thank you very much for'a 7
very fine presentation.
We will proceed.
8
[Whereupon, at 3:11 o' clock p.m., the meeting was 9
adjourned.]
10 11 12 13 4
14 15 16
-17 18 19 l
20 21 22-l 23 1
24 l
25 I
t
-m
,y-
-n
~
1 2
REPORTER'S CERTIFICATE 3
4 This is to certify that the attached events of a 5
meeting of the.U.S.-Nuclear Regulatory Commission entitled:
i 6
7 TITLE OF MEETING: Discussion of Performance Indicator Programs 8
PLACE OF MEETING:
Washington, D.C.
9 DATE OF MEETING: Tuesday, June 9, 1987 10 11 were held as herein appears, and that this is the original 12 transcript thereof.for the file of the commission taken
{
13 stenographically by,me, thereafter reduced to typewriting by 14 me or under the direction of the court reporting company, and 15 that the transcript is a true and accurate record of the 16 foregoing events.
hw -
()4,4----------
18 Ann Riley 19 s
20 i
'21 22 Ann Riley & Associates, Ltd.
r 23 24 25 J
r.e---,
--~~ --,
y,,-
-w,,,,..,,,w-
--,e--------w,-,----,~,--em-e,,,m-m.m-m--,--,-.--,,-,
-.,_.----,--,-,~,-ee.--
.-4' W
Q PERF0 WANCE INDICATOR PROGRAM FOR.
O RATING NUCLEAR POWER PLANTS h
i-
- STATUS OF TliE PROGRAM
- PLANS FOR DEVELOPENT
- COORDINATION WITH INDUSTRY
- C00PDINATION WITH INP0 JUNE 9, 1987 r
I p
i
!~
i
O BACKGROUND OF THE PROGRAM STAFF BRIEFED THE COMMISSIO.N ON A PROPOSED PROGRAM (SECY 86-317)
- ON:NOV 7,-1986 THE COMMISSION APPROVED THE PROGRAM IN DEC, 1986 WITH SOME CHANGES SRM REQUIRED STAFF POLICY ON PROPER USE OF PERFORMANCE INDICATORS DELETION OF ENFORCEMENT ACTION INDEX TAKE INTO ACCOUNT INPO CONCERNS EXPLORE INDICATORS IN SEVERAL AREAS 9
8 1
7 m..
.m,,
STATUS OF THE PROGRAM SEVEN INDICATORS CURRENTLY IN THE PROGRAM
' TOTAL AUTOMATIC SCRAMS SAFETY SYSTEM ACTUATIONS
'SIGNIFICANT EVENTS SAFETY SYSTEM FAILURES FORCED OUTAGE RATE EQUIPMENT FORCED OUTAGES PER 1000 CRITICAL HOURS COLLECTIVE RADIATION EXPOSURES COLLECTIVE RADIATION EXPOSURE RECENTLY ADDED TO THE PROGRAM DATA CURRENTLY AVAILABLE TO STAFF ON ANNUAL BASIS
' ~
QUARTERLY DATA PLANNED TO BE OBTAINED FROM INP0 DATA FOR THE OTHER SIX AVAILABLE FROM NRC SOURCES
.. TWO QUARTERLY REPORTS PROVIDED TO THE SENIOR MANAGEMENT IN FEB '87 AND MAY '87 BOTH REPORTS FORTHCOMING TO THE COMMISSION DATA GENERALLY CONSISTENT WITH OTHER INFORMATION SOURCES 2
.c 1
PLANS FOR DEVELOPMENT INTEROFFICE TASK GROUP (AE0D, NRR, RES, REGIONS) CONTINUES TO REVIEW DEVELOPMENTAL ACTIVITIES MAJOR ACTIVITIES REFINEMENT OF THE PROGRAM RISK-BASED PIs PROGRAMMATIC PIs STATUS DUE TO THE COMMISSION IN DEC '87 REFINEMENT OF-THE PROGRAM DATA PRESENTATION / DISPLAY FACTORING IN PLANT TYPE, AGE RESOLUTION OF STATISTICAL ISSUES RISK-BASED Pls RISK-WEIGHTING OF SOME EXISTING PIs UNAVAILABILITY (SYSTEM / TRAIN FROM LCOs)
TRANSIENT FREQUENCY INTEGRATED WITH UNAVAILABILITY i
3
y
~
l
PLANS FOR DEVELOPMENT (CONT'D)
PROGRAMMATIC PIs (NEAR-TERM)
MAINTENANCE (ITEMS OUT OF SERVICE, LCOs, SAFETY SYSTEM REWORK, CAUSE CODE)
TRAINING (OPERATOR LICENSING EXAM RESULTS, OPERATOR ERROR AS CAUSE CODE)
CAUSE~0F EVENTS (PERSONNEL ERROP, MAINT,, EQUIP. FAILURE, DESIGN / FABRICATION / CONSTRUCTION ERROR).
STAFFING-(VACANCIES, OVERTIME, TURNOVER RATE, CONTRACTOR SUPPORT)
MANAGEMENT (GENERIC ISSUES BACKLOG)
~
PROGRAMMATIC PIs (LONG-TERM)
FOCUS ON PREDICTIVE PI FOCUS ON ORGANIZATION / MANAGEMENT EFFECTIVENESS ADDITIONAL INDICATORS IN SEVERAL AREAS L
L r
4 COORDINATION WITH INDUSTRY CONTACTS WITH INDIVIDUAL LICENSEES ANS EXECUTIVE SEMINAR (SEPT 1986)
REGIONAL WORKSHOPS FOR LICENSEES (JAN-MAR 1987)
COORDINATION WITH INP0 COORDINATION WITH INP0 ON ON-G0ING BASIS FOR DEVELOPMENT AND IMPLEMENTATION JUNE 11, 1986 MEETING WITH INP0/ INDUSTRY ON PI DEVELOPMENT AUGUST 15, 1986 MEETING AT INP0 ON PI DEVELOPMENT NOVEMBER 19, 1986 MEETING AT INP0 ABOUT DATA SHARING SEVERAL CONTACTS FOR REVIEW 0F DATA MAJOR RESULTS OF C0 ORDINATION COMMON DEFINITIONS OF CERTAIN PIs COORDINATION PLAN 5
~
_,.I#
5 COORDINATION WITH INP0 (CONT'D)
C00RDINATION PLAN OBJECTIVES CLARIFICATION OF PURPOSES y.
RECOGNITION OF-LIMITATIONS OF Pls PROVIDE FRAMEWORK FOR C0 ORDINATION TO REDUCE DUPLICATION IN DATA COLLECTION AND DEVELOPMENTAL ACTIVITIES f
CONTENTS / BENEFITS 0F THE PLAN INP0 WILL PROVIDE DATA FOR COMMON INDICATORS AND STAFF WILL SPOT-CHECK (REDUCTION IN THE DATA PROCESSING)
PI REPORTS WILL RECOGNIZE INP0-PROVIDED DATA REPORTS WILL BE IN PDR HELP ENSURE PROPER USE OF PIs IMPROVE PROGRAM EFFECTIVENESS AS A RESULT OF BETTER COMMUNICATION WITH INDUSTRY 6
,