ML20059L839
| ML20059L839 | |
| Person / Time | |
|---|---|
| Issue date: | 11/10/1993 |
| From: | NRC COMMISSION (OCM) |
| To: | |
| References | |
| REF-10CFR9.7 NUDOCS 9311170437 | |
| Download: ML20059L839 (85) | |
Text
-
M4%%W6Nd6%%%WfVfV6%%%%%%%%% tvd!6%%46%%tgyghghgygyggg
?
3
~ AMSMIT Al. TO:
Y Occument Control Dest. 016 Phillips
[
{
s IDVANCED CCPY TO:
The Public Occument ocem G
E 2
//// f-[93 3
CATE:
5' 5
SECY Correspondence & Recorcs Branen g
FROM:
g 3
3 3
3 Attacned are copies of a Comission m'.eting transcript and related meeting 3
cocument(s).
They are being forwarced for entry on the Daily Accession List and j
2 placement in the Public Document Roem. No other cistribution is recuested or recuireo.
g 5l f
Meeting
Title:
/
1 A h
/b -o' ed
/La e h s
hh
~
G m
c 3
Meeting Date:
///ro /93 Open X
Closea
~
3 3
E' l
E C
3 in Item Cescriptien*:
Cooies Advanced DCS C
to POR g
f l
- 8 i
}l l
- 1. TRANSCRIPT 1
1 9,
/, ! !4 [.n omt; d)
(Eif '
I f
W h) lE 2.
5 C
E 5
g 7.
E
- 5 E
=u 6
k m
h h
c M
c' g
t.
k i
c:
w h.
~
c 9311170437 931110 t
- 00R is advanced one copy of each document, two of each SECY pacer.
i' C&R Branch files the original transcript, with attacnments, witheut SECY jj i
@ ill
- acers.
170002 A/RE
..u
i i
i
)
UNITED STATES OF AMERICA i
N JCLEAR REGULATORY COMMIS SION 1
-O i
p.1..
---a;.
BRIEFING ON NRC RESEARCH PROGRAMS ON HUMAN FACTORS l
3C3I, C.5l ROCKVILLE, MARYLAND 356*.
NOVEMBER 10,1993 l
I 3h33' 71 PAGES EAL R. GROSS AND CO., INC.
C O L' R T REPORTER $ AND TRAhSCRIBERS 1323 Rhode Island Avenue, Northwest Washington, D.C.
20005 f
(202) 234-4433 d
j 1
DISCLAIMER
,$A This is an unofficial transcript of a meeting of the United States Nuclear Regulatory Commission held on NOVEMBER,10 1993 in the Commission's office at One White Flint North, Rockville, Maryland.
The meeting was open to public attendance and observation.
This transcript has not been reviewed, corrected or edited, and it may contain inaccuracies.
The transcript is intended solely for general informational purposes.
As provided by 10 CFR 9.103, it is not part of the formal or informal record of decision of the matters discussed.
Expressions of opinion in this transcript do not necessarily reflect final determination or beliefs.
No pleading or other paper may be filed with the Commission in any proceeding 'as the result of, or addressed to, any statement or argument contained herein, except as the Commission may authorize.
e HEAL R. GRO5$
count nooefses AMD TRANSCeGM$
1313 eMost ISLAMe AY9Mut, N.W.
(202) N 433 WASMe4810N,04 te00$
(202) 232 4 000
~
1 UNITED STATED OF AMERICA NUCLEAR REGULATORY COMMISSION i
i j
i i
l
.(d BRIEFING ON NRC RES.EARCH PROGRAMS i
ON HUMAN FACTORS n
i i
PUBLIC MEETING l
i Nuclear. Regulatory Commission i
One White Flint North Rockville, Maryland i
t Wednesday, November 10, 1993 I
l l
The. Commission met in open
- session, pursuant to
- notice, at 2:00 p.m.,
Ivan
- Selin, i
Chairman, presiding.
l l
COMMISSIONERS PRESENT:
IVAN SELIN, Chairman of the Commission KENNETH C. ROGERS, Commissioner FORREST J. REMICK, Commissioner E. GAIL de PLANQUE, Commissioner l
i e
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W (202) 2344433 WASHINGTON. D C 20005 (202) 2344433
,~
2 STAFF SEATED AT THE COMMISSION TABLE:
JOHN HOYLE, Assistant Secretary KAREN CYR, Office of the General Counsel JAMES TAYLOR, Executive Director for Operations 4
.{
WILLIAM RUSSELL, Associate Director for Inspection and Technology Assessment, NRR THEMIS SPEIS, Deputy Director, Office of Research 1
4 THOMAS KING, Deputy Director, Division of Systems a
Research, RES i
l 4
GARY HOLAHAN, Director, Division of Safety Programs, 1
AEOD FRANKLIN COFFMAN, JR., Chief, Human Factors Branch, RES FRED COMBS, Chief, Operations Branch, NMSS l
j i
l i'!
i i
l i
i d
j 1
1 1
4 e
9 NEAL R. GROSS 2
COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
(202) 234-4433 W ASHINGTON, O C. 20005 (202) 234 4 433
I 3-j 1
1 P-R-O-C-E-E-D-I-N-G-S I
i 2
2:07 p.m.
l I
l 3
COMMISSIONER ROGERS:
Good afternoon, j
't 4
ladies and gentlemen.
5 Chairman Selin is-not here and has asked 6
me to open the meeting.
7 I am pleased to welcome members. of the
- l i
l 8
staff to brief the Commission on the NRC Research l
1 9
Program on Human Factors.
The research program is
[
10 intended to provide improved understanding of the
- 1 i
1 f
11 capabilities and limitations of personnel involved in
{
[
12 the operation of nuclear power plants.
i l
13 A large number of safety-related events 14 continue to involve human performance.
It is j
15 therefore important that the non-engineering i
16 activities which relate to safety in nuclear plants l
l 17 and operations be given proper consideration.
I 18 The Human Factors Research Program is 6
t j
19 divided into five interrelated areas:
One, personnel i.
t I
20 performance;
- two, human system interface; three,-
21 reliability assessment; four, organizational factors:
Y 22 and five, material's licensees' performance.
An 23 important element of the research program also s
24 includes the development of standards for reviewing i
25 7
and evaluating advanced control systems.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
(202)2344433 WASHINGTON. O C. 20005 (202) 234 4433 t
,-m.-e.-
e...
...v.,--.,.
.--,...-w,.e,.
,-v..
,,_--,--<,.em
,,,,,,,wu,,m.
,,,_e,,--,
.%.-.,m,,
m.
{
4 l
1 The Commissicn was briefed on the 2
organizational factors portion of the research program 3
in January 1991 and the results of a comprehensive 4
review of the organizational factors research was 5
provided in a SECY paper earlier this year.
I 6
understand that research products from this research 7
are being considered for possible use in routine 8
inspections and diagnostic evaluations.
The 9
Commission is interested in hearing about the progress 10 you are making in this area.
11 Today's briefing will focus on users' 12 needs, research products, and the future outlook of 13 the research program.
The briefing will concentrate 14 on significant research accomplishments over the past l
15 ;
two years.
i 1
1 16 I understand that copies of the viewgraphs i
17 '
are available at the entrances to this room.
['
18 I think the Commissioners would very much 19 appreciate to hear specific results that have come out l
20 of the program and anything that has actually been
)
21 completed would be very good to hear a little bit more 22 about.
23 Are there any other opening comments?
24 Mr. Taylor?
i 25 i
MR. TAYLOR:
Good afternoon.
With me at NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS i
1323 RHODE ISLAND AVENUE N W.
(202) 234-4433 W ASMNGTON. D C 20005 (202)2344433
5 1
the table are members from the Office of Research who 2
will give the major presentation this afternoon, but 3
in addition there are members of the Office of NRR, 4
AEOD and NMSS who are user offices of the results of t
5 this research.
6 Doctor Speis has some opening remarks.
7 DOCTOR SPEIS:
Thank you.
8 Commissioners, Mr. Chairman, it might be 9
useful to provide some background regarding NRC's 10 Human Factors Regulatory Research Program.
If you 11 recall back in 1981, RES established a branch to 12 conduct human factors research.
In 1985, budget 13 limitations and completion of several projects led to i
14 a sharp reduction of resources dedicated to human i
l 15 i factors
- research, leaving only work on human I
16 reliability analysis from 1985 to 1987.
17 But by 1987 the persistence, as you 18 mentioned, Commissioner Rogers, of human errors in 19 reportable events and the recommendations of the 20 National Research Council's National Academy of 21 Sciences led to a revitalization of human factors i
22 research.
t 23 In 1987 then, RES reestablished a Human 24 Factors Regulatory Research Program.
Research i
25 projects were initiated based upon user needs request, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W j
(2 2 2344433 WASHINGTON D C 20005 (202) 2344433
6 l
1 past research experience
- and, where applicable, 2
recommendations of a second report from the National 3
Research Council in 1988 entitled,
" Human Factors 4
Research in Nuclear Safety."
The research projects J
5 addressed the regulatory of fice needs at that time and 6
most of the National Research Council's specific 7
recommendations.
By
- 1989, all of the National 8
Research Council's applicable recommendations were 9
being addressed.
Basically, they had a number of 10 recommendations, I think somewhere around 50, and the 11 majority of them really overlap with our regulatory 12 needs.
So, that 's why we went ahead and addressed 13 most of their recommendations.
14 Since then, the Human Factors Research 15 Program has been mostly directed toward addressing l
16 regulatory needs -identified by the user offices.
i 17 Progress and experience has served to stabilize the 18 funding level for this research and we'll be talking 19 about the funding level in our presentation.
20 I would like to nention to you one area 21 where our research has reached an impasse and that is 22 in the area of organizational factors research.
As 23 Commissioner Rogers said, we reported to you on this 24 issue in SECY-93-020 in February of this year.
The 25 ultimate objective of that research was to see whether NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(202) 234 4433 wASMiNGTON. D C 20005 (202) 2344433
7 1
and how we can translate organizational-performance l
2 into risk.
That is, whether we're able to explicitly i
3 account in PRA management effectiveness as explicitly 4
as possible.
s 5
Even though the research on organizational 6
factors has provided some insights, the direction we 7
took turned out to be very resource intensive and we 8
have reached the point where we have to decide where i
9 we go from here basically.
j 10 At the present time we are still trying to f
i 11 decide if there is something practical or physical 12 which we might do in this area.
Mr. Coffman will 13 discuss this topic further in his presentation.
I 14 Again, the briefing will focus mostly on l
15 recent progress from the research program and the i
i I
16 current plans for the future.
Mr.. King and Coffman 17 will proceed with the detailed presentation.
18 MR. KING:
Thank you, Themis.
I 19 (Slide)
On page 2 is an outline of the 1
20 content of the briefing.
Basically ' 'I'm going to 21 provide a little background and introductory material 22 on the Human Factors Research Program. Frank Coffman, 23 who is the Chief of the Human Factors Branch in 24 Research, will then talk about the content of the 25 Human Factors Research Program broken into the five NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS.
1323 RHODE (SLAND AVENUE. N W 12C2) 2344433.
WASHINGTON, D C. 20005 (202) 2344433 a.
8 1
topical areas that Commissioner Rogers mentioned in 2
his opening remarks, and he'll focus on the issues, 3
the approach to research, the products so far and then 4
our plans for the future in those areas.
Then at the i
5 end I'll say a few words about the long-term plans for i
6 human factors research.
7 (Slide)
Beginning on page 3, as Doctor 8
Speis mentioned, the Human Factors Branch was formed 9
in 1987.
It is in the Office of Research and Frank 10 Coffman is the Branch Chief.
11 The overall objectives of the Human 12 Factors Branch, there are basically three.
The first 13 is to develop technical bases for regulatory 14 requirements and guidance in areas related to human i
15 performance.
Basically that means generate information that can be used to establish and support 16 17 regulatory positions in the human factors area.
That 18 includes looking at a range of issues involving human l
l 19 performance, both reactor and materials licensees in i
20 those
- areas, man / machine interactions, and that 21 includes the use of advanced instrumentation and F
22 control systems, human factors generic safety issues, I
23 and it covers both current and future plant issues.
24 Secondly, an objective of the oranch is to 25 develop techniques and data that accurately measure NEAL R. GROSS I
i i
COURT REPORTERS AND TRANSCRIBERS l
i i
1323 RHODE ISLAND AVENUE. N W.
I i
(202) 2344433 W ASHINGTON. D C 20005 (202) 2344433
9 1
human performance. That includes development of human i
j 2
reliability, analysis techniques and a database on l
i 3
human performance.
6 4
- Thirdly, the branch provides staff 5
expertise on human performance.
Basically they're a 6
resource of human factors talent that supports the I
1 h
7 program offices in licensing activities and responding I
1 8
to questions.
i 1
9 1
Currently, all of the human factors 10 research is driven by regulatory needs or user needs, i
i 11 as we sometimes call them, that come from the program
\\
12 offices.
We received 100 user need requests over the 13 past five years, of which 42 are currently active.
14 These user need requests are usually specific requests
(
15 in scope, schedule and desired end product.
The 16 breakout of how many of those came from the various 1
J
{
17 j program offices is shown at the bottom of page 3.
But 18 I do want to mention that in receiving those user need 19 requests, we do it's been our experience that I
20 there's been good cooperation and coordination among 21 the offices to provide requests that meet maybe
)
I t
22 multiple needs and are not contradictory to each 23 other.
24 COMMISSIONER REMICK:
Would these offices 25 have any technical assistant efforts in. human factors NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
4202) 2344433 W ASHINGTON. O C. 20005 (202) 234 4433
_.,,,.. _.. -..., - -. _.,,, - ~ ~ ~ ~ ~ - - ' ' " " ~ " ' " ' * ~ ' ~
i 4
i 10 j
i 1
also, in addition to research? Would it be extensive 2
or --
3 MR. KING:
Maybe Bill wants -- NRR has a 4
i 4
branch that has human factors --
5 MR.
RUSSELL:
We have a Human Factors 4
6 i
Branch and we have technical assistance.
Most of it 7
relates to activities associated with design 8
certification, current licensing review activities 9
that are ongoing.
But there is some significant i
i 10 interface back and forth between the two and we i
11 conduct frequent meetings with research to ensure that i
l 12 these are coordinated and they're done at least at the s
13 division director level quarterly.
14 COMMISSIONER REMICK:
Do we know if NMSS i
15 i and AEOD have any?
l l
16 MR. HOLAHAN:
AEOD has, in effect, one I
17 section dedicated to human performance and it has 18 contract assistance at INEL.
Most of that is used to l
have human factors experts go out to plants to follow-i 19 20 up on specific events and we're also developing a 21 database of human performance and that's on the order 22 of a few hundred thousand dollars a year.
23 MR. COMBS:
NMSS has two human factors 24 specialists involved in coordinating with Research and 25 also with some contractor support with Lawrence NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE N W 1202) 234 4433 WASHINGTON, D C 20005 (202) 234 4433 I
11 1
Livermore and INEL for human factors and risk work.
2 COMMISSIONER REMICK:
Thank you.
3 CHAIRMAN SELIN:
I would like to follow-l j
)
4 up.
5 First of all, I'd like to thank you on a 6 I different topic, for the preparation you gave me 7
before I went overseas.
You tripled my knowledge of 8
breeder reactors in about three minutes, which was not 9
much of a challenge, but it was very helpful.
10 On this
- topic, following up on 11 Commissioner Remick's questions, I'm sort of concerned 12 about what looks superficially at being either not the 13 right placement or some duplication of some of the t
i 14 database and some of the empirical work.
A lot of the 15 data come in through AEOD and you would expect that i
h the toting up of the empirical data would be sort of 16 d
1 l'
17 a natural function for the AEOD section to carry out 1
18 ij and that Research would have two functions. The first 19 is to do what I'll call non-heuristic, you know, i
20 synthetic research on the factors, experiments or what 21 have yta to supplement the information that comes in 22 tron our licensees.
The second is to try to be the 23 single source of contact and knowledge on everything 24 the Agency knows in this area and some other points.
25 But, you know, we've been running this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W 42021 234 4433 WASHINGTON. D C 20005 (202) 2344433 L
12 1
large database for a long time at quite a significant i
2 expense.
I hope today you'll address whether we still 3
think that's a good idea and, if so, how does that 4
compare to what's going on at AEOD and, if not, what 5
we should do about it.
l 6
The second question I have you haven't 7
really gotten to yet, but something we addressed last 8
February or March and that was where to do the PRA l
9 work or the human factors work that's part of the PRA.
10 I know these are more organizational and management 11 questions than they are research questions, but they 12 do have to do with the management of the research 13 functions.
So, I hope you'll address those as we 14 continue our discussion this afternoon.
15 MR. KING: All right.
Perhaps when we get 16 to the right part of the briefing --
17 CHAIRMAN SELIN:
However you wish to do 18 that.
l 19 MR. KING:
-- we can come back to this.
20 (Slide)
Let me continue on page 4.
21 I need to mention that user needs change i
22 with time.
I think it's a fact of life that as
]
)
23 research results come in, other new issues are raised 24 and so forth, that user needs will change.
To some 25 extent, our research program has been an evolving NEAL R. GROSS COURT AEPORTERS AND TRANSCA!BERS 1323 AHODE ISLAND AVENUE. N W 1202) 234 4433 WASHINGTON. O C 20005 i
(202) 2344433
13 1
program over the past several years to respond to 2
these changes.
Currently, we have 51 projects or l
l 3
separate contrccts, if you will, that are being l
4 directed by the Human Factors Branch.
There are ten i
j 5
project managers in that branch, mostly with human 6
factors backgrounds.
It involves 26 contractors that 7
include a broad spectrum of organizations, both 8
domestic and foreign.
9 In addition to formal contract work out of 10 the
- branch, the branch does maintain extensive 11 interactions with other organizations on human factors 12 subjects.
Those are both formal and informal.
By 13 formal I mean they participate in formal information 14 exchange agreements or participate in committees, i
15 working groups, standards committees and so forth in 16 the human factors area.
By informal, they maintain l
17 l good working relationships with a
number of i
18 i organizations that provides for a free exchange of 19 information.
All of this results in most of the 20 active regulatory needs being addressed in accordance 21 with the priorities from the user offices.
I put the 22 word "most" in there because we've had to negotiate on i
23 schedule sometimes due to work load in other areas and 24 priorities in other areas.
25 Page 5 shows the FY '94 funding for the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W (2;2 234m33 V. ASHINGTON D C 20005 (202) 234M33
,=
i 14 i
1 branch, broken out by the five topical areas.
The 3
2 five year plan shows pretty stable funding in the 1
3 human factors area.
We have about the same level of 1
4 funding in there for FY
'95 and anticipate l
5 approximately a $6 million program in the years beyond 6 i that.
1 7
Now I'd like to turn it over to Frank I
t 8
Coffman who will go through each of these five topical 9
areas and try and highlight the major points and focus 10 on the progress and plans.
I do want to emphasize the 11 slides are not a comprehensive list of everything that 12 they've done, but we tried to pick out the more l
visible and important items.
13 14 MR. COFFMAN: For each of the five topical 1
15 l areas I'll cover the issues and then kind of a l
16 !
characterization of the research program, then focus b,
17 l in on recent products and then what our plans are.
In 18 the first area, which is personnel performance, this 19 deals with the issues, primarily the fact that has 20 been mentioned already, that a
large number of 21 operating events involve human errors.
The Agency is 22 aggressively pursuing a determination of the causes.
23 So, there was a need determined for a method, a
24 standardized method to be used across the Agency for i
25 investigating events to determine what, in fact, are NEAL R. GROSS l
COURT REPORTERS AND TRANSCRIBERS 1323 AHODE ISLAND Av'ENUE. N W
.202) 2344433 w ASHINGTON. D C 20005 C02) 234-4433
~
15 1
the root causes of those that involve human 2
performance.
3 Then there's also the need to characterize 4
the predominant areas of human error.
This might be 5
an appropriate place to mention briefly what we're 6
doing, how the Office of Research is involved with the 7
databases.
That is that we're involved with AEOD and 8
NRR on task force looking at the possibility of a 9
coordinated database. In addition to that, the Office 10 of Research maintains the NUCLAR database, which is 11 not so much data on causes as it is data for human 1
l 12 probability, human error rates for comparison with t
i 23 I those human error rates that are used in probabilistic I
ij 14 h risk assessments. Then we 're also trying to provide--
l 15 trying to autcmate a technique to get the data that's I
16 collected from one of our projects, which I'll r
}
17 mention, the human performance investigate process, to j
l 18 I get the data that's collected from that and fold it l
'l 19 into the database that NRR uses as HFIS, Human Factors 20 Information System.
21 Another issue addressed in the personnel 22 performance area has resulted from the review of
}/
23 recent events.
More specifically, the New Years Eve 24 event where there was a simultaneous scram of both 25 Units 1 and 2 at Sequoyah and there were questions NEAL R. GROSS COUAT REPOATERS AND TRANSCRIBERS 1323 AMODE ISLAND AVENUE. N W (202) 234 4433 WASHINGTON. D C. 20005 (202) 2344433 o
r.-
16 l
raised about the adequacy and the utilization of 2
staff.
So, the research program is addressing that 3
issue also.
A 4
Then the next item is the regional offices 5
have requested guidance on the effects of plant 6
environments on performance.
This is short of the 7
health effects, but how does specific things like 8
heat, light, lighting -- heat and lighting, noise and 9
vibration, how do they affect performance short of 10 having health effects.
11 Then there remains some uncertainty about 12 the fatigue effects of shift length and overtime as 13 far as it might affect safety.
So, the research 14 program is addressing that also.
15 COMMISSIONER REMICK:
Frank, hasn't that 16 research been going on for a decade, the eight hour, 17 12 hour1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> shift question and so forth on fatigue?
i 18 MR. COFFMAN:
It has been going on for 19 some time, yes, sir.
20 COMMISSIONER REMICK: When do you foresee 21 that some resolution of the question --
22 MR. COFFMAN:
We didn't list that.
On i
i 23 page B I'll touch on -- at the top of page 8 I'll I
24 touch on that.
~
25 COMMISSIONER REMICK:
Next a facetious NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W I
(202) 234 4 33 WASHINGTON O C 20005 (202) 734 4 33
I 17 1
question on the bit of noise and so forth.
Is music 2
included in that?
You need not answer that.
l 3
MR. COFFMAN:
No, sir.
4 4
COMMISSIONER REMICK:
Has there been a i
5 report on the staffing to handle significant events?
6 Is there a report out on it yet?
I think an 7
information notice went out, but does Research have a 8
report on that?
9 MR. COFFMAN:
We do not have a report on 10 that.
I 11 COMMISSIONER REMICK:
AEOD?
Does AEOD 12 have a --
l 13 MR. HOLAHAN:
I believe it has come up as i
14 an issue on some individual diagnostic evaluations and 15 IIT teams, but I don't think there's a specific study 16 l on the subject.
l l
I 17 COMMISSIONER REMICK:
Okay.
The reason I I
l 18 i
- ask, on some recent foreign visits I felt some i-j
{
19 staffing was minimal and if there was a report, I'd 20 like to be able to send it to the people.
j 21 MR. RUSSELL:
I recall we have recently 22 sent a SECY paper to the Commission where we addressed 23 issues of staffing, particularly the role of the STA 24 and the dual role STA or the stand-alone STA and we identified some events which occurred and the approach 25 f.
NEAL R. GROSS COURT REPORTERS AND TRANSCR:BERS 1323 RHODE ISLAND AVENUE. N W 4202) 234-4433 WASHINGTON. D C 20005 (202) 234 4433
18 1
was essentially that we would follow-up on events and 2
where necessary we would look at allocations of tasks 3
to staff.
If we concluded that there were 4
insufficient staff to meet and carry out the existing i
5 regulatory requirements, then that would be a basis 6
for concluding potentially that they would need 7
staffing beyond the minimums that are required by the 1
8 regulation.
9 Ne are also waiting for, and Frank will 10 mention this later, in FY '94 there is supposed to be 11 a report that's completed, which we'll talk about in 12 just a moment, in which we agreed to provide feedback 13 to the Commission once we receive that report.
14 MR. HOLAHAN:
It may be worth mentioning 15 that in some of the operating experience we've looked 16 into it's not so much the number of people on shift as 17 the task allocations.
You might find one individual 18 that is simply overloaded and can't do the tasks 19 assigned when there might be other people available, 20 but they're just not trained or assigned to the right 21 tasks.
22 COMMISSIONER REMICK:
Yes.
But there's 23 not a document available yet that one could send out?
24 MR. RUSSELL:
Not yet.
i 25 COMMISSIONER REMICK:
Yes.
Okay.
Thank NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W (2^2) 234 4433 WASHINGTON D C 20005 (202) 2344433 r
1
l 19 1
you.
?
MR. COFFMAN:
(S1ide)
I think we're'on 3
page 7.
4
.To characterize.the research program as 5
involving learning from tha experience of others, both 6
inside and outside the nuclear industry and then to 7
perform some individual studies of our own.
To 1
f 8
emphasize recent product, as was requested at the l
9 beginning by Commissioner Rogers.
We have been quite i
10 successful in the development of-the human performance l
11 investigation process as a
standard method for
{
12 investigating events that involve' human error.
This l
13 has been used and is currently beir.g used' in Region I Ii 14 and by Headquarters personnel.
15 MR. RUSSELL:
In fact, if I could expand 16 on that, we've been using it in the Human Performance 17 Evaluation Branch where we provide assistance in 18 follow-up of events in the regions.
But in the last t
19 two months it's been used at the Vermont Yankee AIT in j
20 October, Comanche Peak special team inspection in 21 November, McGuire AIT in September, Big Rock Point 22 special inspection in October and ' Susquehanna in 23 November.
In each case the feedback that we've been 24 getting is that this has been helpful in looking into 25 the contributing factors to the human performance '
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
}
1202) 234 4433 WASHINGTON. D.C 20005 (202) 2344433
.,.. ~. -
30 1
problems.
So, this is one where we have-seen a
~'
2 benefit in organizing our approach to evaluating i
3 events.
4 COMMISSIONER ROGERS:
But is that process i
5 used in.the absence of an event that triggers a look 6
at --
4 7
MR.
RUSSELL:
No.
It is oriented to 8
follow-up to events.
i 9
COMMISSIONER ROGERS:
Well, that's fine, 10 but we always ought to be striving - to anticipate a
i 11 things rather than simply react to them.
I just i
i i
12 wonder what processes we have that might possibly s
13 discover causes that would be unearthed by this human t
l 14 performance investigation process that we have in i
l 15 place.
16 MR. HOLAHAN:
Well, AEOD is using, in i
17
- effect, the same process from the same research.
l l
18 Although I think you might say we're following events, 19 it's not necessarily a
reactor scram or some-20 significant event like that.
We're looking for i
21 situations in which you can learn something. about 22 human performance.
It might be as simple as 23 miscommunications in the control room that didn't 24 really result in a significant reactor event.
But we 25 have found that the best way to get this kind of human NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W, (202) 2344433 WASHINGTON. O C 20005 (202) 2344433 j
21 i
1 performance information is to go and talk to the i
r 2
people who did something right or something wrong very 3
shortly after they did it.
But that's also a very i
f 4
resource intensive way to collect information.
5 MR. COFFMAN: I think the rese. arch program '
l i
6 gets ahead or the Agency gets ahead of some of the
{
7 areas by looking at some of these events.
For 8
example, we have been requested to look into those 9
events that specifically communications has been 10 called out as a contributing element to really clarify i
11 what is meant by the communications, how did it in 12 fact contribute to the event.
I mentioned that part' f
I 13 of the research products on this effort was l
14 development of training material because training was i
15 done for some of the regions and at headquarters and 16 that material i's be'ing incorporated in the curriculum J
17 at the technical training center.
18 (Slide)
The plant on the next viewgraph, 19 number 8, this shifts over to take a look at our plans 20 and we're doing the study of shift duration and.
21 overtime.
There are two studies involved.
One is 22 looking at the experience that has occurred in the 23 industry and the other is a laboratory experiment, 24 actually we're wrapping this up, where we looked at 25 performance degradation between eight hour and 12. hour NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W (202) 234 4 33 WASHINGTON O C 20005 (202) 234M33
..~r..-,
22 1
shifts at the Institute for Circadian' Physiology.
i 2
Basically there was no difference discovered,.no significant difference discovered in the performance I
3 i
on different tests by the ope.rators, actual operators i
4
(
5 during that laboratory experiment on a part test 6
simulator.
i 7
The next is the second quarter of '94 we t
8 expect to have completed this handbook for.the j
l 9
inspectors on the effects of environment and we're 1
10 supporting -- on that next item we're supporting AEOD j
1 11 in their study looking at - the effects of high-12 intensity lighting.
Actually it's programmed-high-13 intensity lighting at the operations center and how it l
14 might be advisable or unadvisable to use such a
'i 15 system.
i 16 There are no existing reports, but the 17 reports are planned that is minimum staffing levels and the u'tilization of the staff and that's the last 18 19 item there. Our work is to provide a technical bases.
20 We were asked to provide - technical bases to either 21 confirm or that could bi used to modify 10 CFR 22 50.54(m) for both the operating staff and the 23 functions that are required to directly the support 24 the operating staff.
25 I'd like to change to a new topical area, NEAL R. GROSS COURT AEPORTERS AND TRANSCRIBERS i
1323 RHODE ISLAND AVENUE, N W.
4202) 2344 433 WASHINGTON. D.C. 20005 (202) 2344433
l 23 1
if I could.
l 2
COMMISSIoriER REMICK:
Frank, before you 3
leave that, on the bit of the lighting for the Op 4
Center, how do we decide whether we're going to do or 5
sponsor independent research versus hiring outfits 6
that are expert in these areas?
One that comes to 7
mind is Circadian, that the staff had work done a 8
decade ago in some of these areas, I believe.
How do l
9 we decide whether we're going to conduct research or 1
10 call in people that that's tm '
area of expertise to 11 help us?
12 MR. COFFMAN:
I think there are two parts j
13 to your question.
One is how do we decide on what 14 research we're going to conduct.
It's basically 15 driven by the user offices.
When they have an-interest or a need, then that's primarily what drives 16 17 us.
As far as who does the research, that is -- we 18 have several contracting processes and it's a rather 19 rigorous process for determining who might be the best 20 for doing the research.
Perhaps I didn't address the 21 question.
22 COMMISSIONER REMICK:
Yes.
I guess my 23 question is is research needed in the effects of 24 lighting on op Center personnel? I thought there were 25 outfits that specialize in that knowledge as basically NEAL R. GROSS COURT REPORTERS AND TRANSORIBERS 1323 AHODE ISLAND AVENUE. N W (202) 2344433 W ASHiNGTON. D C, 20005 (202)2344433
24 1
consultants and you need not do research, but --
2 MR. COFFMAN:
Well, these are the folks 3
that are involved.
They approached --
4 MR. TAYLOR:
Do you want to answer it?
5 MR.
HOLAHAN:
I'll give you my best 6
understanding of the situation.
I wasn't there at the time the user need was written, but my understanding 7
8 is in effect this is AEOD asking Research to run such i
9 a contract because of their expertise in dealing with 10 the contractor.
So, the contract is being let by 11 someone who understands the technology better than just those of us who are trying to put together the 12 13 operation sector.
14 COMMISSIONER REMICK:
It's really not a 15 research project.
16 MR.~HOLAHAN:
It's not re?lly research in 17 the sense of most other ones, but it's a service that 18 they're providing.
19 MR. COFFMAN:
(Slide) The next topic area 20 is the human system interface, which is our largest 21 area in the branch and is the highest priority area 22 for us.
We've continued to work closely with NRR and 23 their activities on digital INC.
24 The overall issues you can see as we 25 characterize them as the digital systems are being NEAL R. GROSS CCURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE N W G02) 234-4433 WASHINGTON. D C 20005 (202) 234-4433
35 1
included in the plants. There is a need for technical i
2 bases for the review and certification of advanced 3
designs and also for the upgrading of current plants.
I 4
The technical bases work that we're doing is in two i
5 areas.
One is the first being on the systems I
6 themselves, what should be the regulatory positions on 7
systems, and then for the effects on the operator.
8 So, we're working on both aspects of that.'
9 We're headed toward the research 10 program is headed toward the development of standards 11 for both the software and the interface design or the 12 displays and the effects they might have on personnel.
i l
13 COMMISSIONER REMICK:
You'd be a good one 14 to understand now, how would that contrast with the 15 technical assistance that NRR is seeking to help, I
16 assume, in these same areas versus what is being done
{
17 for Research and will the Research results be helpful J
j 18 to NRR in their evaluation --
19 MR. RUSSELL:
Let me illustrate with some 20 background.
Wo briefed you on what we had learned 21 when we visited France and the Bugey simulator for the 22 N-4, which is an advanced sinulation. facility.
Some 23 of the work that they did comparing operator 24 performance in normal control rooms, the advanced 25 control rooms and looking at the tasks they had to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W 1202) 2344433 WASHINGTON. D C. 20005 (202) 2344433
26 1
perform, they found that there was a significant 2
difference in how an operator would spend their time, 7
1 3
navigating through
- menus, et
- cetera, to gather 4
information as compared to walking over to a panel and 5
having a lot of information displayed at one time.
6 That had the potential for change.
7 We've also talked about some of the work 8
that's been done at Halden, at the research facility, 9
where they are specifically looking at some of the implications for human performance of using displays 10 11 and advanced technology.
In most cases the perception i
12 has been that introduction of advanced technology is 13 always a good thing to do and improves the situation.
14 But there has not been a lot of good research done and 15 so we have some requests that are supporting us in 16 those areas broadly.
f 17 We also have work going on which is 18 technical assistance which is assisting us in review 19 of the process of how they are developing control room l
20 design reviews, in particular, how they have handled 21 the layouts of displays and things and we've used 22 guidance that currently exists, much of which is being 23 updated and we have requests to research to update 24 that guidance based upon information display 25 technology and things that are happening.
So, the old NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
4202) 234 4433 WASHINGTON. O C 20005 (202} 234 4433
)
i
27 1
1 guidelines that we had are in the process of being 2
updated.
j 3
So, one I would characterize is trying to 4
understand broadly how the roles'of the operators may i
5 change with the introduction of new technology, what l
6 they may be doing with their time, how that might i
7 affect things.
Secondly, how are they interfacing 8
with the displays, how is the information portrayed?
l l
9 In both cases, research is providing information which 1
10 is then being incorporated into publications which we 11 then factor into the reviews as we're applying them on 12 a case basis.
~
13 COMMISSIONER REMICK:
So it is coming in j
l l
1 l
14 a timely manner that you can incorporate in the f
I o
15 current reviews?
16 MR.
RUSSELL:
We've incorporated-the 17 processes in the current reviews and in most cases we l
18 have put the standards in what we have called tier 2 19 materials, so that if there are improvements in the l
20 standards or changes in technology, we've been careful 21 in the advancer eactor reviews not to lock in a 22 particular tecnnology, but to rather focus on a 23 process for how that technology is proven and how the 24 operator interfaces with it and what their roles are.
25 So, we've been very cognizant of that and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W (202) 2344433 WASHINGTON. D C. 20005 (202) 2344433 1
,,.,,~,..,,,,_,,--..,,,,rI
l, 28 i
1 there's been a lot of interaction back and forth j
2 between the staff and the Human Factors Branch doing i
3 those reviews and the Research staff.
\\
4 COMMISSIONER REMICK:
Thank you.
l 5
MR. COFFMAN:
(Slide)
If I could go.to q
6 page 10 and talk about the recent products in this i
i 7
area.
There are a lot of items actually on pages 10 i
8 and 11 and I was just going to hit the highlights of.
4 9
them, which basically are the first three items on i
10 page 10.
That is that the staff has developed draft 11 guidelines for the human engineering reviews of f
l 12 advanced control rooms and these have already been l
i 13 used for the review of the ABWR and the System 80+ for 14 the design certification.
They will be used for-
{
15 evaluating upgrades of operating plants.
t 16 The second item deals with the fact that i
17 in the past, coming out of the Halden project has been
{
18 reports on the development of computer-based operator I
i 19 support systems. What we have motivated is reports on j
t the insights and the guidelines that might be used by I
20 21 regulatory organizations of which there are some
{
22 members in the Halden project.
The first report that j
l 23 we've received is this one on lessons learned out of l
24 ten years of experience at Halden at the test and I
s 25 evaluation methods that they've used on computer-based
[
NEAL R. GROSS j
COURT REPORTERS AND TRANSCRIBERS
'323 RHODE ISLAND AVENUE. N W L
(202) 2344433 WASHINGTON. D C 20005 (202) 2344433
-..u.
29 I
I systems.
There are two more reports scheduled and j
2 we've asked them actually for a total of six reports f
3 and I'll mention the other two here in a few minutes.
4
- Also, the staff held a
workshop in 5
September on digital system reliability and nuclear' 6
safety.
From that workshop we received feedback from t
7 those experts that the Agency had not previously heard 8
from concerning the potential safety issues.
We also
[
I 9
provided them proposed regulatory -- well, frameworks 10 for proposed regulatory positions and then we heard i
11 from them also on research.
So, this was a way of I
j 12 continuing the in-depth interaction with experts in i
13 the state-of-the-art.
The experts pointed to some 14 potential sources of errors for us.
We knew about 15 these, but it was the emphasis that was given to them.
l 16 One is in the ability to capture specifications for 17
- software, the need for tools for computer-aided 18 software engineering tools during the design and 19 during the audit.
There is a trend toward the use of 20 modules or blocks of previously developed and used 21 code and that appears to be something that is growing 22 in use.
Then they suggested the need for an error 23 collection and tracking and analysis system or 24 activity so that characterization of what kind of 25 errors have been occurring and where the emphasis NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W G02) 2344433 WASHINGTON. D C.10005 (202) 234-4433
. - - -. ~,. -,
30
)
i I
1 should be put could have a solid basis.
_j 2
These points are being considered i
3 primarily for what should be the relative emphasis in
)
4 the regulatory activities and in the research i
5 activities.
And then the other products are there, i
6 are listed there.
7 (Slide)
I was going to go on to page 12, 8
which begins to discuss the plans.
The plans are 9
broken up into two areas.
One is the systems area, 10 which is covered on page 12, and then the operator 11 effects is on the next page.
12 Again, the emphasis is on technical bases 13 and one of the requests that we received was what 14 should be the technical bases or what is the technical 15 bases for requirements on software error analysis.
16 There are two parts to this, both of which the 17 research program is addressing.
One deals with the 18 classification of errors to guide the acquisition of 19 error data and then the other is the study of 20 detection and analysis techniques, how one might 21 detect and analyze the errors that might occur during 22 the life cycle development of the software.
23 The next area is to develop guidelines for 24 verification and validation of expert systems.
This 25 has focused primarily upon application of verification NEAL R. GROSS COURT REPORTERS AND TRANSCRIDERS 1323 RHODE ISLAND AVENUE, N W.
(202) 234 4433 WASHINGTON, D C 23005 (202) 2344433
6 31 1
and validation for the knowledge-based portion of 2
expert systems.
3 I'll go on.
There are --
4 COMMISSIONER ROGERS:
How are you doing 5
that? How are we getting at that basic knowledge that i
6 you want to fold into the expert system?
7 MR. COFFMAN:
Let me call on an expert.
8 Let me ask Leo Beltracchi, who is our project manager 9
on this project.
t 10 MR. BELTRACCHI:
What we actually did was l
l l
11 to conduct an experiment and we had a control group t
12 and an experimental group.
We actually had seated 13 errors in two expert systems and compared the 14 performance of these two groups in terms of being able 15 to determine errors. We found that through the use of 16 the experimental system where they had equivalent of 17 case tools, they were actually able to detect most but 18 not all of the errors.
We found it was an effective 19 way of assessing the knowledge base.
20 COMMISSIONER ROGERS:
Well, I was really thinking of how you develop your -- you know, how you 21 22 get your original collection of material that you're 23 building the knowledge base on.
24 MR. BELTRACCHI:
Oh, you're talking about 25 knowledge acquisition then.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W (202J 2344433 W ASHINGTON. D.C. 20005 (202) 234-4433
2 32 l
1 COMMISSIONER ROGERS:
Yes, right.
2 MR. BELTRACCHI:
Okay.
We did not look f
3 into that aspect of it with regards to this -- in this 4
program.
We were looking at the existing expert 5
systems and how we would verify and validate them.
l 6
COMMISSIONER ROGERS:
I see.
t 7
MR. COFFMAN:
Thank you, Leo.
8 The second report from Halden is mentioned 9
there in the middle and that's on lessons learned from 10 verification and validation experience that they have I
11 had at Halden over the last ten years and it will 12 address such things as the use of formal methods and 13 testing techniques and the use of testing.
14 One project we have is to develop a 15 software audit tool or the prototype of a software 16 audit tool for use by NRC reviewers where they would 17 be looking for common code within the element that's 18 supporting different functions, different outputs from 19 that code.
Then a project which we're trying to get 20 underway which has been requested is to look at 21 programming languages, looking at their 22 characteristics and
- how, in
- fact, the unique i
23 characteristics of the language might be problematic j
24 in a safety application so that, coming out of this I
25 would be guidance for the reviewers that when - a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS l
1323 RHODE ISLAND AVENUE. N W.
(202) 234 4433 WASHINGTON, D C 20005 (202) 234-4433
i 33 1
program comes in in a given language that they would 2
have some hints as to what could be the potential l
1 3
preblem areas for that language.
4 (Slide)
Then to look at the plans on page l
i i
5 13, for the effects on operator workload.
A l
6 typographical error in that first line is that it's e
]
7 the fourth quarter.
It's not the first quarter of i
1 j
8 fiscal '94, it's the fourth quarter of fiscal '94 that 9
we'd expect to complete draft guidelines for human r
10 engineering reviews.
These are -- this is because we 4
i 11 will be going through CRGR and public comment.
A lot 12 of the material coming out of Halden was used in the 13 development of these guidelines.
14 We have reports, two reports on the 15 effects of computerized procedures on human i.
16 performance.
We're assessing the effects of digital i
4 q
17 systems on operator workload and the third lessons i
18 learned report from Halden deals with what they've j
19 learned over the ten years on man / machine interfaces.
20 It summarizes their experience with workload and how I
21 they have made decisions between allocating tasks to 22 automation versus to the operator.
It includes other 1
23 things such as large screen displays.
24 COMMISSIONER REMICK:
Since those items 25 have to do with staff review, and I look at the time NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W (202) 2344433 WASHINGTON, D C. 20005 (202) 2344433
34 1
scale, how is that going to help NRR in its review of 2
the evolutionary and passive plants?
3 MR. RUSSELL:
I believe the comments I 4
made earlier, we are not locking in a particular 5
technology, we're using design acceptance criteria as 6
the approach to the control room. design and to the INC 7
system designs so that there is the capability to 8
incorporate both newer technology from the standpoint 9
of types displays, et cetera, and also to factor in 10 the lessons learned from the standpoint of how you 11 display those on the instrumentation and tools that 12 you use.
So, we have been careful not to specify 13 particular man / machine interface technology, but 14 rather a process for evaluating that and going through 15 a V&V, and how you do testing, including man-in-the-16 loop testing with simulation.
17 Now, we concluded for the evolutionary 18 plants that the role of the operator was not going to 19 substantially change from the standpoint of their 20 involvement, use of systems, et cetera.
That is the 21 approach to emergency procedures are still pretty much 22 the same, but we did feel that for the passive plants 23 that they were sufficiently different in the context 24 of using non-safety systems, et cetera, that we would 25 require more extensive man-in-the-loop testing as a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVEtJUE. N W.
(202) 2344433 WASHINGTON. O C. 20005 (202) 2344433
e J
J 35 1
part of the V&V process, where they would be using the 2
actual displays and information.
3 COMMISSIONER REMICK:
So, you do not need
~
4 this information for reviewing the DACs themselves?
l 5
MR. RUSSELL:
That's correct.
6 COMMISSIONER REMICK:
It's the l
7 implementation of the DACs that you'll need this for.
8 Is that it?
l 9
MR. RUSSELL:
We did review standards and i
10 information that's available based upon current 11 technology that would be used and to the extent that 12 technology is used, we have approved the standards 13 associated with that technology.
But as we did that 14 review, we put it into a tier 2 status, that it's 15 resolved if that's used, but we did not lock it in to 16 the point where we'd need to go back to a rulemaking 17 if they wanted to introduce new technology.
So there 18 is a process for handling that.
19 COMMISSIONER REMICK:
Okay.
Thank you.
20 MR. RUSSELL:
I might comment, and I know 21 some of you have been to Halden.
But I think it's 22 probably one of the better research facilities from 23 the standpoint of conducting these types of 24 experiments.
They have a simulator that they can 25 reconfigure quite easily to different display NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS f
1323 RHODE ISLAND AVENUE, N W.
(202) 234 4433 WASHINGTON, D C. 20005 (202) 2344433
?
36 1
technologies using projection screens, et cetera.
2 They have access to operators that operator the plant 3
that is simulated and they've done quite a bit of work
{
4 in alarm reduction and other things.
So, it's an area 5
where I think from a program office standpoint we get 6
a lot of results for relatively modest cost and it's 7
one that is not duplicated here in the U.S..
8 MR. COFFMAN:
(Slide)
I'll shift to the 9
next area, which is organizational factors, a topical 10 area on page 14 and just mention.
that by
~
l 11 organizational factors we mean such things as the 12 quality of communication of the organizational internally and externally, coordination of the work, 13 14 that is the degree to which the coordination of the 15 work is formalized, decision making, such things as 1
16 the degree to which the decisions are centralized, the 17 making of the decisions are centralized, assignment of 18 personnel and resources and then some more vague l
19 things more difficult to measure, like culture, the 20 values and practices.
21 The initial issue, as was mentioned, was 22 to measure these factors and then fold them into PRAs.
23 The products to date have -- well, we've identified 24 factors.
We've kind of somewhat got convergence among 25 our contractors on the factors and we have developed NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W 420;h 234403 W ASHINGTON. D C 20005 (202) 234 4433
- ~.
37 i
1 methods to qualitatively mecsure those factors.
The i
2 methods at this. point have been used by behavioral 1
3 scientists.
We've tried these methods at two plants.
I 4
They've been good performer plants and documented our.
(
5 results.
We've got some preliminary attempts at 6
developing a method for the quantification of the i
7 risk, and this is what I'might call the creative step l
i 8
in the process and it's very difficult.
We have been 9
able to discover how organizational factors can create-l 10 dependencies across systeme so that you can have 11 dependencies that occur between dissimilar components I
i l
12 and dissimilar systems.
So, there has been some b
13 progress.
But as was mentioned, we did this I
4 i
14 1 I
comprehensive evaluation of the program and concluded i
that there was progress but it's resource intensive, i
l 15 t
l 16 that the current project should be focused on what j
17 l might be useful for inspections and diagnostic i
l 18 evaluations. We should monitor the werk of others and 19 that NRR and RES should continue to coordinate on what 20 further work might be done.
21 (Slide)
So, on page 15 that's what you'll 22 see.
That's what we've been doing. The monitoring of 23 the work of others has been even the activities of 24 Institute for Nuclear Power Operations, looking at 25 their activities.
They do not have any research going NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(202s234 4433 WASHINGTON. D 0 20005 (202) 2344433 t
..v
--r n---.
.-,,--,-,...,,.-.,-...-n
38 1
on, but basically their activities are to do plant and 2
utility evaluations using peers.
We've been aware of 3
NUMARC's activities in this area to survey the 4
industry on this topic.
We're aware of what MIT is 5
doing in their program.
We're aware of what SKI is 6
doing in Sweden and U.K.
AEA technology work.
Then 7
there's work going on at the National Research 8
Council.
9 The plans are to develop the training 10 materials for incorporating the organizational 11 factors, measures into diagnostic evaluations, but 12 that will in all likelihood require some 13 demonstrations.
But the key questions in this area 14 are the v lidation of the methods and the resource-1 15 j intensiveness of collecting the data.
So, we're in t
i' 16 the process of meeting across the offices and trying l
17 to prepare recommendations for senior management later I
18 l this calendar year.
19 l
COMMISSIONER ROGERS: What's the smallest I
20 organizational unit that you can focus on in this l
\\
b 21 program?
I 22 MR. COFIMAN:
The unit has been the power 23 plant, not to go beyond the power plant.
Within that 24 power plant we have focused on departments and I don't i
25 think we've gone -- it's just departments.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W (202) 234-4433 WASHINGTON D C. 20005 (202) 234-4433
1 39 i
1 COMMTSSIONER ROGERS:
Just departments.
I 1
2 MR. COFFMAN:
Yes, sir.
3 COMMISSIONER ROGERS:
It doesn't include, 4
say, the operating crew as an organization?
5 MR. COFFMAN:
No, sir.
I guess I misheard 6
the question.
It includes the operating crew, but --
7 COMMISSIONER ROGERS: As an organizational 8
unit.
9 MR. COFFMAN:
The answer is yes.
What I 10 was thinking is we also have -- we had another project 11 looking at trying to evaluate the performance of the 12 operating team itself, which is a separate project.
13 COMMISSIONER ROGERS:
Would that be in 14 this --
15 MR. COFFMAN:
It would be in this area, 16 yes.
It's not' tied in with the attempt to quantify 17 the risk.
18 COMMISSIONER ROGERS:
It isn't? Why not?
19 Isn't that one of the biggest things that you ought to 20 be looking at?
t 21 MR. KRAMER:
Joel Kramer, Human Factors t
'22 Branch in Research.
23 Some of our work at Brookhaven looked at 24 measuring operator crew performance and the i
25 organization factors associated with that and NEAL R. GROSS COURT REPORTERS AND TREJSCRIBERS t
1323 RHODE ISLAND AVENUE N W.
(202) 234 4433 WASHINGTON, D.C 20005 (202) 234 4433
40 l
1 developing an algorithm to play that into risk to t
2 recalculate core damage based upon the organizational f
i
\\
3 influences on both operations and maintenance errors.
I a
4 COMMISSIONER ROGERS: So you are trying to 5
get a quantitative risk measure out of an examination 6
of the operating team as an orga.izational unit.
Is I
7 that correct?
t I
8
!y MR. KRAMER:
Right.
i t
4 9
MR. COFFMAN:
(Slide)
To go to the fourth i
i 10
- area, human reliability analysis with the i
11 probabilistic risk assessment methods and.
i 5
)
12 applications, page 16.
The issues here are focused l
l 13 primarily on two items.
One is to develop, methods 14 that can be used in the evaluation of the tech specs, i
1 l
j 15 and the other is to try and improve or validate human 16 reliability estimates. The program has focused on-the -
17 development of these methods for looking at changes in i
18 such things as surveillance test intervals and the 19 effects of dependent failures, the configuration of 20 systems and the methods that are applicable to low l
power and shutdown -- application of-the methods to 21 l
I 22 low power and shutdown operations.
23 That's on the tech spec aspect. As far as -
24 the issue dealing with the validity and ways to improve human reliability estimates, we're finishing 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(202) 234 4433 WASHINGTON. D C. 20005 (202) 2344433
., _, ~....,.. - _ - - -.
.,--n,,.w.-.,--
r+..a.~,,,,,
..m,,,,
- evn,
i 41 1
[
l up soma projects on evaluating the errors of I
2 commission where we're trying to model the errors of l
l 3
intent, the formation of these intentions by the 4
operators and what might contribute.
We've looked at I
i 5
28 teams by way of trying to validate that model and I
l l
6 we're trying to analyze at this point the empirical 7
evidence from those evaluations.
i 8
The last item there is to determine the 9
feasibility of inferring error rates from the data 10 available to the NRC through the simulator portion of i
11 the requalification examinations that take place.
12 By way of recent products in this are 13 covered on page 17 i
14 MR. RUSSELL:
Frank, if we could go back i
15 to the last one for just a moment because this came 16 about as a request from NRR.
We were seeing -- after 1
17 we made revision to the simulator portion of the 18 scenario reevaluating crew performance, we were still l
19 seeing a fairly high failure rate on some scenarios, 1
t 20 indicating that human performance, even in a crew 21 environment, was not satisfactory.
If you just look 22 at the number of exams that we give and the number of 23 times that they fail, particularly if you're in a l
l 24 requalification examination scenario, it gave an indication of an error rate that was much higher than 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS l
1323 RHODE ISLAND AVENUE, N W.
I i
(202) 234 4 33 WASHINGTON D C 20005 (202) 234M33
j 42 l
1 the typical error rates that are used in probabilistic l
1 i
i 2
risk assessment analyses. Maybe an order of magnitude 3
higher or so.
4 So, we started collecting this data 5
through our examination activities.
Where there were i
6 critical tasks that were-not performed that were crew 7
critical tasks, collecting that data, and trying to 8
understand because these scenarios were scenarios that l
l 9
had been validated, reviewed by management prior to 10 administration and given to crews that were qualified 11 crews.
s l
12 So, we've been collecting that data, 4
13 putting it into a database and we've asked Research to I
i 14 look to see what they can discover from that and what it might imply by way of error rates or what it might 15 i
16 imply by way of potential regulatory changes either in 17 how you address some of these, are we putting too much 1
1B reliance on operators and inould there be some design i
l' 19 changes.
So, this was an area that we were exploring l
20 where we wanted to make use of our data from i
i 1
21 examinations and see what we could learn from it.
So, i
22 we thought this was as close as you can.get to the i
23 actual scenario.
You've got the tension, the stress, i
24 the sweaty palms and everything else from the i
e 25 standpoint of the operators being evaluated, and we NEAL R. GROSS l
COURT REPORTERS AND TRANSCRIBERS 1323 AMODE cLAND AVENUE. N W (202) 2344433 WASHINGTON. D C. 20005 (202) 234-4433 1
43 1
were finding that the error rates were different than 2
that which you would get out of handbooks or 3
cookbooks.
4 so that's why this is being investigated.
5 We feel that this is one that should be completed 6
relatively quickly to see what we can learn out of it 7
and whether it makes sense to continue to collect the 8
data from our exams on failure rates and compare 9
scenarios, et cetera.
10 MR. COFFMAN:
(Slide)
On page 17 there's 11 a list of reports which compose methods or rules for 12 use in improving the way the tech specs are evaluated 13 using risk-based evaluation methods. The first two of 14
- these, on allowed outage times, surveillance test 15 intervals were used already on the ABWR on the South 16 Texas reviews.
In addition, there have been over ten 17 topical reports from the vendors on individual systems 18 that these method were used in the evaluation.
The i
19 dependent failures is a method to sort information l
20 available to us about different events for there being 21 candidate, common cause events.
So, it's a screening 22 methodology.
23 The item there mentioned as checklists is l
24 for evaluating -- it came out of this work on trying 25 to model the errors that occur during operator l
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W (202) 2344433 WASHINGTON. D C. 20005 (202) 2344433
44 i
1 formation of intent to act and it's a checklist for 2
what makes events mentally or cognitively demanding.
3 For example, such things as the sequence of queues 4
that the operator receives, the time interval between 5
the queues and maybe his predisposition to focus on 6
safety systems when problems occur in the balance of 7
plant.
Then we're also maintaining this human error 8
database that I mentioned before.
By the way, the 9
human error database, NUCLAR, also contains hardware 10 failure rates, just for convenience of use or review.
11 COMMISSIONER REMICK:
Just for 12 clarification, I can conclude the way the words are in 13 here that the Human Factors Branch is doing the risk-14 based tech spec improvement program.
I assume that 15 Research is doing that and you're talking about the 16 human factors input to that.
Am I correct?
i 17 MR. COFFMAN:
No.
Most of that work was 18 done actually in the branch.
l 19 COMMISSIONER h-ICK:
It was?
20 MR. COFFMAN:
And the branch used to be 21 called Reliability and Human Factors Branch.
22 COMMISSIONER REMICK:
Ah-ha.
I see.
23 Okay.
But it's broader than human factors.
24 MR. COFFMAN:
Yes.
25 COMMISSIONER REMICK:
All right.
Okay.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
(202) 2344433 WASHINGTON, D C. 20005 (202) 234 4433 1
i 45 1
MR.
COFFMAM:
(Slide)
As far as our 2
plans, they're listed there.
Report on the risk i
f 3
perspective of tech specs that require shutdown.
We 4
plan to complete the documentation of a report on the
{
5 study of the risk impact of diesel generator 6
maintenance experience that has occurred during 7
actually it's already been used where we evaluated 8
experience during power operations and we plan to i
9 complete the work by looking at experience during l
10 outages.
11 We plan to issue a handbook because these l
12 methods might -- because there might be an inventory 13 of methods or there will be an inventory of methods on 14 how to evaluate tech specs using risk-based 15 methodology. We plan to issue a handbook to guide the 16 reviewers as to which method might be appropriate.
17 Then if we analyze -- as we complete the 18 analysis of the simulator portion of the 19 requalification
- data, we're going to have 20 characterized that data and then we're going to 21 determine the feasibility of making inferences on 3
22 human error rates.
If that's successful, then there 23 would be more work planned to follow-on and actually 24 use a more empirically-based approach.
If not, then i
25 that would define the limits, the capabilities of the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(202) 2344433 WASHINGTON. O C. 20005 (202) 2344433
46 l
1 information we have.
2 (Slide)
I'd like to go the last area, 3
which is on page 19, which is a
topic called 4
performance of materials licensees and it addresses 5
issues relating to actual and potential human errors 6
in medical misadministrations and in unnecessary 7
exposures during industrial radiography processes.
8 The research program at this point 9
involves studying the functions and tasks performed 10 during the medical application as remote afterloading 11 brachytherapy, manual brachytherapy and teletherapy, 12 and then the industrial radiography.
This would 13 include the research includes looking at 14 procedures, the human system interface itself, the 15 training involved, the organization and the management 16 involved and then the impacts of malfunctions.
We 17 have draft reports on teletherapy and remote 18 afterloading brachytherapy.
Those have been h
19 completed.
20 If you look over on the next page at the 21 plans, the plans include --
22 COMMISSIONER REMICK:
Excuse me.
Before 23 you go to the plan, any major findings in the draft 24 report?
25 MR. COFFMAN:
Well, no, I don't think so, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 kHODE ISLAND AVENUE, N W.
(202) 2344433 WASHINGTON, D C. 20005 (202) 2344433
~,.. -..
47 1
but Jay, do you want to -- there are major findings, 2
but whether they're surprises or not is --
3 MR. PERSENSKY:
I'm Jay Persensky of the 4
Human Factors Branch.
5 Yes, there are a number of findings in 6
each of the areas that Frank mentioned as far as some 7
weaknesses in training, weaknesses in the human system 8
interface.
One of the things that has come out, 9
particularly because of the remote afterloading 10 brachytherapy incidents that have occurred lately, is 11 issued related to the treatment
- planning, the 12 treatment planning computer and how it interfaces with t
13 the other systems.
That seems to be across all the 14 different types of therapy.
So, there will be a 15 number of recommendations that come out of these 16 reports and issues that should be followed up on or 17 addressed in the near future.
I 18 COMMISSIONER REMICK:
Thank you.
l 19 COMMISSIONER ROGERS:
How do you see a l
20 follow-up taking place?
Say once your report is out 21 and the findings are there, what do you see happening 22 after that?
23 MR.
PERSENSKY:
- Well, that will be 24 dependent on the user office primarily, the follow-up 25 in terms that we will provide the information to the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 12/3 RHODE ISLAND AVENUE N W.
(202) 2344433 WASH NGTON. D C. 2000S (202) 2344433 l
48 1
user office, NMSS in this case.
We've talked about i
2 different kinds of things. Some might include further j
3 research.
Others might include the use of voluntary 4
standards or voluntary changes on the part of the 5
industry.
But perhaps Fred can address that better.
6 MR.
COMBS:
Right.
We're currently
{
7 reviewing the draft report on remote afterloading i
8 brachytherapy at this particular point.
Where we 9
don't have the results of that review yet, but what we i
10 envision is that by taking a look at the human factors i
11 aspects, it gives us another perspective to somewhat l
12 validate some of the things that we've seen or would 13 see empirically.
It could very well be that we may 14 end up having to change procedures.
We may end up 15 requiring additional training, depending on exactly 16 what we're finding as the source of serious error in 17 the field of brachytherapy.
18 COMMISSIONER ROGERS:
Well, I guess the j
19 question is is this work stimulating any kind of l
i 20 companion activities in the industry itself that would 21 follow-on on this, or are we the sole players in this 22 game?
23 MR. COMBS:
At this particular point we 24 appear to be almost the sole players.
A member of my 1
25 staff has worked with the Association for the i
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
(202) 2344433 WASHINGTON. D C. 20005 (202) 2344433
49 1
Advancement of Medical Instrumentation in looking at 2
the human factors aspect of the design of medical 3
devices.
We understand that that work which is done 4
by Amy will soon be the source of a new ANSI standard.
5 So, there is work going on and we are a part of it, 6
albeit a small part of this particular point.
7 COMMISSIONER ROGERS:
Okay.
Thank you.
3 8
MR. COFFMAN:
There are always research 9
findings, whether they're surprises or not.
10 (Slide)
By way of completing this, on 11 page 20, to discuss the plans, is to in fact complete 12 the results on remote afterloading brachytherapy and 13 teletherapy.
But we have plans to do the work on 14 manual brachytherapy, but that's pending some 15 confirmation of the user need that has occurred 16 recently, that has come up recently.
17 There has been an interest expressed in 18 the development of an inspection method somewhat of 3
19 the type like the human performance investigation 20 process for use by materials licensees.
So, that's i
21 potential work that is planned. NMSS is reconsidering 22 the need for any future work on industrial 23 radiography. The user need on that came out about the I
24 same time that the rule changed and so there's been 25 evidence to show that might be effective, the rule NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
(202) 2344433 WASHINGTON. ' C 20005 (202) 234 4433
50 1
might be effective and no further research is needed.
2 I've covered a lot of items because the
+
3 program is quite diverse.
But if I were to emphasize 4
the major items, I think I should say that this human 5
performance investigation process has been a useful 6
product that's come out of the work and it is 7
affecting the way we do business, the way the Agency 8
does business.
The guidelines for the review of the 9
human engineering aspects of advanced control rooms 10 and displays has co.ne out of the work and is currently 11 being used for those reviews and then the methods for 12 the risk-based evaluation of the tech specs as major 13 products.
14 MR. KING:
Thank you, Frank.
15 Let me just take two minutes and complete 16 the briefing with a few words' on the long-term 17 outlook. We see a stable budget as projected over the 1
i 18 next several years at about $6 million per year, as I l
1 19 had mentioned before.
We anticipate over the next l
20 couple of years that the work in the branch is going 21 to be dominated by user need requests.
Beyond that 22 point in time we think there will still be some user 23 need requests, but like other research programs we 24 need to start thinking about the long-term goals once 25 we get over this hump of being dominated by user need NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W (202) 234 4433 WASHINGTON. D C. 20005 (202) 2344433 j
l S1 l
1 requests, things like identifying the long-term human i
2 factors needs, looking at what are the issues in front 3
of us, human performance, advanced instrumentation l
\\
4 control, man / machine interface, whatever it may be, 5
what kind of staff and contractor expertise do we want 6
to maintain, what kind of facilities do we want t.o j
7 have access to or maintain ourselves, what do we want I
l 8
to do with the human reliability database and also do
{
9 we want to continue on and is there a need for i
1 10 additional work in the human reliability analysis l
{
11 development and methodology in that area.
And 1
12 continue to look at our involvement in standards l
13 activities and international programs.
I think at 14 this point these items are more questions on the 15 table.
We don't have any answers yet, but we would 16 anticipate over the next year or to to be working on 17 these and trying to come up with our long-range plans 18 in this area.
19 With that, we complete the briefing and 20 respond to any questions you have.
j 21 CHAIRMAN SELIN:
Commissioner Rogers?
22 COMMISSIONER ROGERS:
Well, have you ever 23 suggested any studies or any additional information 24 that might be brought to bear on our application here 25 in NRR or NMSS that did not come from a user need NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(202) 2344433 WASHINGTON, D C 20005 (202) 2344433
52 1
request?
2 MR. COFFMAN:
I don't think so.
I can't t
3 think of any.
So much of this area kind of couples 4
together that sometimes the user need will be focused 5
in on one thing and through the conduct of the 6
research and maybe even experiences that occur it will 7
finally refocus a little to get at the heart of an 8
item that wasn't explicitly called out in the user 9
need.
But no, I think most of it's driven by user 10 needs and most of the items have been identified as 11 user needs.
12 COMMISSIONER ROGERS:
Do you say anything 13 about the human cognitive reliability techniques that 14 some people.have been using and their possibly F
15 application here to some of these
- studies, in 16 particular the one I noticed with respect to some 17 question about the on page 11 of your report, i
18 research plan report, you mentioned the human system 19 interface, that you couldn't seem to see a difference 20 between different display types.
I think that was 21 where it was on page 11, but at any rate someplace in 22 here.
Have you thought about actually doing some 23 studies using human cognitive reliability techniques 24 there?
25 MR. COFFMAN:
The work you're referring NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W (202) 2344433 WASHINGTON. D C. 20005 (202) 234 4433
. ~ - - -
53 1
to, I believe, is the work done at University of 2
Illinois where we were looking to find a measure that 3
would be used to evaluate the displays themselves.
4 Those measures would be tied to how they affect 5
operator performance.
That was partially successful.
6 It was not -- we were not able to tie it to the 7
quantitative recall of the operators, but there was 8
some indication it could have affected his ability to 9
diagnose a problem.
10 So, that's going to complete it.
But the t
11 work that is ongoing and appears promising is the work
~
l 12 at Halden, looking at measures for the ability of the l
13 operator to remain aware of the status of the plant 14 systems.
It's referred to as situational awareness.
15 So, there is work underway at Halden to explore a 16 means, a method, to assess this and use it as a way to 17 then evaluate designs.
18 COMMISSIONER ROGERS:
I guess I'm just 19 puzzled about this diesel generator testing program, 20 where that fits in.
I've often wondered why we 21 couldn't ever come to closure on that thing.
I see 22 that it still turns up as part of your studies, the i
i 23 plans for the future report on the risk impact of 24 diesel generator maintenance strategies.
What's 25 involved?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE N W.
(202) 2344433 WASHINGTON, D C. 20005 (202) 234-4433
54 1
MR.
COFFMAN:
I'd like to ask Carl 2
Johnson, who is project manager on that, to explain 3
it.
l i
4 MR. JOHNSON:
I'm Carl Johnson. I believe l
5 we did come to closure on that.
This report that's 6
referenced here is to document some work which was --
7 the bulk of this was reported to you in a SECY paper 8
last February on the proposed diesel generator rule 9
where the question came up AEOD observed substantially 10 higher maintenance unavailability of diesels than was 11 used or was estimated at the time the original 12 blackout rule was developed and what about that? NRR
{
13 collected the data.
This project evaluated it, found i
14 that there is a substantial amount of time out of 15 service during operation and evaluated the risk of 4
16 that.
That was summarized in the SECY paper that i
17 showed that although maintenance unavailability is 18 important, that diesel reliability is more important.
19 The thing that has not been done or it was 20 not done at that time was what about the maintenance i
21 unavailability during plant shutdown and the risk 22 significance of that.
The data that NRR collected 23 showed that diesels were out of service about 12 24 percent of the time during shutdown.
The shutdown 25 PRAs which are being done in another branch in 1
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W, (232) 234-4433 WASHINGTON, D C. 20005 (202) 234-4433 1
4 55 1
research have reached completion and this project and 2
a couple of related projects are looking at what's the 3
risk significance of that and, in particular, when is 4
the better time to do different kinds of maintenance.
5 That's being wound up now.
6 So, I think we are -- yes, we have reached 7
closure on that.
8 COMMISSIONER ROGERS:
Okay.
I gliess I 9
understand what you're looking at.
l 10 That's all I have.
11 CHAIRMAN SELIN:
Commissioner Remick?
12 COMMISSIONER REMICK:
I found the use of i
13 the requalification exam error rates quite interesting, although those are not necessarily 14 8
15 validated data.
I don't know any better source of 16 data than that perhaps.
But it raised a question in 17 my mind.
Do we ever use our own simulators a't the 18 training center to do any research, although I realize 19 we don't have certified or licensed operators there?
20 We're probably using trainees most of the time.
But 21 do we ever use our own simulators for data 22 acquisition?
23 MR. COFFMAN:
Yes, we have and it was an
~
24 attempt to again look for measures of how the design 25 would affect performance.
So, we have on a past NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W (202) 234 4433 WASHINGTON. D2 20005 (202) 2344433 i
56 1
project.
There are difficulties in scheduling and in 6
2 reconfiguring simulators that are intended.to retain 3
a high --
4 COMMISSIONER REMICK:
Yes.
Okay.
5 Jim, I found it very helpful to have the 6
various offices here at one time so we can get some 7
specific examples or responses to questions.
I found 8
that very helpful.
9 As a general matter in all the research 10 presentations, I'm always interested in knowing what 11 you're doing.
I become more interested when I hear 12 why you're doing it and I become almost excited when 13 I hear about results and uses.
So, just as a general 14 matter, I would ask that in the future you plan on 15 giving us more specific results and how they're being 16 used.
I continue to be impressed how the Human 17 Factors Branch, I think, is an excellent example of 18 using a variety of research providers.
You don't go 19 just to one laboratory, national laboratory, but I 20 think through the years you have used a variety of 21 research providers, depending on what expertise they 22 offered and I compliment you on that.
23 Thank you for the presentation.
24 COMMISSIONER de PIANQUE:
I would also 25 second the notion.
If you can give us some nice juicy NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W (202) 234 4433 WASHINGTON. D C. 20005 (202) 234-4433 s
57 r
1 results occasionally, I
think it would be very 2
interesting to all of us.
3 I'm wondering if you can give me a general l
impression.
It's clear that some of the problems 4
5 you're dealin.
with are unique to a power plant I
6 situation, whereas others are extremely general, like 7
the effects of lighting, the effects of noise, 8
sequence of computer commands and things like that.
9 Can you give me some qualitative idea of how much of 10 what you do can draw from research that's already out 11 there and be applied versus research that has to start f
12 from scratch for your particular application?
13 MR. COFFMAN:
Well, obviously, the first 14 step we always take is to try and assess what is out 15 there --
t 16 COMMISSIONER de PLANQUE:
Right.
f 17 MR.
COFFMAN:
-- so we don't reinvent i
i 18 anything.
I'd say in most cases, in the majority of 19 cases that we find information available out there, 20 but sometimes it has to be adjusted.
21 COMMISSIONER de PLANQUE:
Given the fact 22 that I'll follow-up on Commissioner Rogers' 23 question, I guess.
Given the fact that you often do 24
- this, comb the literature, it's also a
little 25 surprising to me that there's not more information NEAL R. GROSS COURT REPORTERS AND1RANSCRIBERS 1323 RHODE ISLAND AVENUE. N W (232) 234-4433 WASHINGTON D C 20005 (202) 2344433 l
58 1
going back in the opposite direction, that most of 2
what you're doing is coming from a user request rather 3
- than, "Oh, look what we discovered out there in the 4
literature and you folks ought to know about it."
5 MR. COFFMAN:
Well, I think it may not 6
have come out in the briefing, but I think what you'll 7
find is that there's a lot of interaction in the draft 8
products and results as they come in are shared with 9
the user offices and that's why we find ourselves 10
- ometimes in -- we're still finishing up the formal 11 documentation of the report when the method is already i
12 being used.
So, I think there is a lot of flow.
13 MR. RUSSELL:
Let me add one other thing.
14 That is I think as a result of the interactions, and 15 I'm speaking now to the NRR/Research interactions, 16 that there are a lot of times when you're not able to 17 point to which individual in the dialogue back and 18 forth identified the
- need, but once there's an l
19 agreement on our part that this is something that 20 needs to be done, we generally document that and 21 provide it to them in a user's request.
22 COMMISSIONER de PLANQUE:
So your user's 23 requests are easier to count than their ideas that 24 come to you.
Is that sort of what you're saying?
25 MR. RUSSELL:
Well, no.
I think part of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(?O2) 234 4433 WASHINGTON. D C 20005 (202) 2344433
I 59 l
l 1
it also maybe goes back in the past in that there was
{
2 a perception that was important to have the program 3
office endorsement of the activity.
So, what its 4
genesis was is less important than the fact that both 5
agree that this is something that needs to be done.
6 So, the fact that there are a lot of user requests i
7 from NRR doesn't mean that we're sitting over here and 8
thinking up all the research that needs to be done.
9 It's more a two way street and there is a standing f
10 frequent meeting back and forth where they talk about t
11 the research products, what's going on and many of the 12 people that are over there now used to be in NRR and 13 it works both ways.
14 So, I would characterize this as one area l
15 that has been working well between the program office 16 and Research.
So, I'm sure that they could point to i
17 sentences and things that are in NRR user requests --
18 COMMISSIONER de PIANQUE:
That sound 19 familiar.
20 MR. RUSSELL:
that were written by 21 folks from Research.
22 COMMISSIGNER de PLANQUE:
Okay.
Fine.
23 Thank you.
24 CHAIRMAN SELIN:
I have to admit to being 25 a little bit puzzled at the end of this discussion.
NEAL R. GROSS 4
COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(202) 2344433 WASH"4GTON, O C 20005 (202) 2344433
60 1
I agree with my colleagues' remarks, particularly 2
Commissioner Remick's remarks about on one hand the 3
utility of having users and researchers here, although I
4 according to Mr.
Russell you guys keep switching 5
places, so I'm not sure who is who.
6 On the other hand, the characterization of 7
the program I really find very confusing.
Sometimes 8
it sounds as if we have a budgety kind of -- oh, what 9
shall I call it.
It's not petty, but a cash fund.
10 We've got $6 million to answer users' requests and the 11 objective of the program is to do'what we can within 12 a given budget, which on the one hand is not a trivial 13 amount of money, on the other hand if we're able to 14 get some real insight into these very concrete 15 questions on the human factors, given the enormous 16 amount of work that goes into the engineering and the 17 maintenance, it's certainly a justifiable amount of 18 effort.
19 On the other hand, we talk about the 20 program, about long-term goals and the program is 21 years old.
We still don't have the long-tena goals l
22 and that makes it sound more like a self-starting 23 research program that has a number of objectives which 24 might be put out.
But there aren't many results that 25 are long-term results that are on the table. A lot of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 AHODE ISLAND AVENUE, N W.
(202) 234 4433 WASHINGTON, D C. 20005 (202) 234-4433
61 1
the discussion about the objective is to validate 2
methods.
It's basically internal, validate methods i
l 3
and establish a database, et cetera.
I wouldn't say 4
it's research for research sake, but it is research, 5
building up both methods and a database that could 6
then be applied afterwards.
7 So, I really don't know, I don't know 8
today, I didn't know when we had the meeting almost a 9
year ago, exactly what kind of a human factors 10 research program we have.
Obviously it's some 11 combination of these two, but it's still not clear to 12 me the top down approach.
A different kind of a 13 discussion that talks a little bit less about the 14 researchers speaking to research junkies and more from 15 a point of view, "Here are the objectives we're trying l
16 to carry out.
Some of it is customer satisfaction, i
17 some of it is internal.
Here's how we're putting the l
18 resources together.
Here's what we have found out.
I 19 Here are the issues,"
would eventually be very 20 helpful.
In particular, there are some of these 21 activities, particularly the database activities, that 22 have been going on for a very long time.
How do we 23 know when we're done? Maybe we're never done.
Maybe 24 the idea is that we're just continually investing in 25 a better database so we can gather the answers to the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
(202) 234 4433 WASHINGTON. D C. 20005 (202) 234-4433 1
\\
i 62 I
I 1
users and as long as the users are satisfied, if they 2
had to pay the bill themselves, then we have a good 3
program or maybe we have some concrete objectives.
4 l
But I have to admit that it's not that much clearer to 5
me now than it was two years ago what kind of program 6
we have, what drives it and how do we measure 7
satisfaction.
How do we know that we're doing a good 8
job? How do we know that we're doing a reasonable job 9
but could do better?
It's just not that clear.
10 Now, this is not a huge program, so I'm
?
11 not so much concerned about how we're spending one 12 percent of our budget, although it's a fair amount of i
13 money.
I am more concerned that everybody has 14 identified management and human factors as the huge i
15 uncharted area at least of reactor performance and now 16 with Mr. Combs here on the material side.
The real l
17 question is how much of a dent are we making this 18 area? Should we be doing more or less or are we doing l
l 19 the right thing by responding to the users' requests 20 or should we have more of a research-driven program?
l 21 At some point we really have to address those l
22 questions.
Or maybe you just have to explain to me 23 why it is clear to everybody else and it's not clear i
24 to me and then I'll go away happy.
But I still have l
l 25 sort of -- it's an hour after a Chinese meal.
It was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE N W (202) 2344433 WASHINGTON. D C 20005 (202) 234-4433
_, -,..~ -
63 1
very tasty, but I have this empty feeling in my -- not 2
stomach, but my mind at this point because I really 3
don't know what we have in front of us and it's not i
4 the highest priority.
5 MR. TAYLOR:
We'll take that challenge.
t 6
We'll take that.
7 CHAIRMAN SELIN:
Doctor Speis, did you 8
want to add something?
9 DOCTOR SPEIS:
No.
We'll take the 10 challenge.
11 I just want to add one point that you 12 mentioned earlier, tell me more about the PRA aspects 13 of human factors.
The only thing I would like to say, 14 that there are two aspects to a PRA.
One of them is 15 human errors in performing operations and doing tasks 16 and what errors could be made that could lead to an 17 event, and also during the event itself, what wrongful 18 interventions can take place that could lead you to
[
19 the wrong result.
20 In that area, the classic work that has 21 been around for a long time has been a handbook by 22 Swain, a cookbook as Bill mentioned earlier, and this j
23 has been based on Air Force data which was adopted to 24 some extent to nuclear operations.
So, one of the 1
25 programs -- in fact, the bulk of our effort has been i
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W WASHINGTON. D C. 20005 (202) 2344433 I
(202) 234 4433 i
4
*----*e~'
-~ '*-
- ~ - " ' ' * '
' * ~ " * ~ ' ' ~ ~ ~ ' " ' ' ~ * '~ ~ ' ' ' ' ' '
~
64 1
to improve on that handbook, to come up with human 2
errors that are more relevant to what's going on in 3
the nuclear industry basically.
4 CHAIRMAN SELIN:
But before you get off 5
that, I had a question last year --
6 DOCTOR SPEIS:
I was going to say one more l
7 thing about that.
i 8
CHAIRMAN SELIN:
Okay.
Sure.
i l
9 DOCTOR SPEIS:
The other thing was the 10 other aspect is the organizational factors, whether we 11 can point --
12 CHAIRMAN SELIN:
I want to talk to the i
13 first part because you --
14 DOCTOR SPEIS:
Go ahead.
All right.
1 l
15 CHAIRMAN SELIN:
And that is that I asked 16 you last year, I didn't really get an answer then, I 17 didn't get an answer now, what happens if we went 18 away?
Is the industry doing this work and are we i
19 doing are we just doing sort of regulatory j
20 confirmation or are we trying to do basic work that 21 you would have expected the operators to be doing? If 22 you can run a power plant, you're going to train 23 dozens of operators, you would think that you would 24 want the best factors yourself.
Why does this fall 25 upon us?
Why is there such a gap out there or is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE N W.
(202) 234 4433 WASHINGTON. D C 20005 (202) 2344433
. ~.
65 1
there a bigger program to which we're just doing the f
i 2
last ten percent?
j 3
DOCTOR SPEIS:
I think quite a bit of 4
improvements and understanchng has been gaineri to make S
this data more relevant to nuclear plant operations.
6 But even though that experience and that feedback goes 7
back to the plants, we still see errors and problems 8
coming up.
The objective, I guess, like in every I
9 other area, is to keep improving and seein? --
r 10 CHAIRMAN SELIN:
But I'm missing l
11 something.
Is there a major industry-funded research 12 program in this area and we're just trying to validate t
13 it or are we doing front line research that no one 14 else has done?
15 DOCTOR SPEIS:
I'm not so sure that there 16 is any coherent and concentrated effort on the 1: art of 17 the industry.
18 MR. RUSSELL:
I'm not aware of any.
19 DOCTOR SPEIS:
We're doing most of the 20 work in this area basically, yes.
'l 21 MR. RUSSELL:
In fact, because of the 22 concern about human error rates, and this came up --
23 we had some very interesting information presented to 24 us by the French regulatory authorities where they had 25 spent literally 100 staff years or better running NEAL R. GROSS i
COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W-(202) 2344433 WASHINGTON, D C 20305 (202) 2344433 3
---,n
--s
--r---
'~"
e
66
]
1 experiments for the advanced control room and then 2
running them on a hybrid control room and then on the 3
Bugey control room simulators and looked at the error 4
probabilities under normal conditions and under l
J 5
stress, and we found that these were significantly 6
different than the kinds of numbers that were coming 7
out of the handbooks that you would generate from the e
process of using either Alan Swain's methods or other 9
HRA methuds with the handbook data.
10 As a result of some of that uncertainty, 11 what we've done is we've essentially requested that i
12 they do sensitivity studies. as a part of the PRA 1
13 reviews that are being performed for the advancef 14 plant designs to try and look at the importance of the j
i 15 particular human actions, to see which ones are really a
16 important from a
risk perspective.
So, we're 17 essentially varying the error rate from I:ero to one to 18 ti,
get measures of the importance to overall risk 19
- r. f these tasks that have to be performed and then 20 we're looking at it from the standpoint of whether 22 that task should be automated or not to eliminate it i
22 and so we're using this as part cf task allocation and 23 that's the way we're using the tool because there is 24 a great debate over what you use for numbers and what 25 is the uncertainty when you're putting human error NEAL R. GROSS t
COURT REPORTERS AND TRANSCR:BERS 1323 RHODE ISLAND AVENUE. N W.
(202) 2? S O WASHINGTON. O C 20005 (20n 2344433
67 i
1 probabilities in.
2 So, because of this lack of good data, we
{
O find that often we have to look beyond that, do
)
1 4
sensitivity
- studies, look at other alternative l
5 approaches because you cannot put high reliance on 6
recovery actions or some of these other things.
7 CHAIRMAN SELIN:
Well, I'm not surprised
]
8 to hear that say the vendors come up with some 9
analyses and we have to do a lot of work to check 10 that, to look at some sensitivities, to explain that.
11 I am more surprised to hear that in terms of the i
12 operation of today's plants there isn't a lot more
{
13 work than there seems to be going on funded by the 14 industry itself to take a look at the ef fectiveness of 15 their own training methods.
I mean they spent a 16 fortune on the training and the operations that 17 result.
18 So, the question is is there more going on 19 than we know about, is there not going on? Have they 20 tried it and it just turns out to be very hard to if 21 ;;
invest money usefully?
22 DOCTOR SPEIS:
No.
We know that there 23 isn't that much work because, for example, we're 24 reviewing the IPEs now and the information that the 25 people are using are that deri w from this classic NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W.
(202) 234 4433 WASHINGTON. D C. 20005 (202) 236 4433
68 r
1 1
Swain handbook.
Okay?
2 CHAIRMAN SELIN:
I see.
3 DOCTOR SPEIS:
So, the reason I mentioned 4
it is because a sizeable part of our work is focused 5
in this area and trying to understand and improve 6
better on human errors and then translate them into 7
quantitative attributes.
I 8
CHAIRMAN SELIN:
I'm very glad you brought 9
that out.
Obviously the less confidence you have in 10 the supply of information, the more you have to be 11 sensitive to the sensitivity of the use, the way these 12 figures figure into the FRA.
But if the situation is l
13 as you describe it, I guess I'm sort of concerned that i
14 we have this rather large research vacuum out there 15 that we're trying to fill ourselves rather than also 36 encouraging the license community to take steps to 17 fill that on their own.
f 18 I did interrupt you, Doctor Speis.
You i
19 were talking about organizational factors also.
i 20 DOCTOR SPEIS:
Well, that's another area i
21 that there is noth.tng in PRAs right now as far as 22 quantifying the effectiveness or non-effectiveness of
\\
23 organizational factors.
That's where we discussed 24 today we spend a sizeable amount of money, for i
25 example, something like between $4 and $5 million the NEAL R. GROSS COURT REPORTERS AND TRANSCRIDERS i
1323 RHODE ISLAND AVENUE N W.
(202) 2344433 WASHINGTON, D C. 2000S (202) 234 4433 I
69 1
last three or _%/ years and we have basically reached
{
2 an impasse. That's the area that we're kind of taking 3
an step back and trying to decide where we go now 4
basically.
l 5
The point I was trying to make, that in 6
some of these areas the work was kind of exploratory.
7 It wasn't -- the answer wasn't obvious the moment we 8
started pursuing those areas.
So, unfortunately this 9
work has those attributes, the human factors work.
I 10 guess I'm not trying to justify everything, but those 11 things have to be taken into account.
12 MR.
RUSSELL:
I guess I
could just 13 illustrate how extreme the situation is.
At the time 14 we had our senior management meetings to review plant 15 performance, one of the facilities that ultimately 16 ended up identified as a
facility that needed 17 additional attention by the NRC has, if you believe 18 point estimates, the safest plant based upon their IPE 19 in the United States.
So, you have the two extremes 20 where the IPE is telling you one thing and yet on the i
21 other hand here's a facility that we're extremely 3
22 concerned about from the standpoint of management l
23 performance errors and other things.
So, that 24 situation is one that exists and has for some time.
25 We've seen that even back at the time of Zion, Indian NEAL R. GROSS COURT REPORTERS AND TRANSCRIBE R$
1323 RHODE ISLAND AVENUE, N W.
(202) 234-4433 WASHINGTON. O C 20005 (202) 234-4433 2
,,-r, "w w.
70 1
Point action plan and the concerns there where we were 2
doing the PRA reviews at Indian Point and looking at 3
the difference between the two units and there were s
i 4
very different performance between the two units and 5
yet you could not recognize that through the PRAs that 6
were being done.
7 CHAIRMAN SELIN:
They must have hired the 8
same people who did the RBMK PRAs.
9 Look, in addition to the point that I 10 first threw out, which is what's the motivation of the 11 program, I continue to be quite concerned about if 12 this stuff is so terrific why are we the only people 13 doing it, to put it in simple terms.
So, I would add 14 to Commissioner de Planque's consideration about the
~
4 t
15 general literature on human factors not specific to 16 nuclear plants, a concern about whether there is or if 17 there isn't, why isn't there more work being done and 18 sponsored not by other federal agencies but by the 19 industry on the human factors work as applied 20 specifically to nuclear power plants?
^
i 21 Maybe one of the alternatives is not so 22 much to try to do this all ourselves. Maybe it is the 23 most efficient way for us to do it and then in effect 24 charge this back out to the industry through our fee 25 structure with all its overhead.
But maybe a better NEAL R. GROSS i
COL;AT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE, N W I
(202; 2344433 WASHINGTON, D CJD005 (202) 2344433
71 l
l 1
1 way would be for the industry to take on some of these 2
questions themselves directly and see if they can 3
satisfy us with their results as well as our doing the 4
work.
5 Fine.
Thank you very much, Mr. Taylor.
6 (Whereupon, at 3:38 p.m.,
the above-7 entitled matter was concluded.)
8 9
1 10 11 12 13 l
14 1
15 16 17 18 19 20 21 22 23
^
24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVENUE. N W.
(202) 2344433 WASHINGTON. O C 2000!.
(202) 2344433 I
4-m-
,-m._u.
-=.m.-aa,-s_.,,m
.-a.
a,.4sw_-
---m-.-..a-%,,.-m_..m.
s%.
m-__u--
p..
.=
..wi...wa..wa4; ua._a.,
suuvu,,#,.m_.,.ww-.n.
h I
I l
I l
l l
I k'
l I
' I
' i i
1 t
1
(
l 5
i l.
- I 5
r I
i
- t i
I l
1 l
l l
l f
i l
h 1
I i
s l
4 I
i
t CERTIFICATE OF TRANSCRIBER This is to certify that the attached events of a meeting of the United States Nuclear Regulatory Commission entitled:
TITLE OF MEETING: BRIEFING ON NRC RESEARCH PROGRAMS OH HUMAN FACTORS PLACE OF MEETING: ROCKVILLE, MARYLAND DATE OF MEETING:
11-10-93 were transcribed by me. I further certify that said transcript' ion is accurate and complete. to the best of my ability, and that the transcript is a true and accurate record of the foregoing events.
v Reporter's name: PETER LYNCH l
s t
I I
e NEAL R. GROSS cover amoetens me veAMscenens 1333 AMOSE 85 LAND AVEMut. M.W.
(202) m wAgeeMSM N,DA '20005 (202) 232 6
l COMMISSION BRIEFING ON HUMAN FACTORS REGULATORY RESEARCH PROGRAM NOVEMBER 10,1993
CONTENT BACKGROUND CURRENT RESEARCH ISSUES, APPROACH, PRODUCTS, AND PLANS
- Personnel Performance e Human-System Interface
- Organizational Factors e HRA/PRA Methods & Applications
- Performance of Materials Licensees FUTURE OUTLOOK 1
i
- (
_.---,__3
+
- +.
BACKGROUND HFB formed in 1987.
Overall Objectives of HFB:
- Develop technical basis for regulatory requirements / guidance in areas related to human performance
- Accurately measure human performance
- Provide staff expertise on human performance in commercial nuclear activities Human factors research driven by regulatory needs. Have received 100 regulatory need requests of which 42 are currently active.
AEOD 2 active requests NMSS 4 active requests NRR 36 active requests 3
't BACKGROUND Continued Regulatory needs change with research results and regulatory e
circumstances Current research program consists of 51 projects directed by HFB/RES e
(10 Project Managers) and involves 26 contractors (Gov't agencies, Nat'l Labs, private firms, universities) and international organizations Maintain interactions on human factors subjects:
e
- formal (2 foreign countries plus domestic industry and government organizations)
- informal (5 foreign countries plus domestic industry and academic organizations) i Currently most of the active regulatory needs are being addressed in e
i accordance with priorities from user offices I
1 i
4 4
___..-.,__-,e
-.m u--
u
. m
.m m
m
HUMAN FACTORS RESEARCH FUNDING Topic FY 1994
- Personnel Performance
$ 1,037K
- Human-System Interfaces 2,919K
- Organizational Factors 308K
- HRA/PRA Methods & Applications 1,258K
- Performance of Materials Licensees 976K l
Total
$6,498K i
l 4
i
1 PERSONNEL PERFORMANCE Issues i
Over 50% of reportable events involve some form of human error:
i Need for a standard inspection method to determine root causes of events involving human error Need to characterize predominant areas of human error Adequacy of plant staffing to handle significant events Effects of plant environment on' human performance Fatigue effects of shift length and overtime L
6 2
4
wn---
+. - - - - -~.
,w--
....-w r.
. ~ *.
--2
-4
PERSONNEL PERFORMANCE Continued e
Research Program Involves learning from experience both in and outside the nuclear industry and studies of human performance Will broaden staff's knowledge in areas related to human performance Recent Product l
Human Performance Investigation Process; NUREG/CR-5455 (being
.used by inspectors during event inspections) 7 l
PERSONNEL PERFORMANCE Continued Plans Study of effects of shift duration and overtime on human performance; 2Q/FY94 Handbook on effects of environment on human performance; 2Q/FY94 Study of effects of hi-intensity lighting on Ops Center personnel; 4Q/FY94 i
Reports on basis for minimum staffing levels for current and i
advanced designs; 4Q/FY95 s
8 3
I a
I
(
m-r.
~_ _ _ _ _ _ _ _, _ _ _ _ _ _ _ _ _ _ _ _. _ _. _ _.
HUMAN-SYSTEM INTERFACE l
t e
issue Digital control / display systems are being developed for use in current and advanced plants What should be the technical basis for regulatory positions on the use of digital control / display systems in safety-critical functions What are the effects on operator workload / performance Research program is directed toward the development of standards and guidelines for both software development and interface design, and considers existing standards / guidelines and experience (nuclear and i
non-nuclear) i l
9 i
i I
t HUMAN-SYSTEM INTERFACE Continued Recent Products Draft guidelines for human engineering reviews of advanced control rooms (Draft NUREG/CR-5908)
Report on lessons learned from test and evaluation experience on computer-based systems at Halden; HWR-336 Workshop on digital systems reliability and nuclear safety 1
Report on review of current standards for development of safety-critical software; NUREG/CR-5930 4
I
)
{
10 i
-. ~.....
HUMAN-SYSTEM INTERFACE Continued Report on evaluation of conventional software verification and validation techniques; NUREG/CR-6018 Resolution of human factors generic issues on annunciators, local control stations, and procedures; NUREG/CR-5458 and 5572 Graphic display software developed at Halden is being used at Technical Training Center to create displays of simulation data i
4 1
4 11
,- ~,,,,.
r----
,.e,.
---m--c-r.--wm,,
..,,s.
HUMAN-SYSTEM INTERFACE Continued -
e Plans l
Technical basis for digital systems in safety-critical functions Develop technical basis for requirements on software error l
analysis; 2O/FY94 Develop basis and guidelines for Verification and Validation of Expert Systems, 3Q/FY94 i
Report on lessons learned on verification and validation during i
j software development at Halden; 4Q/FY94 i
Develop software audit tool prototype for use by NRC; 1Q/FY95 I
j Report on safety attributes of programming languages; 40/FY95 t.
l I
I l
12 I
h l
HUMAN-SYSTEM INTERFACE Continued Effects on operator workload / performance Complete guidelines for human engineering reviews of advanced control rooms; 1Q/FY94 Reports on effects of computerized procedures on human l
performance; 3Q/FY94 Assess effects of digital systems on operator workload; l
3Q/FY95 Report on lessons learned on man-machine interfaces with computer-based systems at Halden; 4Q/FY95 i
i i
I 13
ORGANIZATIONAL FACTORS lssue involves the feasibility of (1) measures and criteria to consistently evaluate nuclear power plant organizational performance and (2) methods to translate organizational performance into risk Research Program reviewed work of others and focused on identifying organizational factors important to safety and their impact on risk e
Products Identified organizational factors and developed methods to rate their relative importance. Tried at two plants; NUREG/CR-5538 Preliminary attempt at developing a methodology to quantify risk Evaluation of research program; SECY 93-020 l
14
ORGANIZATIONAL FACTORS Continued Plans Monitor work of others in this area Develop training for incorporating organizational factors into diagnostic evaluations RES/AEOD/NRR are evaluating the feasibility and practicality of further research.
Key questions:
Validation Resources required for application Recommendations to senior management this calendar year 15
i----
HUMAN RELIABILITY ANALYSIS /
PROBABILISTIC RISK ASSESSMENT METHODS AND APPLICATIONS issues include the need for methods to evaluate Tech Specs from a risk e
perspective and for means to improve and validate human reliability estimates Research Program Focused on developing methods to evaluate Technicai Specifications using risk assessment in the areas of surveillance test intervals, i
dependent failures, configuration of systems, and low-i power / shutdown operations Evaluating factors important to errors of commission.
Inferring error rates from data available from requalification examinations 16
HUMAN RELIABILITY ANALYSIS /
PROBABILISTIC RISK ASSESSMENT METHODS AND APPLICATIONS Continued e
Recent Products Methods to improve Tech Specs using risked based evaluations:
(NUREG/CR-5425), allowed outage times (NUREG/CR-5775), surveillance test intervals (NUREG/CR-5993), dependent failures (NUREG/CR-5641), configuration management Checklist for evaluating conditions that could lead to human error in i
cognitively demanding events; NUREG/CP-0126 A computerized library of error probabilities; NUREG/CR-4639 17
-w -
HUMAN RELIABILITY ANALYSIS AND PROBABILISTIC RISK ASSESSMENT Continued Plans Report on risk perspective of Tech Specs requiring shutdown; 2Q/FY94 Report on risk impact of diesel generator maintenance strategy; 3Q/FY94 Handbook of methods for evaluating Tech Specs; 4Q/FY94 Analysis of operator requalification data for error rates; 4Q/FY94 l
18
l PERFORMANCE OF MATERIALS LICENSEES Issues relate to identifying actual and potential human errors leading to medical misadministrations and unnecessary exposures associated with industrial radiography Research program involves studying the functions and tasks performed e
during remote afterloading brachytherapy, manual brachytherapy, teletherapy, and industrial radiography Recent Product Draft reports on human performance in teletherapy and remote afterloading brachytherapy 1
1 19
)
PERFORMANCE OF MATERIALS LICENSEES Continued Plans Reports on potential errors and preventive actions on remote afterloading brachytherapy and teletherapy; 2Q/FY94 Report on potential errors and preventive actions on manual brachytherapy; 3Q/FY95 (Pending confirmation of continuing user need) l Develop human error inspection methods for materials licensees; l
4Q/FY95 l
l l
Reconsidering the need for further research on industrial radiography I
in light of experience with the rule change to 10CFR 34 l
l 20
FUTURE OUTLOOK Five year plan projects stable budget at approximately $6 million per e
year User need requests will continue to dominate research program in FY94-96 time frame Beyond FY96 some user need requests are still expected Development of long term goals e
Identify long term NRC human factors needs Technical issues Staff and contractor expertise Facilities l
1 Human reliability data base Assess level of involvement in standards and international programs 21