ML20199E677
| ML20199E677 | |
| Person / Time | |
|---|---|
| Issue date: | 01/21/1998 |
| From: | Advisory Committee on Reactor Safeguards |
| To: | |
| References | |
| ACRS-T-3021, NUDOCS 9802020227 | |
| Download: ML20199E677 (428) | |
Text
.- . ,e
. ., .- .. ." - -]. ;-, p.,-] - / l.,.., ,.,f,.
.' - ' , . + q . .
n
. '*- ; . , , .. , ..,f M .$8@
- ,g... . f . .,
2 -
., '," e-g
- .g
- ,' . n L
- s. .3 .-
- . . + .a. ,..:,..,p..
. . . . . n g.
.m% ~r; q 3g. u,..,.
. . e. . - . .
., _ , 4 ,y r ,
4$ .y ~. ,
= - . '. g : - e 7 h , '.
.N Tp' .' .%. u; h,.
, p, .
, . ( ,' , .. '
.- ~ ,'&
he, f;% -
. '* ,Vc. [.'.... l' '[-: . Q,4.*;
.( _ .
I 9:~ .
g.
w
~..
' l- '
l
.[X'.l'[., r . . . ' l}l} 7 * , y'; [ ,. . u .. ., f. p
' j . s" ,[
,. [ ..
.. : wy ,. h. .,. s
,. , - .e- - . . .. . .. c. ., ,..
. . . .. ;a -
.4 i.
(.1 N ,/,. y , ,/, g -[ ' ' - '
~
R( $..D.. ' , . - ' [ .
. ,' fol ge',A f. 5 ,;
'v, e
, g ,
g ,
.;,,' ,* f 3 ,- (... . ; i
- 8 -
n-Q. . ..-_ {. . ** j - '
. . L 'C g_>
e
- . . .. .~
. . a Y / . ,l }}c.l. q.,.Q,-_.i[
4'l'[.-U
.' h ,9
[ .[. .5
. g . '. . ;l. ' -$ ty. . :
- f,g* \. [?M
- . . s . -
4
.- . v . . . : . -
.i
(- .
....o v.- .
. .' . ; .; t .u n .
4 J, __ 4 s s ,
; ^> -
*y - ,' '
s-
.i .. .r .* ,. .' . 2g.9. ;_...
e ' I [ ' ; 5,e f. hJ, f *. . .,
.,k.,,,
. .. . . , . ., .n.
.), . - -
9
,.. . ,~ ' , y.s. ,,s .;
- u. , u
.h .
* '?. #
.Q 2
",, . *e - ' ;I,.
$ ('
h '. ..'. ." ' A .' - - >
' t.[ K -
s - - ', - ' , (; i
}... ' .c% *' '[, g a.
.T..'[ 1:..
b
. s.. .
.t p \ .- e -*
'h rg.
.{7 ..
.- . , .c - - -
- .- ^ - - c'. .' 3. :. ' .;' ' % ,3 . ' ,.
' ' . , . h'
.- 2 # ,, . . T b_, }
.I g- . ., ' ; I
.f.},
b.P . , i. ( , . , yr - Y . y- . ; f' ,,= ,4 (. - ,~
-'W A ',.., .T' ,'
' . O,
. . .,. , t .
. .; ., , e ,** .
, g - 4 *. ,. - . , .
. .( .
- e. c, J.,, *
}j Q.,7.,. 0 - .f
.. .'f '
~
$ . , 7 ..; .
- U s =
. - e
, . . . . . . _ , . . ,m e, ', f , Qs
- l/ . .-
-'_ ; ~
o . - .
-l .
.fs t < - ..*. Y . }- . > ; . - -*..
, '/ '. .::, .p h.;". .y g l.'.. ,
, ~ , , " .. . . . i l$ -'Y .. '.' .y'. , ug. ; .' .. %
h
'; . : i ;. '- -'.'
~
,, y _ ,
l s. '& y .
.L.., .
;. . eV. w Q..Y % ,f , lS \ '. p- Y; v:s '?:-'1'
- x. ': 'c. *. n , *. .Q. . . .
. , . .. .' 's' e ~
($ ;2 a ." :' I' . %.; m ;e L
,. ..'; %:;,Y h '
'. *Q. -
.. 4 n'. .'.' J. .
+ g.,. - +
- i p"- '
; 3 .A *ty: i .-
{, ; ', - fQ a ,;
." h.' - M.:w*-
I>+ , 'o #
.r p 1,*
.-s,.
..,.,j - , , . . .
' 3' ; , ,
f ,':..
' ^ -
;%% 2 - .
. . , ,, :: : .v ;y ,
,G . '
h.g\.. I
'i m.
,J I , . < Q,g
. .* , .s. p
%y- . .
- .c i, ,;. - e...^
g a r t. s.7
; . e a. ..
f ,'. . .l ; . , . ..NW 1,,' f. .4 . . . s .
, , 3 . ;
q ' #.../.., =; T
,. , .a
($ 4
,c.1 a :- .? .i',
j -)%..:q ; . - s. g, .:.elh
; = - , . ; w . .> . a 4.fy o .
- ' e ; .
A. .. 4 '.t*. ..w *. D? h8
.l. w G . .~j.';.J f.? % ll f l1I.j'fl,}Q:.i.:?
.j '. ^
V' 4 - h: f .)
'[ .
- . ;,9 , z . ., ; ; .$
s . y ,t sy 6 .- -
- s. . - - .
-). ,..,u .. , . . ,
v.-
, -_ . .. ~. ' , i . : a n, ,. ,c.. .. . .., ,. _ , , _ . :: ..L:h.r .
.4, -- p ._y y g
, g , . .,...;^._., - 'T .*..,
.4 ;
9.
, , n;' , .
ag . % . .. ,e -s',.-
- y. -
, . . i . < .
p . .. .. s 4, . . .,
.p r 3 ,' , .s .,:.-
. . .4-. . . w
...F, w:,'c
-L, .. i) 'g". ,, ,' . ;'; .;, g~3 y
,,q.
,j .- .-
. y
, . ar u d.
,; .1 5
' . - . . . . . ' . , , , - : . .* ;9 ,.j. $ .;. , s: i ' ' .. . .. '*.j,% %
[ N. A'-
. on ,-
f , , ' , , ,,- , .. .
.. x.
.. .. w.
g...- , ,.....s.....
. . s . ;-
. :. .,.. - e : + 0. . .
( (c.,.4-
+, .
;g . ; 3 3.x ... . 9,. .. c.;.. q., a%a mp, , . ,. . .. .
.a g...,.. ..,; ;,, .., .
,.. ;. ., y, 7 j g ., .j j ; .:.,. , p,
- ..- . -..yy , . .
g .. . . . __ L.
. ,u . . .. ., .g. : . .. : ..pe / :. . e i < ,:>y l 3;: . : cf.
n .> ; .,: .u , a.;. m , y _ y. h q -
;$, d.. C *1.n .
,.;._, , . . . .. . " .. . , . . . w .. .' p m... ..<.n..;.. ,v,r.;w-
<; . ~ ' , . .g. . :~ . '
-g ::r.. . ..y.Q g y .
w, 5 e,'..
,. d . . .. :
.s. ; .
.% & c.
; mJ. k. ;...;.. . ..y ;ys pg:;; .u g
, . . s ..
O , s, .. -
.<, . .w ..w , , . ,
?.
..x.- n. n p .. . . . . .
+ .
. . , . .- f.. g , .,;. ; .. : ,.: p.s
. - s - - . ,-
. . . , c . . , . .
,y n,.- , . ; g g . j e.
- a. .
*...- ,.w - -
f-
. ...,*; ".. .f. .(.- -
;...*'e-.
~.1 ; ;
s
......v. . 7 4 gy
- w. r , .
.$ .,s ,, ctsQ - p ' . . ,,
,4
;*u' f<p. . . n -
r .. ;. _ -[, , .
- l.*:, y'_ .
, \, : y y . ( f 'g g a. -
,i*
,m.. ...3..,;.... p,%;:.:n 4gg.
.- i-1 . s.. 4< . ..u p.p ,3 l; g.y.g..y, .,
.t
- y. ; _ .;,.q3,7 ,
.. pg e , . - ... . ;... . ::,, .y.
.R .e p ;. , ,- . . : j z :,.
.' _.: , T.cga, ;q - 1, ..Q ; ?p:^ :.
3:.jf,zf m?
" s lu -
3 -
*y U. ,', . . 1f*".R d .' 'h jk
^
e _ ,_ . : s_
- ^** , l. [ 7 :
. F. * . .
,. . ?
pn p;g%g VNM %a.y$., t- x.%h. s. g . ,,;. yL4:..I M..,W d q.[.B 4.q Q;g
~. ~ ./ : . , .s . -
so : ; 3,. : v.m p lc dL.. : ? . . gf y 1 1.( .
..L- 3 < : 1g. . b> y ; . . .g,y.g .
3 .u s j t 3.3 b .
,? '
l g,
. . ' . .. . _ . l ' _ .y
.q ; , ~. .Qup(,,:, j ;:l
. a., , ,
s
.% ..., c s. - -
._y, 1.
-s.
." ^ - ' '.; ',p.M
'L
-4 . . - .. ; . .
. l. - %te
. * - m. .4
.- .. ^ ...',,.
., , y gA 5
M. s
, -' ~
9 -
'-[ .p ,.' ' ,
- - % .- r R
, , _. , ,_ o_
., , 7 . . ; R. . .m,* .
. . . . _ (,' Q? . s.
.* ;s- f, .
W.
- _. , k N . ' l. Y.s.' . ~ f.'.Y $ h ?Y. ',
- 6. .. c. , .- v . .,q. g:x. ., 3 . ,
s, f -3
*J.
a . .' .' ;...: . ,. . , ' " .. >.y.,,,, f' . .. g.,c... %. ,
.- - . . . . . . . . . . . ..? m: .9 , t.'" '
C ' :Ni.. *? N ' - - . -+ - -
; ' ,%5 g .
;. . f- .
, : . k- .
; ,' j. [ - .
. .{ .. ,. .
( , ./- .{'i. . . i ..
- - - - -= - - - - - - - - - ,
h0kh,f$h$m a m _-n , mm m m, p: h?m f,'
- i
}&&
- emm[s _ n wn8 YW[ , 4-Q Q4' '
M g,,x[i
@( , ' W ' , 4/:t
, +
j yg y r o *p' a d ,wy-w" k ' *a , ' u
. y i..
. .,j ,/. yF% ' l-t Y ," % '
1y h[ .b ,@m$rM3@/CQ m m - 3M_$DFFiCIAthRANSCRIPT : OF: PROCEEDINGS . w a s a a~ wp; ey y- o%,u,q y" ul 4 9agu s<n p.a .:nt o y-
~en# p~' ;" ;,,, . <
,t i
7 + tc t t e 9 c , w.e w y;;3 y% e G . f( A ,bfth.f $U $m , w, k} g * , , f Q ,l) _y+ ?
., c y , ' bl
~* >>w q4 W m : m_ M % ,
?. ,. .>' . , J. s ,
N[:wGQW m, spv " n .,,% M W,N_U,CLEAz.--
;sv %> -
REREG. ULATORYtCOMMISSIONz -
,'y+. ] y.; $f'%7,. N .O D'M
%4 < -
N g4 - r' em, t a.km- > ?, 4 t s i
; l' ,
3 h[k.bhAliNISDRhCObMI$YEE'ONLRE wm@ QV CN ,:%ey7m%lm%M W :M
-y u QV d<!
s m v R., CTOR'S1FEGUAR'DS6 . um;w m - - < 1.p g - m R& g 7 .y. a
%M Q.p vw.g q3 ft e(p*y m.r',i r QmM;&
t, s < s .; f. 2 i, ,i^ eM QU Ml/
- x. r ,
N i, s
- W, w'^ - m y 4
< >2)
, 6 i s> vy Nhhh, m r w m_ , d Tbb kh, hO[w :E . . ".y#*b'.
~
e e omwo jp y w&g.. sm; a m, w r wte* s
. t %y e ..
~5+
((h - nyg O r JN L g -a JU% vu "'e'
- m. ,
w-s 1, j
+ wn L ' ^
sWM e% @DS*4, s,JTitlet ' $HUMANiFACTORS ,
.9 . \ / - '
~~
,t >
SUBCOMMITTEEi ry
.&.ThN,D. $ W-$0lf n?w' ^~
.m ubw,4,sv4 s
- w: kh i .
4..
- +
~s cf L.
> :, r N+' y p4Wh
.. _ hm - m fM_ ,%R , P,'m, * {MEETINGL m m m - >
y/q%mwu K f,L ' !Q'W ;gpu , %
- NO y,
s Qi agMQM '1 M(W' JS s , &i * " i 3
-4 Mw ,
w& %, ;_ A,. W,n. a- s m
'.7 4 C, .
n g- ,
"< s;m.xm rp sX E p g %w amm.v; ~m, - n . ,
g , w+ "
@m. ,
'x- w- -
- m. ,
%. n ,
2-
-...am' w. m
%qChg cd t
um
. # vwm,akM s, DDockethoM ; N
'Fh bfu ps4 (MJ . $ h Q.eg4@
-4 -
q % 4p , @ xs~m'
,-c .f *
%j Jn g%np1mwm o ,
w -
,&o%$gD gf W e %r e'r"N g ,
[,{
~m N'
Am .. .. %. ' N L' T '. T RW. . b. , o e W' J.. . WQ ?* JA'q*?* f [ 4.'x :![
. %- m_.
- ?M k 5 5;". . f" JU * * * ~
4hM, ,V.@',@'EOrdeWNoM (AlSBh3002112: 1 3
$m@ww Worl
, ung% w m- , i s . < > >>
wl$Df%_ MN W *
~ %% % Ll .c ,
i q['g %pr mw d @@s*Y l &"s h hs' t y+ x
+
i ?tph - A qv 1 76 ,, 1 4 , - ' t g"" i a-4 [M'NM M2[* , 5
- b c- ..a o s t
q tWs P . ( ;+ . > ,~ yT ' t
/m%g wy r $w x ~y~
j %M s 'a. gv" ",x n , ' < y
.s- , 3 7 c y a}n de n: ,8 +
[ e c j
, e 4 cn. ,s -
f hyhg. -
, t VpT ,._ "
-% L - ' q.
w,whmpp w a> we ;Ml?*;4 t w g:,r m%.S ~% - w,;m - a& ' LQ"w< s S: ,'=
' ' 'w. ^
u-pn egg)nm wmc stw s
,m ; w .. -
- , w.
N a m. Qg
,> n
* ~ * , '
+
s p p,$ $w w%w' ' wp o$ , 1v , a , h - c
% e:k:ck ,z* C M G;>% ya -x. - y : . . c. ~- :
c : C&vg w e m3 LOCATIONit Sc Rockville;M_aryland]
- q w _ ,
w} d- tbM di.. . b'rM:telmW e@ % s; d t M[ tFYf Y
% aw . y 2 ?M v -
ygd s TQ -.2 t. ,
' 5 e g M M M Q t e%d % )R $,3 ' >
"_. ' :l~
yli
,s
- Wednesday; January 21s1997 PAGES:1 - 322 -
@mA Mu Qnp+mn$MMDATEi(n
%w <
c q m,
>nn n; , ,
- w. p
& &w a&Q s W ? 4-
. -. . . ~w , e R$
m , @n Mt MM v n W > . L "s M ' n,seLMthe
' ANN RILEY & ASSOCIATES, LTD.
Codee
& .?
- 1250 I Street, NW, Suite 303
( 3 kI wwA hh 7 , N@Ah . Washington, D.C. 20005 4.s a af -(202) 842-0034 w wm +:w m x W
,p ,q.Qf4 h > gr Si e + < ' ' ,
J DgJ . 4802020227 980121 FDP ACRS
%@h@M%@@lwm@@m-l#N pfg y+i@@S@
xp4 y%. 4 j igg 79 , % q ym
- ~) 0 P 1 PD4 a h f w@
4 , '" _
'(
DISCLAIMER UNITED STATES NUCLEAR REGULATORY COMMISSION'S ADVISORY COMMITTEE ON REACTOR SAFEGUARDS JANUARY 21, 1998 The contents of this transcript of'the proceeding of the United States Nuclear Pegulatory Commission Advisory (h
's_,/ Committee on Reactor Safeguards, taken on January 21, 1998, as reported herein, is a record of the discussions recorded at the r.;eeting held on the above date.
This transcript had not been reviewed, corrected and edited and it may contain inaccuracies. i D J l
1 1 UNITED STATES NUCLEAR REGULATORY COMMISSION t
\ 2 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS N_)
3 *** 4 5 HUMAN FACTORS SUBCOMMITTEE MEETING 6 7 U.S. Nuclear Regulatory Commission 8 Two White Flint North 9 11545 Rockville Pike 10 Room T2-B3 11 Rockville, Maryland 20852-2738 12 13 Wednesday, January 21, 1998 14 [' (,,,,)i 15 The Subcommittee met pursuant to notice at 8:30 16 a.m. 17 18 MEMBERS PRESENT: 19 GEORGE APOSTOLAKIS, Chairman, ACRS 20 THOMAS S. KRESS, Member, ACRS 21 DONALD W. MILLER, Member, ACRS 22 DANA A. PONERS, P 'er, ACRS o 23 ROBERT L. SEALE, t.ainnan, ACRS 24 25 l 1 , I \ ANN RILEY & ASSOCIATES, LTD.
'~- Court Reporters 1250 I Street, N.W., Suite 300 l Washington, D.C. 20005 l (202) 842-0034 '
2-1 PROCEEDINGS , () 2 3 [8:301a.m] CHAIRMAN APOSTOLAKIS: The meeting will now come 4 to order. This is a meeting of the ACRS Subcommittee on 5 Human Factors. I am George Apostolakis, -Chairman of the 6 Subcommittee. 7 The ACRS members in attendance are Tom Kress, Don 8 Miller,-Dana Powers and Robert Seale. 9 We also have in attendance Marty Stutzke from
.10 SAIC, and we expect two more, Professor Mosleh from the 11 University of Maryland and J. Carroll, former ACRS member.
12 The purpose of this meeting is to hold discussions
-13 with representatives of the NRC staf f and : invited experts, 14' to gather information concerning the status of the Human b)
(, 15' Performance and Reliability Implementation-Plan,'the status- l 16 of ATHEANA, a technique for human error analysis, and a 17-pilot demonstration, integration of human factors 18 considerations into the inspection process, and uses of the 19 Human Performance Events Database. R20 The Subcommittee will gather information, analyze 21' relevant issues and facts, and formulate proposed positions and actions as appropriate for deliberation by the full 22 23 -Committee. 24 Noel Dudley is the Cognizant ACRS Staff Engineer 25 for this meeting. b \' ANN-RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 200 Washington, D.C. 20005 (202) 842-0034 1
3 1 The rules for participation in today's meeting h 2 have been announced as part of the notice of the meeting g 3 previously published in the Federal Register on December 24, 4 1997. 5 A transcript of the meeting is being kept and will 6 be made available, as stated in the Federal Register notice. 7 It is requested that the speakers first identify themselves 8 and speak with sufficient clarity and volume so that they 9 can be easily heard. 10 We have received no written comments or requests 11 for time to make oral statements from members of the public
- 12 and just a few words about our invited expert who is here
- 13 now, Mr. Stutzke is a Senior Risk and Reliability Engineer 14 with SAIC. In a previous lifetime, he trained operators in 15 the U.S. Navy and he has worked on more than 20 PRAs, 16 including the Human Reliability Analysis -- and Mr. Carroll 17 just joined us.
18 MR. CARROLL: They wouldn't let me in. 19 (Laughter.) 20 CHAIRMAN APOSTOLAKIS: Okay. So we will proceed 21 now with the meeting, and I call upon Dr. Persensky of 22 Research, of the Office of Research, to begin. 23 DR. PERSENSKY: Good morning. Thank you. 24 Good morning, and I am here to give you an 25 in-progress or status briefing on the Human Performance and h ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
+
4 1 Human Reliability Plan. I am Jay Persensky, I am from the [\
\-
2 Office of Research. 3 I am currently the assistant to the Branch Chief 4 for the Control Instrumentation and Human Factors Branch. 5 Since I talked to you last, last time I was the Acting F Branch Chief, but because of some timing issues, we now have 7 a new Acting Branch Chief who is Tom ~ 1, who is also the 8 Deputy Division Director. I tried to get to him to make the 9 talk, give the talk, but he said no. 10 (Laughter.) 11 DR. PERSENSKY: I -- you know, I figure what the 12 heck. 13 DR. POWERS: Now, we know why he is the -- 14 DR. PERSENSKY: Right. He knows how to delegate. 15 I was never very good at that, I guess, is the problem. 16 We also have Wayne Hodges, our Division Director, 17 and Denny Ross, our -- let's see --;Denny, your title now is 18 Special Assistant to the-Office Director? 19 MR. ROSS: I share the same human factors problem , 0 that you-do. But the answer is yes. 21 DR. PERSENSKY: Yes. Some. We also have 22 representatives from AEOD and NRR. I don't see anybody from 23 NMSS yet. 24 All of us have been involved, as you know, in this 25 plan over the years, as it goes on. There have been a few
;- ANN RILEY & ASSOCIATES, LTD.
(/ A- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
5 1 other minor personnel changes to people who have been () 2 detailed to the branch, returned back to their own branch, , 3 so we have a reduction to some extent. And in NRR, there's 4
---I guess it is beyond rumor, the Human Factors Branch is going to be merged with the Operator Licensing Branch, S so
-6 bringing some groups together over there.-
7 I would like to thank Isabel Shoenfeld and Jerry 4 8 Watell of the RES staff who bsve been serving _on a working 1 _9 group to bring this stuff together, as well as Claire [ 10 Goodwin from NRR. 11 A little bit of background. Going back some time,
- 12 we started with the Human Performance Program Plan, which 13- you reviewed back, I guess the last time in late 1996.
14 Indicated a-number of problems with that plan to management. 15 So at that time, or after that time, the: responsibility of i 16 the plan was transferred to Research. We also added to it 3 17 Human Reliability, as opposed to just the Human Performance 18 aspects. ~ 19 We have had a number of briefings on -- [20 in-progress briefings on this. I know one in June, and the
- 21 last one was in October. At that time you sent a letter
- 22 dated October 8th, indicated that there was a need for high 23 --
still a need for a high level model. We have been trying
- 24. to use ATHEANA, which we were having trouble using as-the 25- model for this particular plan. And the other idea was that ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300
- Washington, D.C. 20005 (202) 842-0034 t
_.m_.- _ . _ _ . _ _ _ _ _ _ . _ _ _ _ _ _ _ _ _ . . _ . _ __ _ . _ ._ _ __ ___ E 6 1 the model should be based on insights from operational l (O} 2 experience. 3 In addition, there have been a. couple of other 4- things that have happened in this time frame. Things, or 5 actually documents have come out that have become sort of a 6 - guiding light: for us in the planning arena. The NRC's 7 strategic plan was published in September, and along with j 8 that with e.he NRC performance plan,--which is based on the 9- operating plans.for each of the offices, each of the_ groups 10_ within the office, and that is part of the GIPA--- ! 11- Government Improvement-something Act. But in terms of the J- . 12 -way-the agency is-going to be rated in accordance with this 13 performance plan, i 14 Status. Where are we? Well, we sent you a draft 11 5 _ version of a plan back in August. It was the plant hat had 16 gone up;to-the_EDO in June -- end of June,-earlyl July. We 17 have not_ written a new plan in toto since then. We have 4 18 been working primarily_in the_ areas of trying to associate 19' this -- a model to bring it together into some' sort of high 20 - level model. 21 As-I indicated, one of the other things chat-has i 22 come about is the strategic plan as our guiding light within l 23 the agency for how we are going to be doing business. So we 24 have been.looking at how we might structure the plan in 25 - accordance with that~ strategic plan, so we don't have a 1 i L l[ ANN RILEY & ASSOCIATES, LTD.
\-- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 4
(202) 842-0034
,- .-.m , , . . ~ _ _ , _ - ,
7
-1 bunch of different ideas floating around.
( )~ 2 We have also had a number of discussions on how we , j 3 mightago about prioritizing the activities within the plan.
- 4 In October I was talking about a scheme which would be an 5- absolute ratings scheme where ---ranking scheme where each l
- ~6 one of the activities would be-rated against a number of l
7- criteria. Then you would do some combining of that n 8 information and then rank the activities across the agency. ,; 9E There has also been proposed within Research a f- 10 comparison technique. called the analytical hierarchy i
-11 process, which we-are also looking into as a potential way 12 of dealing with this plan.
i 13 CHAIRMAN APOSTOLAKIS: These peer-wise comparisons 14 will be done by some key. people in the agency, because:-- 0
~ \ ,s/ '15- and then what? AndLthey take:the average-or--- or you-16- haven't1 thought about it yet?
17 DR. PERSENSKY: We have been looking into the 18 method. The idea is-that-key players, whether it is, you i- 19 know, very high in_the agency or at a lower level, probably
~
- 20 at-some level the branch chiefs, because you have to have l 21 some-familiarity to really do any kind of rating with the-22: actual project or activity.
23 As you may remember, the activities are generally 24 listed as a couple of sentences at most, that have more 25 meaning to the people that wrote it than the people who i ]() ANN RILEY & ASSOCIATES, LTD. Court Reporters . 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 i
+ e-- .tr-- == eer ..-- # -- - - ,,- - -n 3 ,- ,,- .--- e = 6
t 8 1 might be reading it. So there has to be some training n
-() 2 involved with understanding what each of those are. But any 3- ranking thing -- whichever technique we would use, the same 4 peot , would be involved with the actual ranking.
5 DR. POWERS: I have -- I always great misgivings 6 when people tell me that they are going to do peer-wise 7 comparisons. I am not very familiar with this analytical 8 hierarchy process, but I am very familiar with proofs done 9 by Ken Erroll and a guy named Black on peer-wise 10 comparisons. 11 What they demonstrated 20 yearr, ago, I guess, is 12 that collectively peer-wise comparisons 0.re never 13 transitive And they can't be ordered. You can not come up 14 with an orderino that maximizes utility and achieves a O(m/ 15 parieto optimum with peer-wise comparisons. 16 DR. PERSENSKY: Well -- 17 DR. POWERS: Because of their instransitivity. 18 DR. PERSENSKY: Yeah, because of the problems with 19 it. 20 DR. POWERS: You can play games with them. And: 21 do. 22 DR. PERSENSKY: You can do a lot of mathematical 23 machinations to make it look -- to come up with some sort of 24 ranking scheme. We are looking at it, if for no other 25 reason, the entire office, or the office director level, ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
t'4 U 9 1 they are looking at this same concept of how do we () , .2 prioritize amongst the higher level programs. 3- DR. POWERS: Look, I mean it seems to me what you 1 l 4 are going to have to do is show that you can come up with a . . 5- scheme in which you have transitivity. That is, if A is 6- greater than B, and.B is greater than C, then in fact.A is 7 greater than C. And you can not do that when you do 8 collective peer-wise evaluations. 9 CHAIRMAN APOSTOLAKIS: I think what Erroll showed 10 was that if you.have more than one decision maker -- 11 DR. POWERS: That's right. 12 CHAIRMAN APOSTOLAKIS: You can-not come up with a
'13 set of axioms for decision theory like you have with one.
14 guy, because you-violate those fundamental rules. So it
) - 15z depends very much on how you are going to use this AHP, If 16 you use it, say, to see how various players think about the
- 17 significance of these things,.and then you have to use some
,. 18 judgment because there is no -- I mean actually the theorem
- ~ ~ . -
19 -is called the impossibility theorem. 20 DR. POWERS: Well, the impossibility. theorem was,
- 21 in fact,- some of' Ken's later work.
22 CHAIRMAN APOSTOLAKIS: Yeah. f- 23 DR. POWERS: But in peer-wise comparisons, the r 24 problem -- the problem is still the same. I mean that -- I A 25 am sure one led to the other. That absent a benign () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 4 (202) 842-0034 r
,, ,.-..m._m--- , , , _ . . . _ ,r. y y.,, y+.--. m _ - _..
10 1 dictator -- ( 2 CHAIRMAN APOSTOLAKIS: Right. 3 DR.' POWERS: You know, if you have one decision 4 maker there is no problem. The transitivity, by definition. 5 It assumes you have more than one -- actually, I think you 6 have to have three to get into an intransitivity problem. 7 MR. ROSS: Dana -- 8 DR. POWERS: Yes. 9 MR. ROSS: Let me interject here because the work 10 that we are talking about was started last fall at my 11 instigation. Gus has circulated, I think, a memo ta all the l 12 Committec members on this subject the last month or so, ; 13 describing what is going on in more detail, at the office 14 level. I am not talking about the potential for within the gj 15 Human Factors and Human Reliability, of doing it at that 16 level, although that is being considered. 17 I agree that there is -- that you have to be 18 concerned about the transitive relationship, A greater than 19 B greater than C. And when you do this systematic 20 comparison, there are tests for -- that you have to do for 21 possible bias, because in the rankings, you could come up 22 with something you might call intransitive, that the 23 evaluator wouldn't be quick as accurate on being -- on this 24 transitive relationship as you would expect. 25 If you come up with something, total nonsense, (w- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
11 l' then there are tests to divulge that. -The -- () I2- DR. POWERS: The problem I am' working.-- 3 MR. ROSS: Is you look on the web for the uses 4 over the last 15 years of this te:hnique, you would have-5 literally-hundreds, perhaps thousands of case where this has 6 been used around the world. It is not a new, novel or a 7 singular approach. And it may take -- if we ever start 8 tm'.ng it-in. context of decision making, then the Committee 9- might have -- or the Subcommittee, depending on which -- l 1 10 however you approach it, might want to spend more time on 11 it. We are still exploring it now. 12 But I think properly done, with the right kind of 13 substantive training and-normative training, Lit can be 14 useful.
~~g 15 DR. POWERS: The problem I have-with.it, Denny, is 16 that it is too easy to play at - _to game it. I can
[ 17 guarantee'the outcome if you give me;your voting process. , 18 MR. ROSS: Well, I think in order to'come to a 19 judgment, you would have to know more about what we are 20 doing and why we are doing it than you now do. So I think 21 you ought to dismiss it out of hand. You ought to wait and 22 _ gather -- and see what we are doing first, and why we are e
- 23. doing it, and then come to a decision.
24 DR. POWERS: Well, I would hope'I do that. And,
'25 _in fact, in my response to Gus, I have said we have got to 2 , ( ANN RILEY & ASSOCIATES, LTD.
s ' Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
.w. --. . - . . . . - . - _ _ - .
12 1 know more about this. Because I see room for gaming-this
-Ch g 2 system.
3 CHAIRMAN APOSTOLAKIS: Well, yeah, but the point 4 is though that you should not use the AHP as a mechanism 5 that will give you the final answer, because -- because you 6 have stakeholders. At some point, in fact, you do go to a 7 benign dictator. Somebody has to make a decision. And that 8 person has to make sure that all these properties like 9 transitivity are preserved. 10 So as a tool to find out how various, say, branch j 11 chiefs feel about the various iten.s, it would be useful. 12 But it is not a mathematical model that will give you "the" 13 - answer. It will give you insights. Then y,u have to go and 14 deliberate. 15 MR. ROSS: Just to give you, for example, we 16 thought something very provocative came out it. The very 17 nature of a hierarchy means, the way we use it, is the-first 18 step is try to decide by what criteria should you judge 19 various proj ects . And after a lot of discussion, we decided 20 that we would like to consider the safety signi'ficance, the
.21 regulatory policy significance, and the likelihood that the 22 research project would succeed, the success factor. So 23 -there's three criteria.
24 And a number of NR -- research managers did a 25 peer-wise comparison on these three criteria. And NRR did
'\ ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 ;
13 1 theirs separately and, to my surprise, NRR rated the success Oj 2 criterion _much higher chan Research did. _( I would have 3 guessed it would have been the other way around. 4 Of course, the default is just to rate all three 5 equally, which we would do alse. And regulator / policy, 6 which I thought NRR would have pushed up to the high, they 7 had it relatively low. 1 8 So I think just the mindset of the managers in I 9 saying I want this research project more than I want that l 10 one, it was useful to us. So I think you can gain a lot of l , 11 ,information out of it. I am not saying it is an-engine by 12 which you plug in a decision and an answer comes out. No , 13 that's not what we do it for. 14 CHAIRMAN APOSTOLAKIS: Well, since there is (% (_) 15 interest in this, I mean at some point, we ought to discuss 16 how you are using it. 17 DR. PERSENSKY Again, as far as the Human 18 Reliability -- Human Performance and Human Reliability Plan, 19 this issue came up as -- I mean we were moving along in 20 terms of an absolute rating scale. But when the question 21 came as'we may be -- as an office, may be using something 22 different, rather than having a number of different rating 23 scalas going around, it seemed to make sense to be 24 consistent. 25 I'm putting it in here to inform you that this is O ANN RILEY & ASSOCIATES, LTD. U Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
14 1 a possibility, that we are looking at it as a means of
/~
(N) 2 perhaps getting further information as to how we might rank 3 these things. 4 MR. ROSS: One additional thing is that -- 5 obviously, we haven't used this for any purpose yet in 6 research. Usually, in budget space, there's sort of a 7 wolves-chasing-the-sled approach and you have to throw off 8 the lowest-priority baby so the rest can escape. 9 Prioritization might tell you which one to select. 10 We haven't done that yet. But we do know that we have some 11 energetic discussions with the full committee over the next 12 few months on Research with a capital "R", and I think 13 that's the forum where we'd want to continue this subject, 14 CHAIRfiAN APCSTOLAKIS: As a tool for giving you
/^% '(,,i 15 valuable insights into how individual people think, I think '
16 it's very useful. The other thing is that you can identify 17 inconsistencies very easily. 18 Now, how you go from the individual Essessments to 19 the agency's prioritization is something that needs some 20 discussion, and that's where the problems with the 21 transitivity coma about, if you want to do it in a 22 mechanistic way, which you should not do. 23 But at least you know the person or the small 24 group that will make that decision will have more 25 information regarding how various people with diff rent
/~N i ANN RILEY & ASSOCIATES, LTD. \~') Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
-15 1 _ experiences and perspectives view-these things.
() 2-
-- 3
'So, at thic point, I would say keep exploring it, but when you come back, we ought to still discuss it.- Be 4 prepared to answer Dr. Powers' question.
5 DR. PERSENSKY: I believe tne discussion on the 6 AHP is going to'be at the higher level.' 7 CHAIRMAN APOSTOLAKIS: Higher than the ACRS? 8 DR. POWERS: No, higher than the-human performance 9 plan. It's going to be at the-RES office, the big "R" 10 level. Again, all we're trying *.o do is to stay with the 11 mainstream of what is-going on. !
- 12. CHAIRMAN'APOSTOLAKIS: So, who's going to come_and 13 talk to us--about it? You-guys?
14 HMR . ROSS: Yes.
) 15 DR. PERSENSKY: Probably.
16 MR. CARROLL: Has ACRS looked at this agency 17 strategic plan?- i 18 CRAIRMAN APOSTOLAKIS: No .- It was published -- I 19 don't think we were ever asked to comment on it, were we? 20 DR. POWERS: Well, we've certainly looked at it in 21~ some depth. We've not been asked to comment. 22 MR. CARROLL: Okay. So, you know what it -- 23 CHAIRMAN APOSTOLAKIS: Oh, yes.- we knm; wnat it 24 is.
'25 MR. CARROLL: -- is about and how it may relate to O ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
15 1 this prioritization process. 2 CHAIRMAN APOSTOLAKIS: I'm not sure. 3 DR. POWERS: Well, I mean the strategic plan lays 4 out a very high-level arena. 5 Part of that strategic plan says that we're going 6 to focus on the most risk-significant things and that we're 7 going to do research and things like that that address 8 issues associated with the licensing and regulation of 9 reactors, and part of that is where we get into -- we've had 10 difficulties with this human performance program plan -- is 11 that we, right now -- at least one member of the 12 subcommittee does not understand what issues it is that are 13 being addressed 'y this plan. 14 CHAIRMAN APOSTOLAKIS: Well, basically, with 15 matters like this, what they do is they tell you you are 16 about to make a decision and there are many criteria. How 17 do you structure that process? Which criteria comes first? 18 And then you compare -- you focus on -- 19 MR. CARROLL: I understand that. 20 CHAIRMAN APOSTOLAKIS: But the Commission's 21 strategic plan never goes into such detail. 22 MR. CARROLL: Okay. Another question, Jay. Let's 23 wave a magic wand and pretend like you have the perfect 24 prioritization process -- 25 DR. PERSENSKY: Okay. ANN RILEY & ASSOCIATES, LTD. Court Reporters l 1250 I Street, N.W., Suite 300 84 b34
17 1 MR. CARROLL: -- and you also know what each of () 2 these activities that you're prioritizing is going to cost 3 and what's the schedule for doing it and all of that good 4 stuff. What do you do with this? How do you use it when it 5 comes time to ask the management to budget so much for these 6 numan factors activities relative to other activities that 7 research, for example -- to know whether things are 8 importan".? 9 DR. PERSENSKY: I think that's where you get into 3 10 two different prioritization schemes. There is the research 1 11 prioritization scheme at the higher level where they look at 12 it across the program areas, and that's in the control of 13 the division directors and the office director.
~
14 Within this plan, we have to recognize that there ((,,,
/\
15 are different offices represented, the activities cross 16 offices. There are activities that are research projects 17 for which we have to go and say we need money to do this 18 activity. 19 If we have this perfect prioritization scheme, if 20 I had 20 research activities I want to do over the next five 21 years and t'Jey cost X dollars but I only have X minus 50 22 percent, then somewhere there would be some sort of cutoff 23 in that as far as what I would, as the human factors leader, 24 would try to go forward with. 25 On the other hand, if you go over to NRR or the I\ ANN RILEY & ASSOCIATES, LTD.
\w- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 1
18 - 1 other program offices where their priorities deal more with () ? which inspection am I going to support this week, then they 3 may have a different set within NRR. 4 We're looking at an overall -- and one of the 5 problems that we've been having in trying to apply this
- 6 prioritization scheme or select a prioritization scheme is 7 how do you prioritize an inspection against a project -- a '
8 research project on alarm systems or on management and 9 organization? 10 Management and organization is a problem. We 11 don't have the tools to do that. So, it has to be 12 somewhere. 13 But again, the NRR people or the NMSS people that 14 are doing-the inspections -- they have their own priorities, "
) 15 and given the constraints on resources, both people and 1G dollars, each office is going to have its own set within 17 this overall set, and it's probably going to be interleaved 18 in some way.
19 MR. CARROLL: But what tools does the management 20 of RES have to decide that piece of the pie should go to 21 human factors and this piece should go to thermal hydraulics 22 and this piece to something else? 23 DR. PERSENSKY: Well, I think that's exactly the 24 question that Denny was trying to address by proposing AHP. 25 I mean that's one method that they have looked at at the [ ANN RILEY & ASSOCIATES, LTD.
\-
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
h 19 1 office level, and there has been a memo sent to the HRS 2 members, 3 It's not -- again, we still have'-- there isn't 4 unanimity on that.--.that that is the perfect tool. It is a 5 tool. So, they are looking at that again, but that 's at a 6 level.that's above this plan. 7 MR. ROSS: Let me try and intervene. 8 For example, somewhere in the research office, 9 there-is a person who's spent the_last 25 years being 10 extremely knowledgeable on zircalloy metallurgy. 11- So, if we had a research project -- and we do -- 12 and zircalloy metallurgy and that person were elicited under 13 the AHP process, the bias concept would mean he would 14 probably give an extremely high weighting to zircalloy 15 metallurgy _and zeros to just about everything else. 16 _So,'one of the pitfalls of=prioritization is bias, , 17 and we ,.now it. You can start-to_get around bias if you 18 have people who, because of cross-pollenization, have broad 19 interests or you have people who manage multiple _ interests 20 and,-in the-zero sum sense, they would get into bias at 21 their own regret,because what they take from here they might
-22 lose there.
4 -
-23 So, we're-looking at various ways on this process-24 to elicit from people whose bias factor is relatively low.
25 For example, in NRR, at the associate-or deputy O ANN RILEY & ASSOCIATES, LTD. Court Reporters
'1250 I Street, N.W., Suite 300 Washington, D.C. 20005 '
(202) 842-0034 U
20 1 director level, they might have a passing interest in a () 2 subject, because they did it decades ago, but they're 3 managing or responsible for a broad area, so institutionally 4' their bias factor should have gune down.
- 5 On the other hand, a branch chief in human factors 6 -- you'd expect, if he or she were part of the process, 7 .w ould be expected to have bias, and we know it, and it's j 8 something I think we can take care of if we're watchful.
9 CHAIRMAN APOSTOLAKIS: I think it makes the 10 process more transparent. That's what really this does. ' { 11 You know, eventually, it's somebody's judgement. There's no J 12 rule that will give you the answer, but at least you know l
- 13 now how various people think and you know why the top guy i 14 made the decision he made. That's-really the value of these
) 15 methods, and I think rio should lea Je it at that, but I am 16 very interested in the subject, partly because I just i 17 finished a paper on AHP.
18 MR. HODGES: This is Wayne Hodges. There's a lot-19 of constraints that you-have to play with. In Jay's case, 20 when I'm looking at work that I was on for him, one of the 1 l 21 . major limitations is he's-got a very small staff, so he can 22 .only handle so much work. 23 MR. CARROLL: That would be one of the issues.
'24' MR. HODGES: That's clearly one of the issues.
I 25 CHAIRMAN APOSTOLAKIS: But the other point you ( . ANN RILEY & ASSOCIATES, LTD.
% Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
21 1 have to bear in mind is that decision analysts or theorists j f 2 don't like this method, and I think this method is suffering 3 from having been oversold in the past as a decision-making 4 tool. For example, it's very hard to include utilities in 5 this thing. 6 So, it has been proposed as an alternative to 7 decision theory, which really it isn't. So, you have to be , 8 very careful. 9 I mean it's useful at some level, but if you say 10 AHP to somebody who has been working on decision theory, il you're going to get a major -- 12 MR. CARROLL: -- attack. 13 CHAIRMAN APOSTOLAKIS: So, don't present it is a 14 decision-making tool. Present it as a tool that allows you 15 to structure the problem, you identify the criteria, which 16 is really the greatest value of this, and you see how. 17 various people think, you can point out inconsistencies, but 18 at some level, somebody has to integrate it in a behavioral 19 way, in a judgement way. 20 So, I think we are 'ending a lot of time on AHP, 21 and t'1at's not really your subject today, but I, for one, am 22 very interested in this. So, at some point, I would like to 23 see -- in fact, I know that Bill Vesely did the report for 24 the Office of Research, what, 20 years ago, on prioritizing 25 research topics using the AHP?
\
(I ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l l
22 1- MR. CARROLL: -Yes. 2 CHAIRMAN A?OSTOLAKIS. .Yes. So, it's not 3" something that's very new.
-4 MR.' CARROLL: Right.
5 CHAIRMAN APOSTOLAKIS: But-I-know that because, as 6- I say, that paper that I wrote was reviewed by decision. , 7- theorists, and I saw the reviews. So, you have to pay due s 8 respect. 9 The-decision theory itself has problems. -None of 10- these theories is really very solid, because you know, thera 11 is a famous paradox by this Frenchman, Allais, who showed 12 that even the basic. axioms of decision _ theory are violated 13 by people and;so on. 14 But we-are getting really inte very esoteric ' 15: _ discussions here.
-16 DR. PERSENSKY: verhaps.we can do a project on it,
-17 but first we have to prioritize where it comes into --
18- CHAIRMAN APOSTOLAKIS: Exactly. 19 DR. PERSENSKY: -- which gets us into another 20 -problem. 21: MR. CARROLL: You'll have no problemidoing that-22 :with your enlarged staff. 23 DR. PERSENSKY: With my enlarged staff that Jim 24 just-gave me. 25- CHAIRMAN APOSTOLAK'IS: But if this is to become a 1 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-G034
s 23 , 1 major-tool--in the Office of.Research, maybo we ought to
-p)-
( 2- - discuss it on the subcormitteellevel and maybe give you some input. 4 MR. ROSS: Yea. 5 CHAIRMAN APOSTOLAKIS: -It's up to you. 15 MR. ROSS: I think so. We probably would recount 7 some experiences we've had with other methods. I know we 8 used decision analysis theory on NUREG-1150, and you talk
-9' about deviations -- extreme deviations amongst experts 10- there, 11 But we're-wide open. If someone'c got another '
12 technique, we would;be delighted to consider-it. 7.3 CHAIRMAN APOSTOLAKIS: Well, there a"e many other ( 14 techniques, but they suffer from the same problems. )
\, -
11 5 DR. SEALE: I would hope that any discuusion of 16- these processes would also include in it an indication of 17 how you do plan to use the input!of the expert on circalloy 18 and metallurgy. 19 MR. ROSS: Sure. > 20 DR. SEALE: Especially-when you're talking about 21 zircalloy. 22 CHAIRMAN APOSTOLAKIS: Okay,. Jay. 23 DR. PERSENSKY: Okay. Sort of moving more into 24 the specifics of where we are in the-plan, again we're
.25 trying to use the structure of the agency strategic plan.
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
[L - 4 24 1J That structure has a mission, a vision, the various things t : 2- -you see here listed. 3- Beyond just looking at the structure, we've also-41 looked at -- as you know, we-have had our problems dealing 5 with what'n=the best v'. of writing a vision,-what's the
;6 best way of: writing a mission statement, is that there are-7' vision: statements and mission statements, and one way of 8 getting at it is just to-include in.the agency vision and 4 19 mission statements the words that make it fit within human 10 [arformance.
11 There-are some back-up slides at the end of this 12 presentation that is one attempt at doing that, as opposed
=-
13 to our trying to come up-with our own independent, again not 14 _part of the mainstream,;perhaps, kind of approach.. A
\s ,)1 15 There-are also goals within-the-strategic plan 16 which relate very closely to the strategic arenas, which 17 we're going _to get to in a minute. Then there-are 18- strategies and sub-strategies, and what we have in <ur plan i 19 is activities, and the goal is to have these activities fit H2O the sub-strategies of the etrategic plan.
21 If you look at this graphic, you see that, in 22 : fact, what we do or how this might work in the-ideal world 23 is that we would;have our plan which would go-into each 24 office.as its operating plan, which would take the high 25 priority-items, put it into the operating plan, then that 4
/
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington,-D.C. 20005 (202) 842-0034
25 1 goes-into the performance plan, which becomes part of the () ;2 strategic plan, which in fact then goes back and becomes a 3 -driver as it-is-in terms of the format.
.4 At the louer-level,-you have-the activities which 5 lead through planned accomplishments, output of the --
6 outputs-is the terminology used in the performance plan -- 7 and it links to the sub-strategy. Again, there is a 8 feedback. So, that's how the documents fit together from 9 :that standpoint. 10 The strategic arenas are, in fact, the primary 11 ' functional areas of the strategic plan -- the reactors, 12 materials, waste, security, international -- with E 13 international involvement, environment, public confidence, 14 and excellence. 15 I understand that the plan is undergoing rev3 tion 16 currently by a task force and that.the number of reenas may : 17 be reduced, that there is some change. As M'.at happens, our 18 plan will-have to-keep pace with-the agency plan, 19- To give you an example -- a couple of examples of 20 what we mean by: linking this-together -- again,-one of the 21 strategic arenas is nuclear reactor safety. A strategy 22 within that plan is to assure that licensees discharge their 23 responsibility for safe operations. The sub-strategy, which-24 is really more our part of it, is we will regularly assess 25 and measure and report on license performance. So, this-is
^h
%/
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
26 1 sort of our driver for certain activities. () 2 'Our issues or looking at the issues related to 7 3 that, certain things -- and there are maybe a number of 4 them, but right now, as you know, there's a lot of interest 5 in the senior management meeting, organizational 6 effectiveness, so we're looking at ways to measure
'l organizational effectiveness as part of this sub-strategy.
8 On the other hand, there are -- and this would 9 perhaps .tn a researth environment or research activity -- 10 systematic evaluations. This is something that might come 11 more fr)m the standpoint of what AEOD does in terms of 12 looking at the data, trends, etcetera. 13 And finally, the programmatic inspections -- this 14 is something that our regions and, generally, NRR, since () 15 we're talking about reactor safety here, would be 16 performing. 17 So, these are all activities within the plan that 18 relate to this particular sub-strategy. There are a-number 19 of other activities that may relate to it, just picked out 20 these three-to show the progression. 21 Another example -- 22 CRAIRMAN APOSTOLAKIS: And all of these activities 23 can be risk-informed, right? 24 DR. PERSENSKY: Yes el the activities would be-25- risk-informed,-and_we're going o get-to that a little bit i O ANN RILEY & ASSOCIATES. LTD. Court Reporters 1250 I Street, N.W., Suite 300' Washington, D.C. 20005 (202) 842-0034 J
27 1 in our framework. () 2 Another one of the strategies within-the same 3 strategic arena is that we will maintain a research 4 capability, and the sub-strategy-is to parsue the reactor 5 safety research program that has both confirmatory and 6 anticipatory research. 7 .The developing guidance for human system 8 interfaces are, in fact, based on user needs, if 9 high-priority user needs from NRR to develop guidance 10 because they need that in their reviews. So, that would be 11 . more the confirmatory, whereas -- though we had in the pr.st--
.12 a user need for the root cause investigation, we're now 13 looking at an evaluation of the one that we developed about 14 six or seven years ago to try to improve its usefulness.
() 15 So, this is more in terms of, from our perspective,
- - 16 anticipatory._ 1 17 DR. POWERS: Can we chase this one down a little-18 bit, Jay, so I understand?
19 DR. PERSENSKY: Sure. 20 DR. POWERS: You have a root cause investigation-
;21 rapability.
22- DR. PERSENSKY: We have a document called the 23' human performance investigation process that11s used by NRR. 24 DR. POWERS: It's used. You've found that
,_ ____25__ deficient somehow.
ANN RILEY & ASSOCIATES, 7,1D. O'- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
28 1 DR. PERSENSKY: We have gotten some feedback from-() 2 various people that-there are some weaknesses in it, yes. 3 DR. POWERS: Very logical that there would be. 4 DR. PERSENSKY: Right. 5- DR. POWERS: So, now what you see is a need to do 6 something to fix that. 7 DR. PERSENSKY: Correct. 8 DR. POWERS: Okay. Do you have some idea behind 9 all this how much it needs to be fixed, I mean how good it 10 has to be to meet their needs? 11 DR. PERSENSKY: Well, there's a couple of issues 12 or questions related to it. One is there are -- it's used a 13 lot in one particular region, not used in another, and one 14 of the questions that we have for ourselves is why is it not () 15' used in this other region, and the first step in this 16 process is going to the-potential users to get information 17 .from them to help identify what the specific problems might 18 be,-to get them to use it more. 19 DR. POWERS: That's a different spin on things, it 20 seems to me. 21 DR. PERSENSKY: That is one of the spins. 22 DR. POWERS: And that's fair. I mean I'm not 23 being critical, I'm just trying to understand here. 24 DR. PERSENSKY: Okay. 25- DR. POWERS: On the one hand,_you_could have ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
1 29
.1 something that is used, that people say it's not very
() 2
~
convenient to use or it's not very accurate or it's -- 3: something about it that they don't like, that they would 2
-4 like to have it better.
5 - DR.-PERSENSKY: It doesn't go far enough. ! { 6 DR. POWERS: Doesn't go far enough. ; 5 7 DR. PERSENSKY: That's one of the biggest ones. 8 DR. POWERS: Whatever it is, it's used, but it 9 could be better. On the other hand, you could have - -maybe ; 1 10 simultaneously, you have someone that says it's so deficient 11 that I won't use it at all -- t 12 DR. PERSENSKY: Okay. 13 DR. POWERS: -- I just don't like it. 14 Now, you're coming back and saying, okay, I want j h() 15 people to do. root cause investigations -- somebody has said 16 that someplace -- . 17 DR. .PERSENSKY: Okay, j i - 18 DR. POWERS:- -- I presume that's been said -- and
- 1. 19 I would like them to use a consistent process to do root ,
. cause-investigations and this is my consistent process, I'm 20 21 willing to change it, but I want everybody to use it.
22 DR. PERSENSKY: Yes. 12 3 DR.-POWERS: Oktty . So, now,'you have two chores _ 2C --.one, to fix it for one guy and, one, to-fix it a whole
- 25 lot for another guy.
(' ANN.RIL3Y.& ASSOCIATES, LTD.
] '
Court Reporters-1250 I Street, N.W., Suite 300 Wachington, D.C. 20005 (202) 842-0034 putayt.yw.+.-y.-yg,,99-p*Ntry 'ui- Wa%M1-1-=f--+"vff -T* M' - -
- T'~
-"*WW- " - ' T
30 P 1 MR. HODGES: This is Wayne Hodges. Let's see if I () 2 3 can help a little bit, because I'm one of the ones who complained about it. 4 In a former life, I was in a region and had to do 5- some of these root causa analyses or head up some of the-6 root cause analyses, and I identified some problems, and so, 7 when I came to Research, I relayed some of those problems, 8- but as kind of a background -- 9- DR. POWERS And that's a function that we want, 10 right? 11 MR. HODGES: Yes. 12 DR. POWELS: I mean that's why we have percolation 13 within the agency. 14 MR. HODGES: Absolutely. () 15 The agency uses a technique called MORT, 16- Management Oversight and Risk Tree, for most of the 17 augmented-inspection teams and the incident investigation 18_ _ team inspections, which is a very systematic-way of-looking-19 at and doing root cause analyses on problems, and it has in 20 it an element of_looking at the human aspects of what's 21 happened. 22 Independent from that-but supportive of that, the 23 Office of Research, five or_six years ago -- or more than 24: _that now, maybe -- developed this HPIP methodology,uwhich 25 has a better structure for looking at some of the human ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202)-842-0034
gj,h hhhy 3: O- 31 1 performance issues specifically. 2 In trying to apply that technique at some of the 3 plants in the region where I was a division director, I l 4 found it useful, but I found that it didn't go quite far 5 enough in trying to get me the answers that I wanted, and S. 6 so, I brought some of those concerns back to the Office of 7 Research. 8 I suspect one of the reasons that other regions 9 may not use it as fully is they are comfortable with MORT, 10 they have a tool, they are not terribly familiar necessarily 11 with HPIP, and so, part of our problem is maybe 12 familiarization and some advertising that needs to be done, 13 but the basic tool is sound. 14 So, we're talking about an improvement, not a 15 major overhaul. 16 Does that help any? 17 MR. CARROLL: Does the training center offer 18 training on this? 19 DR. PERSENSKY: In fact, when we developed the 20 tool, we developed along with it a one-day training course. 21 It has been integrated with a larger training course on root 22 cause analysis that relates also to the IIT training and AIT 23 training. 24 So, it's being wrapped up because it should be 25 integral to whatever we're doing. It's just that, when ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 n02) 842-0034 l 1
32 1 you're going through the process and say -- okay, this is e () 2 human-dominated event, let's go this path as opposed to the 3 pumps and valves path. 4 MR. CARROLL: That hasn't helped in terms of 5 different regions embracing this? 6 DR. PERSENSKY: It's voluntary as far as taking 7 the course, and I think the integration of it, which just 8 happened, actually, within the last year, may help from that 9 otandpoint. 10 We are, though, also -- again, in trying to 11 identify what are the problems, we're going down to the 12 training center and talking to people who have been through 13 the training program, using what we consider a typical 14 research technique in terms of, really, up front, is identifying from whatever data we can gather what the real 16 specific problems are and then addressing those in the fix. 17 We also are looking at new techniques that have 18 -been developed in other places. This whole root cause 19 analysis has taken on a fairly strong life within the 20 utility industry, as well. It's gone beyond INPO's HPES, so 21 that there's more emphasis from that standpoint in coming up 22 with better techniques that get further into some of the 23 issues. 24 So, we're trying to learn, again, from other 25 places, and it looks like one of our regional inspectors is () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
1 33 1 about to make a -- give a statement here.
-2 MR. MEYER: Well, I would like to provide a little 3 ' regional perspective.
4' DR. PERSENSKY: Glenn, can you identify yourself? i 5 MR. MEYER: My name is Glenn Meyer. I am from l 6 Region-I, and I've spent my time there, and as luck would !- 7 have it, in a_ prior job, I was involved with the development .
- 8 of the HPIP process, and the one perspective I want to- '
9 convey is that it is a limited tool, and that's by design. 10 When we started out to develop HPIP, what we were
'll looking for was something that could be broadly applied with 12 a limited-training burden, so-to speak, because as Wayne
! 13 Hodges referred,-MORT is a fairly -- very detailed tool. j 14 Unfortunately, to be able to uso it, you need on the order
- 15 of five deys of training, which wasn't that likely that you I6 could apply it to all the inspectors. It's a very limited
} 17 set of inspectors that get MORT training. J 18- So, HPIP was designed, and one of the_ things that l 1 19 we started out with was it should be usable after one day's l- 20 : training, That was a constraint, a design input to the I 21 process, give us a tool that's useful but useful after one 22 day's training, and what was-the objective? The objective 23 was to get inspectors to broaden their perspective beyond
- 24 their -- let's say, their hunches, to expand their horizon, 25 and HPIP was very effective in doing that, because when you ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250-I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 _ _ a. _ - . _ . _ - - .
34 1 used the tool, it got the user to think about some of the () 2 other factors that might be applicable to a given event or, 3 you know, problem that they were trying to analyze. 4 So, to tell you the truth, I was a little 5 surprised to hear that it's been integrated into broader 6 training which will make it -- those that get the trainin9, 7 I think, will have a better perspective and more tools. 8 Unfortunately, fewer inspectors will tend to get the 9 training, because it's five days long. 10 So, it is -- keep in mind that initially it was 11 developed to serve a limited function but to be broadly 12 applied to a lot of inspectors, and I think it was effective 13 within that objective. It's not going to be the tool that 14 replaces all the other tools. It's going to be a tool 15 that's usable in the field, that broadens the inspector's 16 perspective to a range of human factors problems, and-it's 17 usable with one day's training. That was the objective. 18 So, I thought that it served that purpose, and now 19 it's being adapted to other uses. 20 MR. CARROLL: Well, again, you bring to my mind a 21 priority problem. We can only afford one day to train an 22 inspector on root cause analysis? 23 DR. PERSENSKY: Just on the human performance 24 aspect of it. Again, as Wayne said or Glenn said, if you go 25 through the whole thing with MORT or other root cause () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
35 1 analysis, it may be a five- to 10-day course. 2 MR. HODGES: The basic course for inspectors for 3 MORT-is a one-week course, and then, for those who train for 4 _the IIT investigation, it's a two-week course.
-5 MR. CARROLL: Do all inspectors get MORT training?
6 MR. MEYER: I'm not certain, to tell you the 7 truth. I don't believe that they do. 8 MR. HODGES: A large number do. A fairly large 9 number do. l 10 MR. MEYER: I don't believe it's actually a 11 requirement that each inspector have that training. l 12 MR. HODGES: I think, when I was in Region I, we , 13 tried.to give MORT to all of our inspectors, but I don't 14 know what all the other regions do. . 15 -DR. PERSENSKY: Some feedback we got at one point ' 16 . was that, to really use this tool effectively, you need 17 three days of training. So, there's a lot of information.- 18- As research,-to some extent, anticipatory-in this 19 case,-we're trying to better define in scope the whole issue 20 based on feedback from-the users. 21 DR. POWERS: The difficulty I'm having here is ! 22_ understanding how I go from sub-strategy to this, the 23 activities you've got down here, in particular " develop l 24 improved guidance for root cause investigation of human-25 performance-related events." t f b ANN RILEY.& ASSOCIATES, LTD. Court Reporters , 1250 I Street, N.W.,-Suite 300 l Washington, D.C. 20005 l i (202) 842-0034 i
. , , - . - - * - - - . --.,-w, .-m, -, ..g..+..o..--.r ,--,,_v.-.--.-,, - _ . - - ---.--._-,m._ -.%-----, ...-,-.w., . - ~ . .,.=m-7
-- - - - . - - - _ . _ _ - - - . . - . _ _ - . _ ~ - _ - - _ - _ _ - - .
36 1 It seems to me I need two things to get there. I () 2 need somebody to say that root cause analyses on human l 3 performance-related events shall be done and somebody else 4 to say or the same person to say, I want a consistent ! i 5 framework for doing the root cause analysis. 6 DR. PERSENSKY: One of the difficulties we have, 7 even in trying to link it to the agency's strategic plan, is 8 some of those same questions -- these things may just as 9 well fit under another sub-strategy, the one we just had 10 before, in terms of regularly assess and objectively measure 11 licensee performance. 12 One way of measuring it is looking at their 13 capability of doing root cause analysis by doing our own 14 root cause analysis. So, that tool that comes out of here O( ,/ 15 may just as well fit under another sub-strategy. 16 Within the sub-strategy of we shall do research -- 17 and that's what the sub-strategy says -- these are some 18 things that we've done in terms of including where the 19 research element of a particular activity might go. 20 If we can hold that just a minute, maybe, with any 21 luck, the high-level model will -- or the attempt that we're , 22 trying -- the most recent attempt at the high-level model l ! 23 may help with that. l 24 DR. POWERS: I'm perfectly willing to wait. I'll 25 caution you, I'm extremely dubious that any model is going e ANN RILEY & ASSOCIATES, LTD.
'-- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 I _ _ .-
37 1 to persuade me that there's a connection between the () 2 sub-strategies and the activities. 3 DR. PERSENSKY: Okay. 4 Speaking of the model, again we're trying to build 5: this or to construct this in a way that it does support the 6 agency's strategic plan. It is -- based on some comments-7 you made or that ACRS made,as far as being event-driven or 8 event-informed -- we can, I guess, use that, since we use 9 risk-informed, we have event-informed -- in terms of 10 identifying activities, event data -- because there's all 11 kinds of things that might come in, it wouldn't only be 12 event. 13 The concept that we have is that there's a 14 functiou orientation in terms of, if you were to do a () 15 function and task analysis of what we do here at the agency, 16 especially in the human performance area, there are certain 17 things that we do. 18 It serves as a way of getting us to activities 19 based on some set of input, and I have a' couple of examples, 20 again, that might lead you through to get to it. 21- The structure or model or whatever you want to 22 call it looks something like this at this point. 23 DR. SEALE: A pentagram. 24 DR. PERSENSKY: Either that or it's a sheriff's 25 star, if you're into cowboy stuff.
=
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
38 1 But in fact, there are a number of inputs or () 2 drivers to any of the activities that we do, anything from 3 experience -- I'm not sure why that's all messed up -- s dxperience from the industry, expefience from other
-5 industries, industry initiatives,-what's going on in the 6 industry right now that may not be event-driven, but we know 7 there are issues such as decommissioning, downsizing,-the 8 deregulation, plants are going to digital I&C -- those are 9 all things that we have to be aware of in deciding what to 10 do in the activity.
11 Things that may be more reactive are risk 12 insights. I mean you can look at the risk insights reports 13 that we get and say, hey, here's something that's a high 14 contribution to 'isk, if we can_ find that in the human 15 performance area. 16 Allegations -- we have to respond to allegations. 17- Allegations provide a source of information to us. 18 Events, assessment of events, when you have 19' multiple events to geP. into. 20 We also get regional requests, especially -- I 21 mean NRR -- again, this is total agency. If the region 22- calls and-says, hey, we need some human performance people 23 'on an inspection at unit X, they have to look at it-in terms 24 of how they're using their people right now, but they're 25 likely to send somebody or call and see if we can send () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
1 39 1 somebody. () 2 We also have management direction. At times, 3 management just tells us, do this. What their input is, we 4 mey not-necessarily have much opportunity to question. 5 But those are all the various inputs to it -- we have risk-significance, here's your risk-informed question,-
-7 and prioritization that would go into actually dealing with 8- the individual activities.
9 At the function level, we have people in the 10 program offices that monitor what's going on, they do 11 inspections, they review documents that are submitted -- 12 FSARs, tech specs amendments, things like that. That's 13 generally in NRR or NMSS. 14 We collect, analyze, and evaluate data. Primary (h 15 responsibility there is AEOD, nct the only ones. NRR has 16 their their human factors information system that they use. 17 We look at data from all over the place as in the Office of 18 Research. 19 From a research perspisctive,Hwe perform research, 20 we do developmental activities Covelopmental in terms of 21 evaluation tools, and we actua+.iy may evaluate various 22 techniques for things.
-23 We airo have professional responsibilities or 24 activities.- We participate in professional societies.
25 Standards activities, I would say, fall in there, or we've O ANN RILEY & ASSOCIATES, LTD, Court Reporters 1250 I Street, N.W., Suite-300 Washington, D.C. 20005 (202) 842-0034
l 40 1 put them in there, and many of us participate in those () 2 3 standards activities. We also participate in international forums such 4 as CSNI, the PWGs that deal with that, and there's also a 5 number of administrative activities. 6 CRAIRMAN APOSTOLAKIS: What is HPIPS What does it 7 stand for? DR. PERSENSKY: The human performance and human 9 reliability implementation plan. 10 So, these are the plan activities. The activities 11 of the plan really are part of our function. They come from 12 the functions that we do that are driven by these outside 13 factors. 14 CHAIRMAN APOSTOLAKIS: So, what you're saying is
) 15 that these are the kinds of things that you will have to do.
16 That's a recognition of -- 17 DR. PERSENSKY: These are the categories of things 18 that we do and have to do. 19 CHAIRMAN APOSTOLAKIS: And you have to do. 20 DR. PERSENSKY: Right. 21 CHAIRMAN APOSTOLAKIS: Now, you mentioned 22 international activities. Is anybody else -- I mean other 23 than the United States -- trying to develop a plan like 24 this? I know that the Germans are very interested. As you 25 know, Professor Wilpert of their ACRS has expressed O 10m RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
41 1 interest. But is there any effort, perhaps, to join forces q() 2 and see what we can get out of it? 3 DR. PERSENSKY: There are some efforts, I believe, . 4 at PWG 1, Principle Working Group 1 of CSNI. Dick Eckenrode 5 from NRR is the representative to the expanded task force on 6 . human. factors. 7 CHAIRMAN APOSTOLAKIS: And they are thinking in 8 -terms of a plan, or is it just human factors? j l 9 .MR. ECKENRODE: Richard Eckenrode from the Human- ! 10 Factors Assessment Branch, NRR. 11 No , PWG 1 is not, at the. current time, looking at 1 i
- 12. a plan. This summer there was a meeting over there with a 13 lot of people in research and so forth to come up with the i
14 , various activities that they thought the PWG 1 should get () 15 involved with, and a plan is not one of them.
- -16 DR. PERSENSKY
- But there is a list of activities, ;
17 areas of interest within that, i 18 DR. POWERS: Do we have that list? 19 MR. CARROLL: .That's-what you used to call a plan'
- 4.
20 before you tangled with these guys.
- 21 DR. PERSENSKY
- Yes.
And'their consultants.
~
22 CHAIRMAN APOSTOLAKIS: 23- DR. PERSENSKY: I do not have that list. l 24 Has that report come out, Dick? E 25 MR. ECKENRODE: That has'been submitted to the PWG lO F ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 400 Washington, D.C. 20005 (202) 842-0034
42 1 1 and basically was approved by PWG 1. It now goes to -- I () 2 guess it's going-to go to IAEA, also.
- effort, to some extent.
It will be a joint 3-
- 4 DR. PERSENSKY
- Yes, AIEA had a part in that, as 5 well, now that I remember.
4
! 6 MR. ECKENRODE: And it also has to go to PWG 5, 1
7 which is the PRA working group. I 8 CHAIRMAN APOSTOLAKIS: But you aren't constrained 9 to work only through those bodies. I.mean since-Dr. Wilpert l 10 has shown interest -- and in fact, every now and then he I 11 sends me an E-mail, you know, what is going on with the NRC , 12 plan, and I direct him to you, I think -- is it possible to 5 13 have a bilateral-collaborationr ' 14 DR. PERSENSKY: There is, in fact, a good .
- 15 possibility, and there's a lot of encouragement at this l 16 point from our management.
17 Jerry Watell is not here today, who was working on f 18 this, because he's in Korea, one, giving a tutorial on 19 NUREG-0700,-the control-room-design, because they asked for-20- it,.but.also, he's meeting with one of the Korean ' i 21 organizations, the Korean Research Institute, because they 22- are interested in some potential collaborative efforts.
- 23. We have been talking with Spain about some work 24 primarily in the organizational factors. We've been sharing 25 some information with them. '
t () ANN RILEY & ASSOCIATES, LTD. Court Reporters 12501I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 7m, ew ,..,,er -cm-.-e.,,--m--e-.m.,,,, - - - < =e ++'*e *w * '*e5 "* * " * " ' '"'v"* '*'""7~ '
43
-1 We're very involved with Halden, as you know.
2 CHAIRMAN-APOSTOLAKIS: .But these are individual 3 projects. I mean the thing about Wilpert's interest is that 4 he really-is interested in the plan, because he would like 5 to see one in Germany, as well. So, maybe that's worth 6 pursuing. I don't know. We don't have to exhaust it now. 7 DR. POWERS: Let me just ask another question 8 about what I'm looking at here. What you've listed here, I 9 think, is -- you have a wonderful name for these things, 10 influence diagrams -- influence diagram on human performance 11 and human reliability implementation plan activities. 12 DR. PERSENSKY: Okay. 13 DR. POWERS: And the regulations don't show up. 14 DR. PERSENSKY: Well, the regulations is one of 15 the cutputs that might be considered down here in 16 administrative. In fact, if there is a need for a 17 regulation, it's usually identified in some way through 18 these activities here. 19 DR. POWERS: What I'm saying is not the need for a 20 .new regulation. The existing regulations don't show up as 21 having an influence. 22- DR. PERSENSKY: Right there. They use the 23 existing regulations in order to do their inspections.
-24L DR. POWERS: Okay. So, you have to read this kind 25 of carefully to understand where the. regulations come in.
i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005
-(202) 842-0034
\. . ..
i 44 1 DR. PERSENSKY: Well, in terms of understanding () 2 what -- you know, our inspectors inspect to regulation. 3 They also are beginning to move towards inspection -- 4 towards performance, but in fact, bottom line is we inspect 5 to the regulations. 6 CRAIRMAN APOSTOLAKIS: What is the idea of showing 7 this diagram? 8 DR. PERSENSKY: Just to show the influences of how 9 the various things interact. 10 CRAIRMAN APOSTOLAKIS: And what you guys have to 11 do. 12 DR. PERSENSKY: And what it is that we do as a 13 group -- 14 CRAIRMAN APOSTOLAKIS: Right. 15 DR. PERSENSKY: -- and what we ch) are, in fact, 16 activities. 17 CHAIRMAN APOSTOLAKIS: Right. 18 DR. PERSENSKY: I mean these are the things that 19 go on, and they're driven by these, let's say, outside 20 sources, but in addition, each of these has its own input 21 and output in terms of identifying activities that we should 22 be pursuing, End we just talked about HPIP, for instance. 23 Well, we are getting some information, some feeaback from 24 these people, generally, saying that we could improve that. 25 So, it comes down in here. Hopefully, when we're dons, we O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
l 45 l 1 will have a better tool that they will then use. () 2 3 CRAIRMAN AFOSTOLAKIS:
. bounded condition for the plan.
So, basically, this is a I mean one has to 4 undericand this to appreciate why the plan will be the way ,
- l 5- it will be Okay.
.l i' 6 DR..FERSENSKY And to run through perhaps a
/ di.$fer7,n? turmat on a specific as to how we might generate 8 an activity or set of activities or-where our activities
. 9 play,Tif there is an event, any event, that requires us to . 10 go out on an inspection, I mean that's one of the activities
- 11 that we do, we inspect. Out of that inspection comes an 12 administretive thing called an inspection report. In fact,
-13 there's other things, that content of that, which are the I 14 findings. These are the findings of that particular
() 15 inspection. 16 One way you might break it up is that there are l 17 some regulatory weaknesses -- we don't have the right tools,
- 18 our tools aren't strong enough.-- HPIP, for instance -- the t r 19 process that we have to go through needs some improvements, 20 or that we're finding ~ problems out there that say the 21 requirements don't allow us to deal with it, or there may be
. 22 problems with the licensee. 12 3 Now, these are a set of potential problems. These 24 . happen to be-the-categories within.HPIP. There may be other 25- categories. There may be weaknesses in any one or all of. II ANN RILEY & ASSOCIATES, LTD. 4 \- I Court-Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
46 1 these, depending on what the situation is of that particular 2 licensee. 3 The flow out of that is that, after you've done-4 the inspection, you may need to take some regulatory action 5 against that particular-licensee, whether ic's a confirmatory action letter or some sort of enforcement 7 y action that would be a fine or whatever. 8 You may find that, hey, there's a need for generic 9 communication, especially if you find through analysis of 10 other incidents that come out a database that we might have 11 -- say, hey, we need to tell the world, the utilities, about 12 this and come out with some sort of generic communication. 13 These are all things, activities that we do and 14 that would show up in the plan. () 15 Again, we maintain databases, we do analysis of 16 that. What comes out of that analysis may be, in fact, a 17 user need, which leado to a res, arch activity, which leads 18 to the development of a tool, which gets us back to the 19 monitoring, inspect, or it may be, hey, it looks like 20 -there's something coming down the pike and why don't we do 21 some anticipatory research?
- 22. To take the next step, just to follow it through 23 . --
24 CRAIRMAN APOSTOLAKIS:' Just a second. Let's look 25 at this. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washingtou,- D.C. 20005 (202) 842-0034 1
47 1 DR. PERSENSKY: Sure. 2 CHAIRMAN APOSTOLAKIS: The bubble there says 3 italics indicate link to the plan activities, and there is 4 no link to the regulatory weaknesses brunch. 5 DR. PERSENSKY: Oh, Those can just go on -- G CHAIRMAN APOSTOLAKIS: Oh, so the italics are 7 below. 8 DR. PERSENSKY: Yes. This same kind of thing may 9 fall out here. It doesn't end at that point. We actually 10 may do some. 11 CHAIRMAN APOSTOLAKIS: So, you didn't want to 12 imply that -- 13 DR. PERSENSKY: I couldn't fit it on the page. 14 CHAIRMAN APOSTOLAKIS: I understand. But you
) 15 didn't want to imply that the plan activities relate only to 16 licensee weaknesses.
17 DR. PERSENSKY: Oh, no, no, no. 18 CHAIRMAN APOSTOLAKIS: Okay. 19 DR. PERSENSKY: In fact, like I said, this -- 20 under this tools is, in fact, perhaps the indication that we 21 needed to follow through with HPIP. 22 CHAIRMAN APOSTOLAKIS: And possibly, you know, 23 with a good plan in place, this may change, too. 24 DR. PERSENSKY: This should all. bc fluid -- 25 CHAIRMAN APOSTOLAKIS: Okay. (\ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington,.D.C. 20005 (202) 842-0034
48 1 DR. PERSENSKY: -- all the time. 2 CHAIRMAN APOSTOLAKIS: All right. 3 DR. PERSENSKY: But this is the process that we 4 kind of go through, if you write it down. This is how we 5 -identify those activities in the plan and how they relate to 6 each other. 7- If we take this lit'tle box down here -- I-did it 8 from the research perspective, since I'm from the' Office of 9 Research. If Dick was doing this, he probably would have 10 done it from an NRR perspective. 11 But again, we have a user need, we use our staff 12 expertise that comes from that professional bubble to define 13 the problem, we look at a number of inputs, whether it's 14 from databases, the literature, experience from other areas, 15 whatever. 16 From that, we-identify what's out there, we come 17 up with our scope and approach, and if there are existing 18 tools, if we can adopt it directly -- for instance, in a reg 19 guide -- we may take an ANSI _ standard, an IEEE standard, 20- adopt it directly, just say this is for us, that's our tool. 21 We may take some other tool and make some changes.- We do 22 that with reg guides all the time, saying we like this 23 except for. 24 Another approach we might take is that there's not 25 -a guideline out there, there's not-some sort of standard,-
,O ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250'I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
, ,.imi
49 1 but there is some data that we can find through the () 2 . literature that we can use to develop the tool, with some 3 analysis and synthesis. 4 We may bring subject matter experts together in a 5 . peer panel or what to provide us some input, again 6 . leading down to thh development of a tool. 7 On the far end is, if there isn't sufficient 8 information, we may have to actually conduct more structured 9 research, whether it's field research, case study research, 10 or laboratory and simulated. These are things that we would 11 do -- this would-lead to data which would have to be 12 synthesized before we could do chat. 13- Out of that comes, again, a database. This 11-4 database may very well be -- since this is coming from () 15 laboratory things -- something that might he une HRA 16 community, because it has some real error data, and again, 17 you come into things like the technical report is a 18' professional activity, these are things that we do-to let 19 the world-know what's going on in the human performance < 2 10 areas or human-reliability areas within the nuclear 21 community. 22 So, this is the flow of how we come up with the 23 various things that are our activities and how they relate. 24 CRAIRMAN APOSTOLAKIS: A comment on this and the 25 .ptevious view-graph. This-is probably working very well for
-- ( ANN RILEY &' ASSOCIATES, LTD.
Court Reporters
.1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 .
i i._
I 50 1 individual user needs or individual events. So, I-guess () 2 what the plan is going to do now is gojng to look at the way 3 of taking a number of these and prioritizing. In other
; 4 words --
5 DR. PERSENSKY: -- identifying and then 6 prioritizing. j 7 CHAIRMAN APOSTOLAKIS: Yes. Because for example, 4 8 in the previous transparency, you say event, and-you do all 9 these things, and that's fine, but now, what if you have, 10 you know, 55 events and they lead you to different 11 possibilities and you don't have the budget to do all of-That's where the plan comes into the picture, right? 12 them? 13 DR. PERSENSKY: Right. l 14 CHAIRMAN APOSTOLAKIS: So, the plan is not really
) 15 addressing so much this individual flow for a particular
. 16 user need, although it might affect it, as you say, it's a 17 fluid thing, but.it will be a framework within which we will
- 18 integrate these activities for a number of events, a number 19 of needs, and so on.
4
- 20- DR. PERSENSKY
- Some of them may not even be 21 event-driven. I mean other drivers are also going to take '
j 22- place in here. They're not necessarily going to be an [ 23- event. I just took that as an example, i l 24 CHAIRMAN APOSTOLAKIS: Right. I understand all 25 that. The point I want to make is that the main reason for 4 ANN RILEY & ASSOCIATES, LTD. Court Reporters <
-1250 I Street, N.W., Suite 300 i- Washington, D.C. 20005 (202) 842-0034 e - , - ~ -w
. a -r-r,m, ew.-w,~:
,.,,...-,w-,-,.,..,,_.,e,-..y~w -,,-m--y. - - , ,--.,-e,e-,c- ,3%yr,,,p.
. , w - wm-e,%c,--m m y +#-,_--,-a,-sc wee -
51 1- the plan is really that there is a need at one level higher () 2 than this, integrating things, putting them together and 3 'prioritizing them. 4 DR. PERSENSKY: Right. 5 CHAIRMAN APOSTOLAKIS: Okay. 6 DR. PERSENSKY: I was going to talk a little bit 7 -about prioritization, but I think we probably spent as much-
-8 time as we need to on that a-little bit earlier. So, I')1 9 just skip right on by--that one.
10 These are criteria that we've looked at, that we 11 talked about, actually, in October, that the branch chiefs 12 have come up with. These are ti,e criteria that RES has used 13 from an AHP standpoint. 14 CRAIRMAN APOSTOLAKIS: I can't see, though, why 15 the criteria will be different depending on the method you 16 use. I mean you say criteria for absolute ratings, criteria L17 for AHP. It seems to me, even if you do AHP, you still may 18 want to consider the reduction of uncertainty as a criteria. 19 DR. PERSENSKY: _Right. I'm just saying those are-12 0 -criteria that were used in the letter that you got from 21 Kronberg, and one of the problems that was identified to us 22- is that, in doing the AHP, as you add-criteria, once you get 23-- about three or four, it becomes a much more cumbersome 24 process. So,-=that was only_the point I was trying to'make, 25 and I'm just saying that here is some information in terms O -ANN RILEY_& ASSOCIATES,--LTD. Court Reporters-1250 ILStreet, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
52 , 1 of where we are right now. 2 CHAIRMAN APOSTOLAKIS: The criteria should be
, 3 independent of the method.
4 DR. PERSENSKY: Right. 5 -CH4IRMAN APOSTOLAKIS: Right?
- l. 6 DR. PERSENSKY: Right. I'm just saying these are
- 7 the ones-that we've come up with so-far. '
8 CHAIRMAN APOSTOLAKIS: You are using the software 9 that has been developed to process AHP and all that. If I it's too big, I mean it's a simple model to -- 10 c-11 DR. PERSENSKY: We have not applied it yet, 12 CHAIRMAN APOSTOLAKIS: Okay. 13 DR. PERSENSKY: I believe the people that have 14 been working with Denny have used the software program to do
~ 15 that.
16 CHAIRMAN APOSTOLAKIS: It's like a fault tree. I 17- mean it can be very big, but still, you can run it very 18 quickly.
- 19 MR. KING: There is_-software that's being_used. I 20 ' don't know the name of it.
- 21 MR. SORENSEN: -This is Jack Sorensen. What
[ 22 they're actually using is spreadsheets to do the data ! - 23 processing. .They're not using the Expert Choice program. 2 4 _- It's the same logic, but they're not using the program 25- itself.
- -_ _ ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 l Washington, D.C. 20005 (202) 842-0034
. _ - - ~r , , , , . . , . - _ - , . ._
,,._,.,..r,_~-_.,,. , _ - - , . , , . - . - _ . , _ , , --___..,m.. , . , , _ - - - ,-
I 53 1 CHAIRMAN APOSTOLAKIS: The incredfble thing about 2 that, though, is that the weights are the -- the components 3 of the principle -- the Eigen vector corresponds to the 4 largest Eigen volume. I mean I'm impressed that Eigen
- j. 5 vectors-and Eigen volumes come into this basiness. It adds 1
6 respectability to it. ' 7 DR. POWERS: Not to AHP, it doesn't.. j 8 CHAIRMAN APOSTOLAKIS: I mean it's very impressive t i 9 -to say I found a large --
- i. 10 DR. PERSENSKY: I shouldn't have brought it up.- I 11 knew I shouldn't have brought it up.
12 MR. CARROLL: Going back to your previous slide, l 13 I'm curious about something. Nhat is national / international ! I^ J 14 efforts criteria? ' 15 DR. PERSENSKY: Whether or not there are things i 16 that are also going on that we can use to leverage our work, 17 and that's part, perhaps, of the cost-benefit ratio, is if ! '18 there are -- someono else is doing this research, we can-19 either join.them or take a'dvantage of their effort. So, if i 20 there is a wider audience, it does play a role in our -- 21~ could play a role in our prioritization. 92 MR. CARROLL: Okay, t ( 23 DR. PERSENSKY: This is the current schedule that 24 we're trying to work against. 25 CHAIRMAN APOSTOLAKIS: So, Jay, you are not I ANN RILEY & ASSOCIATES, LTD. 1 Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
. . = . . - . . - . . = - . _ - - - - - . . . . - . - - -
54 1- actually showing us today he plan itself. () 2 DR. PERSENSKY: We do not have the plan to show 3 you. 4 CHAIRMAN APOSTOLAKIS: You do not have the plan. 5 Okay. What you are telling us -- 6 DR, PERSENSKY: Whht we're telling you -- what I'm 7 telling you is what we're doing right now to re-generate 8 that plan in terms of the kinds of words we're going to put 9 in it, the kind of model or framework we're trying to use, 10- trying to get some-feedback from you, in a sense, because 11 the subcommittee and-the committee has been-relatively 12 negative on our approaches before. 13 CHAIRMAN APOSTOLAKIS: Skeptical. 14 DR. PERSENSKY: Skeptical. -Okay. That's better. () -15 So, we would appreciate some feedback before we put a lot 16 more effort into this-kind of thing. 17 IMt. POWERS: Jay, if-I.were a skeptical person -- 18 DR. PERSENSKY: You said "if"? 19 DR. POWERS: -- why wouldn't I-look at everything 20 .you presented before and say what Jay has shownime are the 21 tools I have at my disposal to put an aura of justification 22 on what I really want to do? 23 DR. PERSENSKY: If you were a skeptical person, 24 you could say that. I think that, in fact, what we are 25 doing, though -- because we started with the same () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
55 1 activities, I mean, in a sense, those activities are still (xi 2 sitting in the back of that draft.
;G 3 But in terms of trying to -- we're going to be 4 taking a re-look at each of those activities in accordance 5 with, one, the agency strategic plan, given the limitations
-6 --
there are certain really big bends, as you pointed out, 7 in the research -- but also looking at it from the 8 standpoint of what are those drivers, why is that activity 9 -here, why is it incorporated in the plan at all, is there 10 comething that we can go back to, whether it's the event, 11 the management directive, or one of those other things that 12 was in that box on drivers, what is it that, in fact, got 13 this activity -- g 14 MR. HODGES: Let me make one comment. Wayne (D 15 Hodges again. ( ,/ I have told my managemero. up through the 16 Executive Resource Board that if I can ever get a plan out 17 of Jay, I plan to make changes in what is actually done, and 18 some of the stuff we've got listed now are placeholders, and 19 they are comfortable with that. 20 DR. POWERS: I think we are, too. I mean I don't 21 think you want to go through with a brutal axe and cut 22 everything off and then start from zero in this area. 23 MR. HODGES: That's right. But I'm just trying to 24 answer your question as to why aren't we just trying to 25 justify what we're already doing. I'm saying we've got some ( ')
' ~ '
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
56 1 placeholders in there.- We think we-need to look at what () 2' 3 we're doing and_maybe change'it, and I've told them that, and they are-comfortable with that. 4 DR. POWERS: Well, I think Jay said something that 5 maybe ILdidn't appreciate.. Maybe I could come back to what 6 Lyou said. 7 On this pentagram diagram, the satanic cult that 8 you've set up here --
- 9. DR. PERSENSKY: Thank you for noticing.
10 DR. POWERS: This is a'more significant 11 -view-graph, maybe, than I appreciated. What you're telling 12 me is that?out of this pentagram will emerge.what-I've been 13- asking-for, which is this tie from the sub-strategy to the 14 activity. () 15' DR. PERSENSKY: From the sub-strategy to'the l'6 activity. The other thing, I think,'is even more-important. 17 It will' link it back to.where it:came from, what was the 18 driver for it,-why are we doing that activity. 19 DR. POWERS: I mean what I've been consistently
- 20- .for.you is what are you going to fix with this research 21 -activity or whatever it is that you're doing.
22 DR. PERSENSKY: Right. 23 DR. POWERS: And-you're telling me quit asking for 24 that, because-you're going to get it out of -- 25 DR. PERSENSKY: -You're going to get it out of O' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300
-Washington, D.C. 20005 (202) 842-0034 l
57 1 this. * () 2. HDR . POWERS: -- when the mavens get together and=
- 3. brew the pot, along with-newts, tails, and dragon eyes,
- 4. we're going to get and I will see that'there is a tie.that 5 .says, indeed, we did have a management direction, from 6 whatever source, that says the people will do root cause 7 analysis and" that they will use a: consistent framework of 8 root cause. analysis, we find our current framework deficient 9 or, if not now, it will be in the future, and therefore, 10 we're doing this activity here-to get this fix. That chain 11 will emerge out of the pentagram.
12 DR. PERSENSKY: Out of the pentagram along with
-13 the flow charts.
11 4 DR. SEALE: Could I ask another question related: 115 to.this? -Wayne indicated that, when he was in the region, 16 he had everyone take your one-week -- pardon me -- your. 17 one-day course -- 18 MR..HODGES: OneLweek. 19 DR. SEALE: Oh,.the one-week course? 3
- 20. MR. CARROLL: One' week in MORT.
21 M R . H O.". 3 3 S : I'm talking about the one-week course 22 in MORT. We tried to get all the inspectors through there. 23 DR. SEALE: Okay.- Now, more recently, you got a 24 one-day course in -- that you've had for inspectors? 25 DR. PERSENSKY: Correct. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
58 1 DR. SEALE: How long has that been in place? ( ) 2 DR. PERSENSKY: Five to six years, I guess. 3 DR. SEALE: Okay. Fine. 4 MR. CARROLL: But only recently has it been 5 integrated. 6 DR. PERSENSKY: It was a stand-alone course. Now 7 it's been integrated into a bigger course. 8 DR. SEALE: Okay. Now, you've got these 9 activities, some of which are at least a part of what this 10 committee has suggested is not cohesive enou . to be a real 11 plan or the implementation of a real plan. When you give a 12 course like that, generally at the end of the course you ask 13 the people who have taken the course what they've learned 14 and so forth. T 15 DR. PERSENSKY: There's an evaluation form that 16 they have to fill out, yes. 17 DR. SEALE: Have you gone back now, after these 18 people have had that course and have been using the results 19 of that thing for three or four or five years, and looked at 20 or asked them now, in that longer look, whether or not that 21 course has met their needs, and have you asked the people 22 who use the product that comes out of that whether or not 23 the root cause analysis that has resulted from the 24 application of those methods tells them anything that they 25 didn't know already?
\~# ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
59 1 DR. PERSENSKY: The simple answer to that question 2 is yes. We are doing that currently. 3 DR. SEALE: Okay. 4 DR. PERSENSKY: We are both going back and looking 5 at the course evaluation form, we are contacting for 6 personal interviews -- I think we have everyone that has 7 taken the course listed. We'll probably end up with a 8 sample of those people to get information based on how long 9 ago they may have taken that course, and those are the kinds 10 of questions we will ask. 11 DR. SEALE: So, you are evaluating the 12 effectiveness of this -- 13 DR. PERSENSKY: Right. 14 DR. SEALE: -- process that now has been in place 15 for some period of time. 16 DR. PERSENSKY: Yes. We are going to primarily 17 those people who have used it, but we've also asked -- done, 18 actually, an internet survey or an E-mail survey just 19 saying, if you took the course, why haven't you used it or 20 do you know anything about it? 21 So, it's been a data-gathering effort to start off 22 with, before we jump into making any changes, because we ; 23 need to get that information. 24 DR. SEALE: Have you asked the people of AEOD who 25 use inspection product material in the evaluation of events ANN RILEY & ASS 0CIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 2 84 b34
60 1 and so forth whether or not the product of inspection and 2 other assessments is a more useful product now in terms of h 3 identification of root cause and so on than it was five 4 years ago? 5 DR. PERSENSKY: We are going to AEOD people and 6 asking them questions. I don't know that that's a specific 7 question. AEOD, in fact, has not been a major user of HPIP. 8 They have had -- 9 DR. SEALE: But they've used the product of the 10 people who use HPIP. 11 DR. PERSENSKY: They use the product. 12 DR. SEALE: Yes. 13 DR. PERSENSKY: I don't know that, in doing that, 14 they've actually said, oh, gee, here's one that has HPIP in 15 it and here's one that's not. 16 DR. SEALE: Well, but you need to ask them what 17 they're talking about. 18 DR. PERSENSKY: Right. You may find more of that 19 in NRR's HFIS database, because the NRR people do use HPIP, 20 so they see -- they may see that if we went back and asked 21 the question. I don't think, again, they've ever asked that 22 question of the database, or whether we could, but that's a 23 possibility. 24 CHAIRMAN APOSTOLAKIS: Can we go back to the 25 schedule, the transparency with the schedule? ANM RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
i 61 11 DR. PERSENSKY: Sure. This is a very' aggressive-2- . schedule. 3 CHAIRMAN APOSTOLAKIS: I.am a little bit 4 . uncomfortable -- I don't know how my colleague feel about 5 .this, but we will really see the plan in March, and then'we 6 will.have.to write a letter, I guess,.in April.. I would 7 like to have a~little more time to see the plan and review 8 it and discuss it with you, and you know,-I don't know 9 what's going to come out of that discussion, so you should l
-10 have a little more time to go!back and maybe work on the
-11 plan a little_more, because the way we're going now --
12; again,'we didn't see the plan today, we're going to see it 13 maybe two months, and.that essentially will be a review of almost a finished-product, because if we.go to the full
~
1<4
-15 committee in April, essentially there will be a period-of-16 -maybe two or'three weeks.
- 17. DR. PERSENSKY: So, you're asking for a 18 subcommittee more in April and a full committee in May.
19 -CHAIRMAN APOSTOLAKIS: Yes. Either we have a 20 subcommittee meeting in the February-March period, where you 21- are really-putting-everything together, or we-leave it as-- 22 -is,- but the full ACRS will have to be moved, say, to May or 23 June, I don't know. 24 DR. PERSENSKY: I'm all for extending the 25 schedule.
? ANN RILEY & ASSOCIATES, LTD.
Court Reporters j 1250 I Street, N.W., Suite 300 ); Washington, D.C. 20005 (202) 842-0034
62 1 MR. KING: We could do that, George. You'd like a O] q 2. subcommittee to review the plan. 3 CHAIRMAN APOSTOLAKIS: I'd like a subcommittee to 4 really review the plan. It doesn't have to be a finished 5 product, but you know, something that -- like you guys did 6 in the last full committee, you know, and maybe you can even 7 have some questions and say we are thinking of doing this, 8 what do you guys think, that kind of thing, but that 9 requires some time. 10 We would hate to see, again, a finished product 11 and, if we're uncomfortable with it, again, to write letters 12 -- I don't want to do that. I would rather make sure that, 13 you know, we are all agreeing with something. 14 So, it seems to me right now that the easiest way b N_/ 15 would be to say draft plan to ACRS mid-March and then the 16 full committee meeting will be maybe May or June, depending. 17 on the reaction of the subcommittee and what you think and 18 so on. 19 MR. KING: Okay. That's reasonable. We'll plan a 20 subcommittee in April, after you have the plan in hand to 21 look at. - 22 CHAIRMAN APOSTOLAKIS: Yes, that's a good idea. 23 What worries me is, you know, that you may not have enough 24 time to go back and work on the plan again after the 25 subcommittee meeting and we would have to write the letter, () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
63 1 you know, before you have a chance to do that. I think that 2 would be best. 3 MR. KING: Okay.
-4 CHAIRMAN APOSTOLAKIS: .Okay. So, maybe the full 5 ACRS would have to be moved.
6 One other thing. We keep talking about operating 7 experience and so on. I really think that -- well, first of 8 all, a lot of-the thinking that goes into the plan here-and 9 the factors that you take into account and so on, obviously, 10 in-an hour-and-a-half presentation you don't have time to 11 prerant to this committee, so sometimes you get comments - 12- because we are not aware of some of the things that you guys 13 'have done, and in fact, today, showing that bubble and the 14 activities and the constraints, I think that's very-15 instructive. 16 .I really think, judging from past-subcommittee 17 meetings, that this committee appreciates real data. So, I
- 18. would really like, in'our future subcommittee meeting, to 19 have someone up there saying,-look, this is what we learned 20 -from analyzing data in the last five-years or whatever, 21 these are_some generic insights, and here are two or three 22 incidents'-- you are familiar with the papers of Zack Pate,
- 23. the former president of INPO -- something like that, and 24 come back and say, now, our plan is addressing these things 25 this way.
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
64 1 I think this committee or subcommittee would O) (, 2 really appreciate that, and maybe you have already taken 3 those things into account, but we don't see them. 4 DR. PERSENSKY: I think, to some extent, the 5 presentation later by Gene Trager is going to talk about 6 what has been found from some of the work that AEOD has done 7 in terms of how we incorporate that in the plan as the next-8 . step. 9 CHAIRMAN APOSTOLAKIS: Okay. Well, that's what I 10 really want. 11 DR. PERSENSKY: Okay. Yes. 12 CHAIRMAN APOSTOLAKIS: And I plan to ask that 13 question of Mr. Rubin, too. I was looking at the inspection 14 guidance here, and that's what I was trying to do. I said,
) 15 well, gee, let me take now this incident. Where would the 23 inspector catch it before it occurs? See, that's the real 17 test. Is there a step there that, you know -- or have we 18 changed the procedure now because of the thing that 19 happened? Is something, you know, escaping our attention?
20 I think this is a reality check that really is very 21 valuable. 22 DR. PERSENSKY: And I think when we get into the 23 next_ step with the plan, we can do that. 24 Now, I can also caveat that with saying that any 25 one incident or any one inspection report is not necessarily [ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
65 1 going to lead to a research project or something else, but 2 it's going to lead to -- it will lead to some other h 3 activities that, again, taken as a whole, those people in 4 the human performance area in the agency do. 5 I mean most of the work that NRR does, in fact, is 6 culminated, in a sense, with an inspection report. They go 7 out, they do inspection, they do the inspection report. 8 CHAIRMAN APOSTOLAKIS: Right. 9 DR. PERSENSKY: But that's what they do. Thac is 10 their activity, and it ends there. It's only when we get a 11 collection of information that it goes on to some other jl[ 12 point. 13 CHAIRMAN APOSTOLAKIS: And the question, then, in 14 my mind, is, if there is an incident that acq' tires some 15 notoriety, why did it happen, why didn't the inspection 16 catch something beforehand? I mean I would like to 17 understand that. There is such a thing as, you know, things 18 happening randomly. I mean it's not a perfect system. 19 MR. CARROLL: Stuff happens, George. 20 DR. POWERS: Well, George, it seems to me that in 21 the same -- 22 CHAIRMAN APOSTOLAKIS: But that presumes -- excuse 23 me -- presumes that it's really aleatory, and I am not 24 convinced. 25 Dana? ANN RILEY & ASSOCIATES, LTD. , Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
66
'l DR. POWERS: It seems to me that there is another
() 2 step, and that is that, having gotten a collection of 3 inspection reports and asked the question of, gee, why are 4 these things consistently happening, you have to have the 5 step in there that says, and it's unacceptable to have this 6 consistently happening. , 7 DR. SEALE: If I may, the thing that Zack Pate did 8 _w as to collect,_ if you will, a set of control room 9_ activities by operators and not only identified these but 10 then made the case that these were unacceptable and, i 11 furthermore, allocated the responsibility for corrective 12 action at levels other than-just the operators who had 13 actually been the perpetrators, if you will, of the-event. 14' - You indicated earlier that there is a possibility
) 15; that your branch may be merged with the operator --
-16 DR. PERSENSKY: No,.that's in NRR.
17 DR. SEALE: I'm sorry, in NRR. 18 DR.=PERSENSKY: Right, l 19 DR. SEALE: That's an interesting marriage to me 20 for the followino reason. A lot of the things that happen 21 in the human factors area that are a problem are not 22 operator activities, they're maintenance activities. 23 Now, it's clear that it's easier to identify, 24 apparently-- and it's been done, at least at some level 25 - the things-that have involved operator error -- Zack ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1250 I Street, N.W., Cuite 300 l Washington, D.C. 20005 (202) 842-0034
67
=1_ Pate's letter, if you will. ] ) 2- Is there a similar collection of things that show 3 where the difficulties in human performance, in a cohesive 4 way now, that have occurred in these other areaslof 5 responsibility have occurred and an attempt, then, to look 6- at, you know, where do we go to find the appropriate stimuli 7 for corrective action?
18 DR. PERSENSKY: That collection- if it is l
-9 anywhere, would primarily come out of inspection reports 10 that have addressed other than operators and'out of the
, 11 various databases we have, particularly the LER database. 12- One of the shortcomings and one of the problems 13 that human factors has had since it's been in the agency is 14 the rule that directs what is to be put into an LER. We get 15 very'little human performance information in our LER, so 16 it's hard to make that comparison.- ! 17 One of the reasons that AEOD took the approach 18: 'that they did-in the HPED program, where they went out and 19- did followups, was just that purpose, to see if we could 20 find out more about what really happened to do that. 21 We also get some of that information from the more.
-22 in-depth kind of investigation, IITs and AITs.
-23 As far as actually trying to coalesce those or 24 collate them and try to draw a lot of-insights, that
-25 activity, if it's happened, has happened in AEOD, it has not O ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
68 1 ' happened in Research. () 2- DR. SEALE: .I-guess what I'm saying is that I 3 think I now know what's been a useful'way.to take the'next 4 step'in the licensed operator-arena, at least to a degree 5_ -that-Zack Pate's-paper and so on did-kind of systematize
=6 some of the concerns with respect to licensed operators.
Somebody needs_to take the next step with-regard 8 to these other activities in the plant, and I don't know, 9 maybe that's even a research activity somebody ought to bite - 4
- 10. the bullet on, or whoever. If there were industry people _j
'11 ' -here, I'd; challenge them to -- maybe that's an appropriate 12 thing for-Zack Pates-to do, you-know, that kind-of thing.
- 13 DR. PERSENSKY: Well, you-have to remember that 14 Zack's paper was also sort of_ driven, in a sense, that-part
) L15- of the~ reason for that was'the NTSB report.
16' DR. SEALE: Sure. 17- DR. PERSFNSKY: 'I mean NTSB did' exactly that. 118 They looked ac a series of incidents. Now, they have a much I
-19 smaller, in a sense, subset, but he used that as the kickoff-20- for his particular paper.
21 DR. SEALE: Now-let's use his thing as a kickoff i 22' -for ours, 23- MR. HODGES: I can't go into much detail because
..24- it-is-still being kicked around and I don't know all-the 25 details, but'I know that AEOD is working on this plan to -() ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
69 1 basically assess all the information that comes through for () 2 the senior management meeting and trying to look at risk 3 significance, and we're worried about those things that will 4 be predictors of performance ar well as, you know, well, 5 these are the bad things that happened, and Research is # 6 working with them in trying to develop some of that, but 7 we've got a ways to go yet. 8 MR. CARROLL: Is improving t' collection of human < 9 performance data through the LER process one of the 10 activities that was on the list? I can't remember, 11 DR. PERSENSKY: That is one of the activities that 12 was on the list, and as I understand it, in the operating 13 plan for AEOD, there is actually some resources being put 14 towards doing that at this point. 15 MR. CARROLL: Why don't you-just go do it? 16 MR. ROSENTHAL: This is Jack Rosenthal, AEOD. 17 MR. CARROLL: This has been kicking around -- when 18 I joined the committee eight_ years ago, we were strongly 19 urging you to do something about that. 20 DR. PERSENSKY: I'll let Jack take the ball here. 21 MR. CARROLL: What have you been doing for nine 22 years, Jack? 23 MR. ROSENTHAL: We made several attempts to 24 improve the information collection. One has to analyze the 25 -data and make a case and go through a backfit process to [ \ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 1
70 1 show that you need the data for regulatory decision-making s 2 and you just can't go out and just arbitrarily collect data. 3 So, it's been very difficult to make the case, and I think 4 that we're now amassing enough data to make the case that 5 there is a regulatory need. 6 MR. CARROLL: When do you believe a rule change 7 will be promulgated? 8 MR. ROSENTHAL: It's in the operating plan -- it's 9 about two years out -- to change 50.72/73. We just put out 10 revised guidance in NUREG-1022, but then we're going to loop 11 right around again and now go to -- attempt to change the 12 rule itself. 13 MR. RUBIN: I'm Stu Rubin, acting chief of the 14 Human Factors Assessment Branch. I ( ,f 15 With regard to the LER rule and the guidance that 16 goes with it, in this past six-month period, NRR, in 17 correspondence to AEOD, urged AEOD to include much-expanded 18 guidance in the area of human factors considerations in the 19 reporting of events in the LER rule, and much of that 20 expanded guidance, at least the framework, was drawn from 21 the HFIS very extensive human factors breakout of human 22 performance, causes for human performance problems, and the 23 status of that guidance change, as I understand it -- it's 24 basically gone through all of the steps in the process, 25 including review by CRGR, and there has been acceptance of ()
\/
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
71 1 that expanded' guidance for reporting human factors b3 hind. () 2 human performance problems and evente.
-3 DR.'SEALE: I would remind the subcommittee that, 4: in the December meeting, the full committee received a-5 briefing on the NUREG-1022 event reporting guidelines 6 changes, and the committee had recommended to the staff that 7 the proposed changes in rulemaking or -- pardon me -- in 8 . event reporting guidance be incorporated in the revisions 9 that are going out, anticipating the changes two years hence 10- or whenever.
11 I just got a letter back from Mr. Callahan in 12 which he has indicated-that the committee's suggestions-in-13 that regard had been accepted and will be included in that 14 guidance that's going out now. 15 So, that'part of the loop is.being closed. 16- MR. MOSLEH: As a clarification, isn't it true 17 that a user need is the boundary condition or the initial 18 condition for your activities, that-you don't generateLthe 19 issues?- 20 DR. PERSENSKY: We have freedom to do both. In 21 the past, most of our work has been aimed at high-priority 22L user needs. We=are working-more towards doing-things that 23 are anticipatory,--if we can identify it, so we have some 24- small portion of our budget that we can do that kind of 25 work. O' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
72 1 Generally, though, it's not basic research that, () 2 you know--- it's generally still applied, looking at things 3 that--we anticipate's going to-come about-in the 4 not-too-distant future. It's not long-term. l 5 MR. MOSLEH: And you have a mechanism through 6 which you can identify potential problem areas. 7 DR. PERSENSKY: Right. And generally, that comes 8 from things like outside -- from other applications. We see
-9 problems that are coming up with the use of digital I&C, for 10 instance, in the aeros.nace industry or the petrochemical 11 industry or something other place. We may be able to 12 identify where there are some specific problems, given that 13 the movement in the industry is towards the adoption of 14 digital I&C, that we can forecast some potential problems,
-15 Any other questions?
16 CHAIRMAN APOSTOLAKIS: That's it? 17 MR. CARROLL: Well, I'd just like to say one-more
-18 time-, I really believe that you need to come to grips with-19 the problem of how many bucks-go to human factors. I think 4 20 it's been too few over the past years, and I think that's
- 21. .true of each of the divisions that we're talking about but 22 particularly about Research, and 1 just encourage you to-23 really look at this very carefully. It seems that there are 24 certain things that.always get all the money they want.
25 CHAIRMAN APOSTOLAKIS: Like thermal hydraulics? O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
73
-MR. CARROLL:
-1 Like thermal hydraulics. Ivan,
() 2 where are you? q 3- CHAIRMAN APOSTOLAKIS ' -There's a reason. Becituse 4 they don't understand the issues there. 5 MR. CARROLL: I think human factors is important. 6- CHAIRMAN APOSTOLAKIS: I think it's a fundamental 5 7 case:of bias. I mean most of the guys who make the 8 decisions have studied thermal hydraulics. 9 DR. KRESS: -There's real science there. 10 MR. CARROLL:- And it's more fun. 11 DR. MILLER: I think you've got it. It's much g 12 more fun, and it's easier to get your arms-around. 13 DR. POWERS:. Jay, one.of the problems that I see 14 with this is that it is relatively easy for the thermal 15 hydrolysis to demonstrate clearly that there is a demand 16 within the regulations to study-thermal hydraulics and to be ^ 17 able to calculate and evaluate thermal-hydraulics -- 18 DR. KRESS: It's driven by the way the regulations 19 are written, you're right. 20 DR. POWERS: -- and that they can clearly show 21 that the deficiencies in the treatment of thermal hydraulics
~
22 are_ unacceptable. 23 The problem that we have consistently had from the !
- 24 time this plan was first put forward is that there is not t
25 now and there was not in the past a clear connection on what () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I btreet, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 !
74 1 it is that we want to fix and why it has to be fixed. () 2 To be brutally honest with you, we've gone about 3 nine months now with the assurance that, within six weeks -- 4 so, it is getting to be a shorter time, I'll have to admit 5 -- that we'll see tilat connection. 6 DR. KRESS: That nine months is about time to have 7 a baby delivered. 8 DR. POWERS: It depends on what happened just 9 recently. 10 CHAIRMAN APOSTOLAKIS: We have scheduled a-11 half-an-hour, which can be longer, at the end of the day 12 where we'll discuss these things, and I hope that you 13 gentlemen will be here. 14 One last thing. Can you make sure that, next 15 time, the hard copy is stapled in the right order? It's so 16 inconvenient. 17 MR. CARROLL: Yes. 18 CHAIRMAN APOSTOLAKIS: Thank you very much. We'll 19 meet again -- 20 MR. CARROLL: There's another problem, too. 21 CHAIRMAN APOSTOLAKIS: Okay. 22 MR. CARROLL: He needs to learn how to spell 23 " anticipatory." 24 DR. PERSENSKY: We need it in spell-check, is the 25 problem. [\
\~
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
75 1 CHAIRMAN APOSTOLAKIS: And the title, " Human L i 2 Reliability Implementation Plan." " Implementation" is not
\_/
3 there. 4 DR. PERSENSKY: " Implementation" is not there. 5 MR. CARROLL: I don't think it should be, George. 6 CHAIRMAN APOSTOLAKIS: But that's what they use 7 inside. 8 MR. CARROLL: Is the plan implementing human 9 performance? 10 CHAIRMAN APOSTOLAKIS: Well, that's what I asked 11 earlier. If you go to view-graph -- 12 DR. PERSENSKY: I meant to take " implementation" 13 out of that, and I did not. 14 CHAIRMAN APOSTOLAKIS: -- number nine -- fb () 15 DR. PERSENSKY: I made a mistake. I made a human 16 error. 17 CHAIRMAN APOSTOLAKIS: Okay. So, the "I" will be i 18 removed, then. 19 DR. PERSENSKY: The "I", as far as I'm concerned, 20 will be -- 21 CHAIRMAN APOSTOLAKIS: Okay. And now we'll have 22 to look for a stapler. 23 We are scheduled to start again at 10:45, but 24 since we are ahead of schedule, maybe we can start a little 25 earlier? Is everybody here? How about something like [\' ) ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
76 1 10:35. . p. () 2 [ Recess.] 3 CHAIRMAN APOSTOLAKIS: Let's start. 4 Stu? 2 5 MR. RUBIN: Yes. Good morning. I'm Stu Rubin. 6 I'm the acting ch ef of the Human Factors Assessment Branch ' 7 in NRR, and I'm pleased to be with you this morning. 8 In last year's October ACRS meeting, the chairman 9 requested the staff to come back and discuss with this 10 subcommittee how the staff integrates human factors 11 considerations into the reactor inspection program and how 12 human factors are covered in inspector training as well as 13 inspection procedures and the guidance that inspectors use, 14 and this morning, we're going to try to arswer scoe of those tO V 15 questions. 16 I should point out that, since consideration of 17 human facters and inspections involves a closely coordinated 18 effort between headquarters and regional staff activities, 19 Glenn Meyer,-who is the chief of the Operator Licensing and 20 Human Performance Branch in Region I, is also here today to 21 participate in the briefing and provide a regional 22 perspective on these issues. 23 First slide, please. 24 I'd like to begin with a quote from the NRC 25 strategic plan in the reactor safety strategic arena. A
- ANN RILEY & ASSOCIATES, LTD.
\-
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 i
77 1 sub-strategy _ is _ included that stat.es that licensee 2 operations and activities.will be inspected to help ensure 3 that licensees identify and resolve problems before they may I 4 degrade-safe operations, and.so, the strategic plan, we ' 5- believe, makes it clear that inspecting for human ) 6 performance issues at reactor facilities is an-important 7 strategy for ensuring adequate public health and safety. 8- Next slide, please.
-9 Now, because human factors consider &tions are in 10 most elements of the inspection program, we thought it might 11 be hulpful to first provide an overview of where and when 12 human factors are included'in-this process and then to take-13 -a closer look at each of the elements = separately, and so, - .14 I'd like to go through that overview now, and then we'll 15 come back and spent more time on the hows within_each of 16: those elements.
17 - The first element -- excuse me. -Oh, yes. I've. 18 been reminded that we've incl Med'a-duplicate of-this ; 19 'particular slide'in the back of your handout that you can-20 tear off, and you can follow along when we get.into the 21.- individual slides, which have been letter-labeled to the 22 same lettering as each block. So, you can kind of follow 23_ along. 24 CHAIRMAN APOSTOLAKIS: I've never seen that 25- before. That's great. ANN ~RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034-
4 78 1 DR. POWERS: This is a human factors dynamo here. () 2 MR RUBIN: I also hope we've stapled the package 3 in the right corner. 4 DR. SEALE: I detect a suggestion that this 5 committee is not only trainable but educable. 6 CHAIRMAN APOSTOLAKIS: But you are giving us a 7 presentacion that's a year old,_ right? 8 MR. RUBIN: It's a continuously updated process. 9 CHAIRMAN APOSTOLAKIS: But if you go to your first t 10 view-graph, the first slide says 1997, 11 DR. POWERS: And we think just have a year 2000 12 problem. 13 MR. RUBIN: Okay. Sorry about that. Back to the 14- future, t \ (_/ 15 DR. POWERS: Don't worry about-it. He's just 16 trying to put you off, 17 MR. RUBIN: The first element of the process 18 labeled as block A is the formal plant performance review, 19 or PPR process, that's conducted semiannually out in the 20 regions for each plant. Review of human performance is 21 explicit within the PPR process for each plant, and an 22 output of this review process are revisions to the 23 integrated inspection plan for each facility, including 24 planned inspections that are intended to assess human 25 performance issues that are identified in that rev_ w rn
) ANN RILEY & ASSOCIATES, LTD.
, Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
79 1- process. () 2 Above block A, in the block labeled G, which-3 represent the-headquarters staff, are daily reviews of event 4- reports and inspector reports. -This is done to try to 1 5 : identify potential safety issues, including human 6 pe.formance issues that may warrant follow-up action,.
- 7. . including inspection-at the. facility.
i 8- For example, an inspector orally reporting that.
.9 there was a high rate of failures on a.recent. operator
-- 10 licensing exam when taken together with either indications 11 of. plant' training program problems from other databases 12 might result.in a decision to conduct a for-cause training 13 inspection at the facility.
14 Now, following the regional-reviews and the daily 15 headquarters reviews, the-procedures and the guidance-for g 16- actually-implementing inspections-that are decidediupon-is-
~
l17- -documented in the reactor inspection program procedures and 18 are shown in blocks B-1 and B-2 for'the core inspection 19- -program and the regional initiative and reactive inspection
=20 -programs-respectively.
21 The boxes along-the bottom that are labeled-F-1, 2 ,- and 3 represents the knowledge, skills, and abilities of 23 NRC inspectors for actually assessing human factors in 24 ' conjunction with these procedures:and guidance. 25 This includes both the headquarters and regional ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
80 1 inspector KSAs, as well as the expert KSAs, as I like to 2 call it, of the Human Factors Assessment Branch staff. 3 MR. CARROLL: What is a KSA? 4 MR. RUBIN: Knowledge, skills, and abilities in 5 human factors assessment. / d We also have specialized centractor support if 7 there is a necd to bring that kind of expertise in. 8 The inspector classroom and OJT-type training to 9 develop these KSAs is shown in box E, at the lower left 10 corner of the chart. 11 Moving back to the center of the chart, in box C, 12 which represents the actual human factors findings that are 13 documented in formal NRC inspection reports and then 14 downstream and to the right of that are the boxes labeled 15 D-1, 2, and 3, which are NRC activities and products that
- 16. stem from the human factors findings that are documented in 17 NRC inspection reports.
18 D-1 is the coding of inspaction report human 19 factors findings into NRC's human factors information system 20 or HFIS catabase. D-2 is the preparation and issuance of 3 21 generic feedback, correspondence such as NRC information 22 notices of human performance issues that are gleaned from 23 these inspection reports, and D-3 are the improvements that 24 we make in inspection procedures in the area of human 25 factors that would result from insights and lessons that we ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
81
?c llearnedasa-resultofwhat'scontainedininspection
() 2 3-reports and the conduct of inspections, as-well as other assessments that m.ght-provide for improvements. 4 MR. CARROLL: On D-2, how many generic 5 communications on human factors were issued during 1997? 6 -MR. RUBIN: I don't have the exact number, but I 7 would say that, in the last six months, there were two that 8 come to mind. 1
-9 One relates to the crediting.of operator actions, 10 manual accions ia place of formerly automated action, 11 1 automatic actions,-as a result of equipment degradations 12 that are found by design reviews or equipment malfunctions 11 3 and that sort,-and the intent there -- and some of those 14 insights were gained from previous inspections at plants,_
) 15 and the feedback was to explain to industry _the expectations
'16 that NRC has for licenseet to review crediting operator 17 actions of that sort in terms of looking at-what procedure 18 is-available, what' training is available, the actual 19 environmental' conditions that the operator would be subject 20 to, and-laying that out, and that's one example.
21 3A more recent example, which I'm not sure has gone 22 out the door, is we've seen recently some cases where 23 = individuals who are sitting for -- had actually sat in on a 24 operator licensing examination were found subsequently to 25 not be: eligible to be seated for-that kind of exam because / ANN RILEY'& ASSOCIATES, LTD. Court Reporters 125n I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034=
82 1 they did not have a sufficient level of prior errerience () 2 .related to the plant or formal education, and so, we took 3 those insights and we put together an information notice to 4- _ feed 1that back to industry. 5 So, those would be two recent examples. 6 MR CARROLL: That's two out of how many generic 7 communications in general? 6 MR. RUBIN: I haven't done a sort. We could 9 provide that information if you're interested. 10 MR. CARROLL: Is it 10 or 100? 11 MR. ROSENTHAL: About 100. 12 MR. CARROLL: About 100. So, human factors really 13 icn't very important, is it? 14' MR RUBIN: No, 7 don't think I would say that. A - ( 15 lot of it, as we can discuss later, comes about as a result 16- of what, in fact, you find and report and chen can.have 17 insights for to actually issue an information notice. So, 18 your end product is only as good as the process that 19 precedes it. 4 20 Okay. The other thing l'd like to mention, we do 21 make, as I said, improvements to the inspection procedures 22 and guidance themselves, and then the arrow to block A 2?- indicates, as I noted earlier, inspection report findings in 24 the human factors area are fed back to the PPR process
'25 itself as one of~the basic inputs to the. review process, and
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
83 1 so, the NRC's inspections of each plant in this area -- that () 2 is, human performance inspections -- are not only continuous 3 and ongoing, but there is continual adjustment, if you will, 4 based on the most recent inspection findings in this area 5 that come out of inspection reports. 6 Looking up in the upper righthand corner in box 7 H-1 on the far right, the most recent human factors findings 8 in the HFIS database are periodically analyzed and sent to 9 the regions for use in both PPR process in block A and also 10 used in connection with the semiannual plant performance 11 screening meetings that lead up to the senior managers 12 meeting. 13 And finally, the HFIS inspection data that's 14 compiled there is periodically sorted and analyzed to 15 identify whether there are plant-specific issues that might 16 warrant a inspection, and I mentioned one, a training-type 17 inspection, if we do see probleme in that erea, and also, we 18 provide assistance to individual inspectors in planning 19 their inspections to allow them to see where there might be 20 areas of special concern that bear on human performance. - 21 That kind of covers the overview, and what I'd 22 like to do now is start the closer look at each one of those 23 boxes and turn it over to Glenn, and he will be covering 24 block A, L-1 and B-2, and then C, which is really a key core 25- -to this whole process; and when he's done, he'll hand it () ANN RILEY & ASSOCIATES, LTD. Court Reporters 'j 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
4 t 84 6 1 back to me, and then I'll cover the rest of the blocks, and () 2_ so, we'll see a little more detail of the hows that all 3 ' these are conducted. l 4 If there aren't any questions on this simplified 5 diagram, I'd like to turn it over to Glenn so we can get l 6 into the details.
- 7 MR. MEYER: _ Good morning. I'm Glenn Meyer. As 8 stated, I have been for the last six years the branch chief
. 9 in charge of operator licensing, and generally, I've spent 10 over 16 years:in Region I, including.being a senior resident 11 inspector. So, I believe I can prearnt a regional : 12 perspective. 4 13 MR. CARROLL: What did you do before that? 14 MR'. MEYER: Seriously? i 15 MR. CARROLL: Yes. What's your background in ' 16 general? i 17 MR. MEYER: I have a unique combination of -- I
- 18 was on' Admiral Rickover's staff. I designed reactor i 19~ components for the Navy for five years, and after that, I 20 l spent five years in the hamburger business, doing restaurant 21 operations, . construction, training, hiring, firing, a whole 22' range of things, So, I like to refer to that as my 23 operations phase.
24' MR. CARROLL: _Why did you go back to nuclear when 25 you were doing something useful? ANN-RILEY & ASSOCIATES, LTD, Court Reporters _ 7 1250 I_ Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
-> my *-zg e-'e-wr y
-=w'k4 1i,. -wan. -m -. -w -----r-y. --yfem,+-'3.-+-gy-r--ig-.p g -*-,wgpqgv.-gy..wy9,.< w.yy-~m,,.., w. y g y gn- sywve tsv ee yy-- - * ' '-~--4 =-9 N'-' g-*= - ' - -
m._._..___.___._-. . . . _ _ _ _ _ _ _ _ . _ _ __ _ 85 1 MR. MEYER: Good question. I found it hard to () 2 apply the combination of nuclear and hamburger into the next 3 phase. 4 DR. POWERS: Wait-till the new generation of bug 5 killers comes out. 6 DR. MILLER: Irradiated foods. 7 MR. MEYER: I do find that the people were able to l 8 get just as excited about a multi-million-dollar reactor 9 vessel as they were about the hamburger they were just i 10 served. I ! 11 DR. SEALE: There may be opportunities for you in I 12 Texas about now. 13 MR. CARROLL: What's your education? 14 MR. MEYER: I received a Bachelor's in chemical 15 . engineering from Cornell University as part of my Navy l 16 commitment and then served with Rickover. 17 MR. CARROLL: You could have a worse major. 18 MR. MEYER: The question I'm here to help with is 19 how does the NRC integrate human factors considerations-into P
. 20 the inspection process, and I would say, in a word, 21 cautiously. I think there's basically a three-pronged f
22 approach. 23_ The first one is evaluation of inspection results 4 i 24 for human performance problems. i 25; Why don't you put the simplified one up? 1 4
- , ANN RILEY & ASSOCIATES, LTD.
Court Reporters-3 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 3 (202) 842-0034
-,-v.-r-.--vr _wv.,- e--w-s r ow-w_r--- -a.- --+ s- --e--wmr=mr-rw
, -r-we-e m- vr y v - +--wre----,*rw- y r- -,v- '
i i 86
- 1 And basically, to a large extent, at the regional
() 2 level, this occurs in block A, PPR, review. the plant _ performance, or 3 4 The second prong is a sensitivity to human 4
-5 performance-concerns-in inspections.- There's-hundreds of 1
j 6 inspectors day to day that are looking at various aspects of i
?- power plant-operation, and they're sensitive to human
! 8 performance, and so, we're going to talk about blocks B-1 s 9 and B-2, where we have a core inspection program. We also l - 10 have regional initiative and reactive inspections that can ! 11 be more specific to problems. 4 12 And then the third area is selective use of human 11 3 factors specialists and expertise, and that's the periphery 4 14 that Stu will be talking to about training and databases and i 15 analysis and things such as that. . 16 So, at this point, I'm going to proceed to talk 17 about the PPR, the inspection program. Are there any topics - 18 or questions that you would want me to particularly focus 19 on? 20 CHAIRMAN APOSTOLAKIS r We'll see as we go. 21 MR. MEYER: Okay. , 22 My perception is that there is a continuing , 4 23 struggle to evaluate human performance in an efficient,
~
24 effective manner at the regional-level, okay? Like I said, > -25 I've been there for 16 years, and it's not easy. l- ANN RILEY & ASSOCIATES, LTD.- )
. Court Reporters J l 1250 I Street, N.W., Suite 300 l 4
Washington, D.C. 20005 1 (202) 842-0034
1 i 87 i But the backbone of our approach now is to use a () 2 performance-based inspection approach that evaluates results ' 4 3 and events and then assesses the human performance behind , 4 those results. 5 So, it's not a direct inspection of human j 1
- 6' -performance but more so a-performance-based review of the j f 7 events and the results that are occurring.
8 Why? A broad-based direct human performance ~ i 9 evaluation tends to be ineffective. It's much easier to i i 10 draw the conclusions from the results that they've 11 accomplished. And the second reason is, even if you're { 12 correct when you go in to evaluate human performance, the !
- 13 facilities are unlikely to change unless you can-tie it to 14 results to show why the performance has been bad or is. going
() r 15 to cause bigger problems. I 16 MR. CARROLL: When you talk about plant j 17 performance review, it's much broader, though, than just , I a 18 human factors considerations, is it not? 119 MR. MEYER: Definitely, Definitely. .I mean it j 20- would be hard for me to come up'with plant performance areas i j' 21 .that didn't have human performance in them. So,-it's , i
?
- 22 frequently there. Most-of the time, it's not solely and 23 -exclusively human factors or human performance.
24 CHAIRMAN APOSTOLAKIS: Excuse me. I'have a 25 question. I read the inspection manual that was sent to us, J ]- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 i Washington, D.C. 20005 , (202) 842-0034' i
,-,--..,a_.,,..~.--
, , - - - , . . . . - . _ . - . . . - - - - . . . . _ , - , , , . , ~ ~ - , - - - < . - - - - - -,
I 4 88 l 1 program applicability, 2015, and you mentioned that it's ,QO 2 performance-based. I guess, in a sense, it is, but it <= 3 depends, I think, on what you define as performance. 4 This seems to be too detailed. I mean is that a
- 5 - wrong impression? What are the-performance criteria that ,
l 6- have been defined here against which you measure or evaluate
- 7 performance? Because the inspectors really look at a lot of ;
i
- 8 details, like cleanliness is being taaintained, ancillary 9 equipment such as scaffolding, ladders, and so on, does not 10 interfere with system performance. I mean you are really l 11 going down to detail here.
32 DR. MILLER: It doesn't seem like there's very 13 much prioritized of what's detailed and what's , 14 performance-oriented. 15 CHAIRMAN APOSTOLAKIS: The idea of 16 performance-based regulation is really that you will have 17 -the performance criteria-at a relatively high level, you 18 will not really inspect every little detail. I mean it is 19 performance-based, because you can see, look, performance is 20 bad because a ladder is in the wrong place, but is that 21 really an important thing? 22 Have you thought about it, how to prioritize these 23 things and perhaps reduce the number and make it really , 24- performance-based according to the current thinking as to 25 what performance is supposed to be? l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
l l 89 { 1 MR. MEYER: Let me segregate performance-based ! 2 regulation which, you know, I would say at this point is an 3' . objective that the Commission is trying to go towards. Most l 4 of the regulations we don't have are necessarily 5 performance-based, but the nature of inspection is what are 6 you evaluating. Are you evaluating the organization, the 7 procedural approach? It tends to be a lot more effective to 8 evaluate the performance of the people. . 9 If it's engineering, the performance would be the i 10 engineering reviews, the modification designs, that sort of i 11 thing, rather than to look at who is assigned and the 12 procedure to be responsible, what's their training, all that - l 13 kind of thing. ! 14 In my area of operator licensing, for example, ; l 15 communications -- corrnunications is a very important aspect i 16 of what the operators do. Are there criteria that we can ' t 17- clearly evaluate them to? I would say probably not. 18- Now, INPO has a standard. They have.three-point 19 communication, where they would want one operator to say 20 what-should be done, the second operator to respond back,
. 21 and for the first person to acknowledge that that's correct.
4 22 The NRC has no-such guidance that I'm aware of,
- 23 but when we go in to do'the operator licensing exams, we do
- j. . 24 evaluate how well they communicate.but on the basis of-can 25 they perform their function, is the performance of what they ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250-I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
90 1 do. () 2 If they are -- if we have a crew of operators and 3 -one operator is unaware of what another is doing, the 4 performance will be affected. So, we re focusing on how 5 they perform and how does it relate to the safe operation of 6 the plant. 7 CHAIRMAN APOSTOLAKIS: No , I understand that. I 8 guess the point I'm raising is that perhaps this is too 9 detailed. I mean you can define performance in many 10 different ways. 11 MR. MEYER: True. 12 CHAIRMAN APOSTOLAKIS: So, they don't communicate 13 very well. I mean, yes, that's an observation and so on, 14 but do you really want to be at that level, especially when () 15 the first -- the objective that's stated here is that -- one 16 of them is safety-related and risk-significant systems are 17 maintained in an operable condition. Maybe you can start 18 with that and see what do.you need'to inspect to make sure 19 that this is true without really going down to the detail of 20 cleanliness and so on.- 21- I mean maybe that's not a fair question to you, 22 but it doesn't seem to me that this is really 23 performance-based the way we perceive that these days. You
- 24. are looking for somebody who is not here.
25 MR. RUBIN: I would note that we do have a O' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
91 1 representative from the branch that is responsible for () 2 maintaining and improving these specific procedures, and I 3_ was wondering if,. Jerry, you had any comment on where we are 4- today in terms of performance-based, risk-based procedures 5 and perhaps where we might be going in the-future to make it 6 even stronger in that_ sense, , 7 MR. MEYER: While he's coming up to the 8 microphone, let me say that the inspection as it occurs is a 9 combination of the-training and the skills of the inspector 10 and the guidance that he has in the procedure. The 11 procedure guidance clearly can be delivered to you. It's 12 going to be hard for us to describe what are the skills, 13 what 10 the art of inspection. 14 CHAIRMAN APOSTOLAKIS: I understand that. () 15 MR. MEYER: But Stu is going to talk to training, 16 where there is a training course about inspecting for 4 17 performance =that talks _about the skills of -- if you're in , 18 the area that that procedure guidance addresses. , 19 CRAIRMAN APOSTOLAKIS: I guess, again, to 20 summarize my point, this is performance-based, in a sense, 21 there's no question'about it, but the question is really , 22 whether this is the right level -- l i 23 MR. MEYER: Right. 24 CHAIRMAN APOSTOLAKIS: -- to set the performance-25 . objectives, if you want to call them that. Perhaps we can ANN RILEY & ASSOCIATES, LTD. Court Reporters .. 1250 I Street, N.W., Suite 300 l Washington, D.C. 20005 (202) 842-0034
92 1 reduce this and maybe come up with some critical activities () 2 or parameters or something that we will inspect that will 4 3 be, you know, the result of some risk evaluation plus other 4 things, depending on whether the risk assessment is i
- 3 complete.
, 6 For example, the first objective is the licensee 4 7 operates the plant safely and reliable and operations, ' 4 8 surveillance, and other activities are performed in 9 conformance with license conditions, regulatory 10 requirements, and licensee commitments. 11 Basically, that tells you you have to inspect 12 - everything, right? 13 On the other hand, you might say, gee, in~a 14 risk-informed framework I wouldn't want any initiating 15
~
events-to occur. So, how can I make sure? What should I 16 inspect to make sure that that will not happen, that we will l' 17 not have, you know, the standard-list of initiators in a PWR 18 and a BWR. l 19 On the other hand, we may not want to go that high 20 level. So, that kind of argument, I'think, would be useful 1 21 to see how-do you decide where to set the performance 7 22 criteria so that, you know, something doesn't happen that is 23 - really an incident, not catastrophic but an incident. 24 I think that would be consistent with the current 25 thinking. ANN RILEY & ASSOCIATES, LTD. Court Reporters
- 1250 I Street, N.W., Suite 300
-Washington, D.C. 20005 (202) 842-0034
-. -. - - _ - - . - . ~ - - . .. - - - - _ - . - - . . - _
93 1 MR. MOSLEH: I was also looking for some links () 2 that would take these lower-level, more detailed ones to a 3 higher-level objective of performance, and normally, in 4 those kind of situations, yvu have a hierarchy or a model, 5 underlying model, that links these things together, and my 6 question was also the same. Is there such a model or 7 hierarchy relating these items together? 8 CHAIRMAN APOSTOLAKIS: Evidently not. 9 MR. MEYER: I would be hard pressed to present a 10 model. Why don't we look at that question again in a 11 moment? I'm going to talk about the PPR process, and that's 12 one where what you're talking about, integrating different 23 aspects of performance, is done. 14 CHAIRMAN APOSTOLAKIS: I understand what you're 15 saying. This has evolved over the years, you know, before 16 .the risk insights from PRAs were given the proper respect, I 17 guess. So, now maybe it's time to rethink the process and-18 say, gee, maybe some of the things we are doing are too 19 detailed, and it's not clear how they are connected to some . 20 higher goal that we really ought to be concerned with. 21 MR. MEYER: I will say that the inspector still 22 sets out to meet the objective, the higher-level objective. 23 The procedure gives him some guidance, not that he has to 24 follow each and every step to -- 25 CHAIRMAN APOSTOLAKIS: My comment really does not () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
94 1 address the inspector, because he has to do what you are () 2 telling him to do. 3 LMR. MEYER: Okay. 4 CHAIRMAN APOSTOLAKIS: But the people who design 5 -this-maybe-ought to think about what I just said - 6 MR. MEYER: Okay. 7 CHAIRMAN APOSTOLAKI3: And that may be part of the 8 6evelopment of the-plan, the human performance plan, to 9 revisit these things and make sure that they are truly. 10 risk-informed and performance-based the way we understand 11 the terms now, not'10 years ago. 12 MR. MEYER: Okay. 13 CHAIRMAN APOSTOLAKIS: I'm trying to be 14 constructive, in other words. () 15 MR. MEYER: Right. 16 CHAIRMAN APOSTOLAKIS: Marty, you wanted to say 17 something? 18- DR. STUTZKE: Yes, I was wondering, is this the 19- first revision of this document, or is this an original 20 document? I notice there's hash marks down every page. 21 MR. MEYER: .Which document? 22- DR. STUTZKE: The inspection procedure, 7177. 23 MR. MEYER: That's been revised many times.
'24 That's the basic procedure that the resident and senior 25 resident use day to day. So, that procedure existed over.10
=
/
. ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
95 1 years ago, when I was a senior resident inspector. The l 2 resident inspection program has been around for most of two 3 decades. I ! 4 DR. STUTZKE: I mean this reads like a very 5 typical Navy procedure to me, George. 6 CHAIRMAN APOSTOLAKIS: And you are surprised? 7- DR. STUTZKE: No, not at all. I mean it's a 8 guidance on how to conduct an inspection, not'what to 9 inspect. < 10 DR. MILLER: In that respect, it's not really 11- performance-based. ;
, 12 DR. STUTZKE: But I would suspect that inspectors-13 sometimes have a -- call it a hit list, but they go out and
! 14 look for specific things based on things they've learned, 15 heard, overheard,.and that, then, would be 16 performance-based. 17 CHAIRMAN APOSTOLAKIS: Not always, though. 18 DR. MILLER: It's not very well prioritized if ' 19 you're just using it on hearsay, i 20 CHAIRMAN APOSTOLAKIS: Well, again, all this is 21 performance-based. The question is whether this is the L 22 right level to inspect performance. That's really the l 23 question. l .. L 24 MR. MOSLEH: Actually, it's the right level and l 25 the right category of items, a 4
- O ANN RILEY & ASSOCIATES, LTD.
Court Reporters
-1250 I Street, N.W., Suite 300 l
Washington, D.C. 20005 (202) 842-0034
- . - - .~ - - . . . - - - . . . - - . - - - - -- -. - - - - - - - - -, --
. - . - . ~ - . - . - - . . _ _ _ . - - . . - - - . - _ - - . . . - . - . _ . _ - . . - . - - .
1 96 i i
- 1. CHAIRMAN APOSTOLAKIS: Things that are really I
() 2 relevant to safety. 3- MR.-RUBIN - Again, when we provided that 4 procedure,-the intent really was to provide'the committee
- S with an opportunity to'see how the human factors components 6- are built into and integrated into the procedure, and that l
7 one, which is a central procedure for inspectors, now has 1 8 significantly more human factors elements embedded within it i 9 than its predecessor, and if you would read through that t 10 with a highlighter and highlight what you might capture as 11 human factors, I think there would be a significant amount. 12 The issue of performance-based was not something that we 13 intended to pursue here. 14 CHAIRMAN APOSTOLAKIS: I wonder what the :
) 15- industry's view of this is. Do they think this is too 16 intrusive?
17 MR. CARROLL: They've accepted it. 18 CRAIRMAN APOSTOLAKIS: They have accepted. 19 MR.. CARROLL: It's been there for so long that I 20 think-they just sort of say -- 21 CHAIRMAN APOSTOLAKIS: It's like the fire -- f
-22 Appendix R, right? They've accepted it.
23 MR. MEYER: It can be intrusive if it's done ; 24 improperly. If the performance that's being focused 1on is 25- not directly tied to the safe operation of the plant, it can i t - ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I-Street, N.W., Suite 300 Washington, D.C. 20005 ; (202) 842-0034
97 1 be intrusive. If it deals with those factors that are () 2 important, then it's aLreality that they need to address. 3- MR. CARROLL: Well, you know, for example, 4 material condition, cleanliness and all that gc.' stuff, 5 that came about not because it has much to do with safety, I-6- don't think, ltt came about because we had-a-couple of 7 admirals in charge of the agency. That's what admirals look 8 for. 9 MR. RUBIN: In defense of that point, I think many 10 plant operators and regulatory staff would agree that, when 11 you do have an uncluttered, cleaner environment, it is 12 easier for the human to spot things that are abnormal. So, 13 it may not have a direct benefit on safety, but it does have 14 the indirect benefit of making a little clearer the things. 15 that are not correct. 16 MR.JCARROLL: I think utility management ought to 17 be concerned that thay have a ship-shape plant, but I'm not-18 sure that the NRC needs to -- 19 CHAIRMAN APOSTOLAKIC: In other words, INPO should
'20. have something like that ia their inspection manual, but not -21 us . - '22 MR. MEYER: Well, let me give you an example to 23' bear-up on your point, and that is I du operator licensing 24 ~ e v.a m s , my staff and I, and I went to one facility and 25- noticed that they had the safety parameters display system, i
O ANN RILEY & ASSOCIATES, LTD. Court Reporterc
, 1250-I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
98 1 -It's-a computer system to measure significant parameters in 2 the event of an event similar to what we would be doing 3 during the exam, and on the display system was a little red 4 placard that said, NRC commitment, unless being used, this 5 system should always be aligned to the top display, okay, 6- little red placard, and so,-I said to them, why is that 7 there? If we can't trust these operators to know what-8 display is the right display, then we've got bigger problems 9 than telling them what to do with their display system. 10 To bear out on that, I mentioned that to them, and 11 of course, in the scheme of things, nothing happened. So, I 12 had the opportunity to be there later and decided to take 13 things into my own hands and went into the control room -- 14 this was-the simulater which replicates the control room 15 --and said to the shift supervisor, I don't think that 16 should be there, I'd like to get rid of it, will you help ' 17 me, and he said, well, certainly. So, we went over and 1F ripped it off, and he was quite pleased to do that. 19 Unfortunately, the-licensing-department got 20 involved, and they-insisted it had to be put back on until 21 it was properly removed and, you know, remove this, that, 22 and~the other thing, but yes, the NRC can get involved in 23 things that don't relate to safety. It happens. It 24 shouldn't but it does. 25 CHAIRMAN APOSTOLAKIS: But we should try to ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W,, Suite 300 Washington, D.C. 20005 (202) 842-0034 1 . . . . . . . .
99 1 prevent'that -- ! () 2 MR. MBYER: Troe. 3 CHAIRMAN APOSTOLAKI3:. - as much as we can. So, ' ny first reaction to this --'and evidently, our invited 4' i 5 experts are having the same reaction -- is that it's too '
\
detailed,-but this is1not-your presentation, so why don't we ! 7 go ahead with it? 8 MR MEYER: So, if I may talk about the PPR 9' process, what the PPR process is is an iterative process 10 that's done on a six-month cycle to tcke a look at the range 11 of information that we can integrate performance assessments 12 from. So, it's an integrated assessment of performance. 13 It's also geared towards determining what should be done-in
.14 the next six months, what changes can we make to the I 15 inspection plan.
16 The process involves a review. Now, this is 17' where, as we had said, is there a model? .Not exactly. It's 18 the experience.of the people involved. You know,_I think 19- that, subjectively, they know what they're looking for, but , 20 can it be put on paper and explained to you? I'm.not 21 certain. -
'22 But the sort of things that they look at are l i 23 inspection reports, allegations that may have been received, .
I l 24 what the enforcement history is, event reports,- the 50.73 25 -LERs, also performance indicators that the AEOD people ANN RIiEY & ASSOCIATES, LTD. O- Court Reporters l 1250 I Street., N.W., Suite 300 l Washington, D.C. 20005 (202).842-0034
. - - . - , . - - _ _ . . _ - _ - ~ . . - . . _ . - - - - . . - , . - .,,
100 1 provide. There is aigo the human factors database () 2 3 information system, insights from different sources on human-performance. 4- MR. CARROLL: Who's involved in this process? 5 MR. MEYER: At the regional level, speaking for 6- Region I, it's the top regional management, the-division 7 management,_and usually four different functional areas. 8 There's a functional area coordinator, a person that's 9 responsible._ That would be a lead inspector or engineer, 10 and he would, you know, synthesize different facts, and 11 then, they go into a meeting. It takes four days long, 12 typically, in Region I, where each plant would be looked at 13 on the order of two hours, and they go through tunctional 14 area by functional area and talk about, well, in () 15 maintenance, what have we seen, you know, it seems as though 16- that there's quite a number of events-happening in 17 balance-of-plant systems, are they sensitive to that, should 18 we inspect in that area, that sort of thing. 19 MR. CARROLL: Are there residents involved in 20 this? 21 MR. MEYER: Definitely. The residents would 22 typically be the lead presenter in the operations area, and 23 they, being the most knowledgeable about that plant, are 24 _ depended on to provide perspective. 25 MR. CARROLL: When you say operational area, does O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
101 1 that include maintenance -- () 2 MR. MEYER: Yes. 3 MR. CARROLL: -- and rad protection -- 4 MR.-MEYER: Yes. 5 MR. CARROLL: -- and chemistry? 6 MR MEYER: What tends to happen 11s that the 7 residents or the project management side of the region-8 handles operations and maintenance.- The staff side, DRS, 9 tends to handle engineering and plant support.-
-10 MR. CARROLL: Okay.
11 MR. MEYER: And.one of the tools that's used in 12 this process is something that's called a plant issues 13 matrix. -We have some examples.on page 7 and 8, and what the 14 plant issues matrix does is tend to take -- it attempts to () 15' take the different things that have been in the inspection 16 report and provide the details.
- 17. Now, this is a somewhat-out-of-date example drawn
-18 from the procedure that describes how to do.the process.
19 Yhis'is a' process that's not yet two years:old, and what I'm 20 going to talk about that doesn't show here is the data, the 21 discrete findings, are also categorized as to functional 22 -area and also category, where it's broken out into five 23 different areas. 24 But I've gotten ahead of myself just a little. 25- Let's go.to slide 5. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street,-N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
J 102 1 So, the SALP functional areas of operations, () 2 maintenance, engineering,.and plant support -- that's the 3 basic system that's used to take this data in the plant 4 losaes matrix and organize it, but now what we're addressing 5 also, I would say concurrently, is to look at categories of 6 plant performance, and those are operational performance, 7 material-condition, maintenance-type _. area, human 8 performance, specifically, engineering and design, and also 9 problem identification and resolution. 10 What we have found is that a given facility's i 11 ability to identify problems, to analyze them for root 12 cause, and to actually effectively correct the problem is 13- crucial to how they perform over a period of time, so that's 14 called out specifically as a category that we attempt to 15 assess. , 16 MR. CARROLL: Where does ra'd protection, 17 chemistry, things like that, fit in, security? 18 MR.-MEYER: Okay. Right now, by1the categories, 19 that's considered operational performance. In other words, 20 it's those aspectsJof safely operating the plant. , l 21_ CHAIRMAN APOSTOLAKIS: -If I take a reasonably , 22' well-knowntcase like the Northeast Utilities, why wasn't it
-23 caught before it hit Time magazine? Which one of these 24 bullets was not implemented =right, or is there a sixth 1
25 bullet that's missing? l ANN RILEY & ASSOCIATES,-LTD. t L Court Reporters 1250 I Street, N.W., Suite 300-Washington, D.C. 20005 (202) 842-0034 _ . . _ _ . _ . _ _ , _ . _ _ _ . _ . _ _ _ ,__._._ _ _ _ .__. _ . _ _ . . ____._._a
103 1 MR. MEYER: That specifically relates to () 2 allegations and how allegers are treated and how their 3 concerns are addressed. That's the sum and substance of a 4- lot of Time magazine. 5 DR. POWERS: Well, let's be accurate here. It was s 6 caught before it hit Time magazine. It hit Time magazine 7 after it was caught. 8 MR. MEYER: Thank you. 9 CHAIRMAN APOSTOLAKIS: Well, yes, but it was 10 really out of hand, wasn't it? 11 MR. MEYER: Region I was aware that Millstone was-12 struggling with performance and had been for years. The ) 13 agency took -- now we're dealing with-the question, what-are 14 the tools that the agency has to address substandard 15- performance? 16 I know, in the Millstone case, that issues were 17- raised.. The agency's highest managers went and spoke to the 18 Millstone board to convey our. concern. I mean other than an 19- orderfto shut down --
'20 CHAIRMAN APOSTOLAKIS: So, is it the first bullet, 21 then, operational performance?
22 MR. MEYER: Yes, I would agree, but part of the 23 problem is problem identification and resolution. They 24 ~_ continually refused to identify the problems and address 25: them as such and correct them. It was always we have a plan ANN RILEY & ASSOCIATES,-LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
-l
104 1 that we're working on, two years from now things will be () 2 fine. Well, two years later, they weren't fine at all. 4- 3 So, I would argue that certainly you saw it in i 4 operational performance, but one of the key problems was 5 problem identification and resolution. i 6 DR. SEALE And there was a chronic mate.Aal ! 7 - condition problem. 1 j 8 MR. MEYER: True. And the way that they dealt f t 9 with engineering and design was also unacceptable, 10 CHAIRMAN APOSTOLAKIS: Yes, but we're saying this 11 after the fact. 1 12 MR. MEYER: We were saying it at the time. 13- DR. POWERS: I think what he's saying is that 14 inspection had noted these things. It's the subsequent 15 ' followup that has come_under question not only from Time 16- magazine but within the agency and some of its other t- 17 oversight _ bodies. l 18 A ' lot of td,is is what leads to revisions in the 19 - senior management process and. things-like that, because a i 20 lot of. things were going in a way the agency didn't want j 21 them to go for a long time before dramatic action was taken.
- - 22 I think there are other instances where you can 23 say, gee, why isn't the inspection program working as well 24 as we'd like it to work --
25~ MR. MEYER: Okay. 4 () LANN RILEY-& ASSOCIATES, LTD. Court Reporters 1250 I Street, .N.W., _ Suite 300 Washington, D.C. 20005-(202) 842-0034
--,,.,-,,-~,,,e -..,r--~<,,,rm v-+ 'w- -
'*w've'v~ -"~4' ' * ' ' ' " * " * ~ ' ~ ~ ' ' ' ~ ' ' * ' ' ~ ^ ~ ~ ~ " ' ' ^ ~ '
105 1 DR. POWERS: -- and some things that have come up () 2 recently include things like coatings, I believe, there were 3 instances, some others, I think, if I put my mind to it. 4 Not only have they been identified for us, they've been j 5 identified for us by the inspection program as weaknesses 6 that the,"'ve detected in the inspection program. 7 MR. MEYER: Part of the inspection program -- I've 8 talked to two of the prongs of the core program and the 9 initiative program, but there's a third prong that'c generic 10 issues or what we in the field would call temporary 11 instructions, where an issue gets identified, and sometimes 12 it's across the board, okay, everybody's going to be 13 inspected once in this area. 14 I handle the maintenance rule, and so, a baseline () 15 inspection is being done at each and every facility once, 16 and therr, are other examples where they may choose to only 17 inspect, say, 20 percent of the plants and then go from 18 there, but that's a third prong where a given issue would be 19- inspected with the objective of looking at it, assessing it, 20 dealine with it directly once, and then moving on. 21 What we're talking about is the continuing, year 22 after year, inspection program. 23 DR. POWERS: I think it's true that -- we've 24 cert:11nly heard about the maintenance rule inspection, and I 25 thir.k the response on this committee's part was that it was i ANN RILEY & ASSOCIATES, LTD. 3 Court Reporters l 1250 I Street, N.W., Suite 300 l Washington, D.C. 20005 (202) 842-0034 l
1 i , 106 1 1 fantastic, very good job being done there. 2 I think it is this ongoing -- which is always a 3- problem -- I mean it takes eternal vigilance, and anything _4 that becomes repeated-is -- suffers from repetition -- that ; 4 5 is the concern here. 4 6 MR. MEYER: Right. One of the things -- I don't ' 7 want to derail into the maintenance rule but material ! 8 condition -- the nature of the maintenance rule is that they 9 have to establish what are the important systems, what are 10 the functions of that system, and then, in analyzing events, I 11 if they have a functional failure, they have to assess, was 12 it a maintenance preventable functional failure? 13 Well, what is frequently involved _in that aspect. 14 is human performance, people that didn't identify the issue 15 or found the problem and didn't correct it right the first 16 time and that sort of thing, and if they have enough of ; 17 these maintenance preventable functional' failures, then they , 18 exceed their performance criteria and they have to focus on 19 that system. 20 So, although it's a' maintenance rule, 21 maintenance-dominated-type aspect, human performance is a 22 key factor in that. 23 CHAIRMAN APOSTOLAKIS: You only have 35 minutes 24 left. 25 MR. MEYER: What I've done is taken human i ANN RILEY & ASSOCIATES, LTD. Court Reporters i 1250 I Street, N.W. , - Suit e 300 Washington, D.C. 20C05 (202) 842-0034
;--,--. .-- - - -- ~ , - . - - - - - - - - - - - - - --- - - - - -
t 4
- 107 1 performance, that category, and split it out into some
() '2 specifics that we can look at briefly, and what I have here 3 is the guidance that the program has provided us of how to
-4 categorize -- in other words, what-events would -- I think 1
5 this would be HP and then it's A, B, and C. i 4 6 So, if-you're seeing -- it gives you the ability 4 7 -in the plant issues matrix, or PIM, if you're seeing 8 repetitive events that relate to human performance or,
; 9 specifically, the knowledge, skills, and abilities of the 10 mechanics, the operators, the HP technicians, if you're l 11 seeing that, then it proviaes a help to say, hey, we think i 12 that there is a basic training problem here, why aren't 13 people _given the skills and abilities they need to do the
- 14 job.
() 15 Then we can go off and inspect in that area, be
- 16. addressing that issue with the facility. So, human h
17 performance -- 18 CHAIRMAN.APOSTOLAKIA: That's an interesting
'19 . thing, though, because it-comes'back to this morning, when 20 Jay said that, you know, stuff happens. So, the criterion, .
12 1 .then, as to whether there is a problem with the training or 22L with the procedures or something is that something is 23 observed several times. i 24- MR. MEYER: Right. The objective is to catch it i 25 at a low level. There's small performance issues, events, > t () -ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N W., Suite 300 Washington,1D.C. 20005 (202) 842-0034
108 I 1 toe sorts of things that you can draw an insight from that 2 the problem gets addressed before it becomes a bigger 3 prvblem. 4 Can we go in and look at that area in and of 5 itself and make any conclusions? No , it's a very hard thing 6 to do. So, we deal with the small problems that are found 7 and integrate from that. 8- We have a couple of slides -- I'll start with the 9 first one. Again, this is a sample of what a plant issues 10 matrix would look like, where the details of events, l 11 inspection findings, allegations, the basic data that goes 12 into this evaluation process, is provided for all the people 13 involved, and they get to see, are there trends, are there 14 specific areas that are a problem? 15 The thing that we don't see of the major 16 categories that are being coded -- it helps the process. It 17 focuses it into one area or such. 18 So, what comes out of the PPR is, as the overview 19 showed,.the core inspection -- there's-certain inspections 20 that are going to be done month in and month out, year in 21 and year out, are basic inspections, but the areas that they 22 focus in can be adjusted, and if the PPR sees, well, 23- goodness, it looks like they're insensitive to 24 balance-of-plant equipment, then we could do something in 25 t. hat area. ANN RILEY & ASSOCIATES, LTD. Court Reporters. 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
I 109 1 Put up slide number 9. () 2 DR. SEALE: Just a second. 3- MR. MEYER: Okay. 4 DR. SEALE: You've listed on these two slides a 5 series of thi* , which have specific human performance i 6 content. 7 MR. MEYER: Right. 8 DR. SEALE: In reality, your plant issues matrix 9 has.a lot of other things in it, too. 10 MR. MEYER: Yes. 11 DR. SEALE: Hew many -- typically, how badly are 12 -- to what extent are these issues submerged in a larger sea 13 of issues in the matrix, things that don't have anything to 14 do with human performance?
) 15- MR. MEYER: I couldn't give you a. specific answer 16 on that. The plant issues matrix typically run for like i
17 eight to nine pages. 18 DR. SEALE' Well, that's what I'm thinking. 19 MR. MEYER: And they-would have six or seven, say,
-20 events or items on each page, and if you're interested in 12 1 that, we certainly could provide examples. I don't know if 22 there is evaluations of --
23 MR. CARROLL: Wouldn't it be to say, though, that 24 virtually everything that makes a list like this has some 25 human performance --
-ANN RILEY & ASSOCIATES, LTD.
Court Reporters l' 1250 I Street, N.W.,' Suite 300 Washington, D.C. 20005 (202) 842-0034
110 1- MR. MEYER: Yes, I would agree I'd be hard
~() _2 pressed to come up with something that would make the list
- 3. that hs Oened absent human performance.
14 DR.-SEALE: Okay. 5- MR. MEYER: Slide number 9. 6 What we have here is,-in the operations area, 7 there could be an assessment of performance, and we've given 8 -a few examples of the kind of things that could show up 9 that, on the bottom half, recommended changes, where there-10 could be a specific initiative done or there could be a 11 focus within the existing core inspection to address the 12- specifics of_what we're concerned about. 13 rexP. slide. [ 14 MR. MOSLEH: Can I go back quickly to this issues 15 taatrix? 16 MR. MEYER: Certainly. 17 -MR. MOSLEH: Since this thing makes it a longer
=18 list and a-lot-of them, as you said, have some' influence 19 '
from human factors issues, some element of human factors, 20 what, then,-qualifies one-item as a human factors? Is it a 21 root cause-being recognized of being human factors? 22 MR. MEYER: In other words, we have the five 23 categories, and one is human performance. f 24 MR. MOSLEH: You have a list of, say, 50 items 25 here -- ANN RILEY & ASSOCIATES, LTD. N-- Court Reporters 1250 I Street, N.W., Suite 300 Washington,-D.C. 20005 (202) 842-0034 i
111
- 1- MR. MEYER: Right.
() 2 3
-MR, MOSLEH: -- and ther. you say, well, all of them, or most of them, have some human factors element.
4 MR. CARROLL: All of these do. 5 MR. MEYER: -Certainly. Well, they were picked out 6 for that reason, 7 MR. MOSLEH: Then they're picked, you pick them 8 from that list, longer list, and identify these or label 9 them as human factors issues. 10 MR. MEYER: Right. 11 MR. MOSLEH: _ But what kind of criteria do you use? 12 MR. MEYER: Not necessarily. You know, there's no 13- effort to say what percentage are human factors or human 14 performance. There's no output that assesses the data and 15' says, oh,,they're,'you know, .so high on the scale or 16 anything like that. The output is what are we going to do,.what are we 18- going to inspect to further address this area? 19 MR. MOSLEH: So, if something is identified as 20 being primarily a-human factors as a result of some root 21 cause analysis -- 22 MR. MEYER: .Right. 23 MR. MOSLEH: -- and that's based on a particular 24 classification scheme, a way of identifying what is a root 25 cause? [ N-ANN RILEY & ASSOCIATES, LTD. Court Reporters 1260 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
112
-1 MR MEYER: Again, I have to say there's no
() 2 specific model that.I can point to that would say, when-3 human factors reaches 30 percent of the plant issues matrix, 4- then we will do this. It 's aul assessment looking at the-5 data and trying to come up with a response-that fits that ) -6 -_ data._ 7 MR.-MOSLEH: But there you need to identify a 8 cause,_a description of what happened. 9 MR. CARROLL: You might help him oy explaining 10 where slide 9 comes into play, 11 MR. MEYER: Right. Let's flip to 9, and what 12 we're seeing-here is -- we have the plant' issues matrix. 13 There's also a write-up as part.of the PPR to say, hey, our
.14 ' assessment is that there have been a number of recent O
(m ,/ 15 operator errors and that represents a continuing trend, and 16 the-corrective actions they have taken are not fully
'17- successful.
18 So, this could be.one of the conclusions that the 19 PPR would draw.- It could have been drawn by the senior 20 _ resident in part of his write-up that goes in, or maybe the 21' regional administrator would say, gee, I really see a lot of 22 operator errors, are we doing something about that? 23- Well, the what are we doing then would be, well,
.24 let's do a one-week inspection, let's focus on operator 25- errors. It might involve bringing a resident inspector from
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 ' I
113 1 another site. Sometimes we would call up-NRR and-they would () 2 provide a human factors specialist. 3 We might choose to look at an extended length of 4 time in the control room to see, well, what's behind this? 5 Is it that they're not -- the errors are caused because
'6- they're not-working with the-maintenance and the HP and the 7 chemistry people, or don't they have the skills, or what's !
8 -there? 9 And so, what we're attempting to do is what i 10 concerns can we develop and what are we going to do about 11 it, and it might be to say, hey, the residents that are 12 there, when they do the core inspection, they should 13 continue to specifically look.at this area, what-are they 14- doing about operator errors?
) 15 -There's implications that they have a corrective 16 action program, they're aware of it also, but apparently 17 it's not working, so let's focus on that in the core, let's 18 help-the people there out with some regional people, some 19 headquarters people, you know, let's address that, let's see 20 if we can't draw a firmer conclusion.
21 MR. CARROLL: This example slide is a followon_to 22 the issues that you've documented. 23 MR. MEYER: Te.sht . 24 MR. CARROLL: I mean they're consistent. 25 MR. MEYER: Right, We're providing examples here, ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
'i
114 1 so it doesn't necessarily follow on one plant, but yes. () 2 -MR. CARROLL: If you go through the events that 3 _ occurred, you can see how you came to the conclusion that 4 operator error is a continuing trend. 5 DR. SEALE: When you say " operator," do.you mean , 6 licensed operator, or do you mean plant equipment operators, 7 as well? 8 MR. MEYER: It could be both. They would have to 9 be specific. Typically, it's the licensed operator. 10 DR. SEALE: See, that's part of the problem, I 11 think. 12 CHAIRMAN APOSTOLAKIS: Again, I'm going back to 13- what has happened, like Wolf Creek. Why did that happen? l 14 Why didn't this inspection process prevent.that from "
) 15 -happening? Is that something we have to live with, that 16 things will happen, or was there a structural problem in the 17 inspection process that. allowed such a thing to happen?
18 MR. MOSLEH: Key to that is some way of linking 19 the-observable to the root cause, and I was wondering if 20 there is such a link. 21 MR. MEYER: It's a subjective process based on the 22- skills and experience of the people performing it. 23 CHAIRMAN APOSTOLAKIS: Okay. So, now this 24 happened. Now, you said earlier that, for something to 25- really become epistemic, which is, you know, part of the () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
115 1 knowledge base, it has to happen several times. On the n ( C/ ) 2 other hand, Wolf Creek, you don't expect it will happen 3 several timer. 4 MR. MEYER: No. 5 CHAIRMAN APOSTOLAKIS: Did anything change here as 5 a result of that to say, gee, you know, maybe we should look 7 at this kind of thing, so in the future such an alignment < 8 will not take place again? I mean was two processes taking 9 place at the same time and created the path and they lost 10 water, 11 What does one learn from that? I mean isn't the 12 first question why did it happen even though we have such a 13 detailed inspection procedure in place? Is that the wrong 14 question? (3 (_) 15 MR. MEYER: Well, flip to the oversimplified 16' diagram. 17 When I said we look at small events to try and 18 draw conclusions, again, given the risk importance, one 19 risk-significant event could be serious enough to say, wow, 20 how could this happen? 21 I'm not familiar with the Wolf Creek example, but 22 my guess is that the agency sent a team of people, probably 23 using outside help, to fully understand how could this have 24 happened. 25 One of the things that is done is to look at, hey, [\ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
116 1 the core program of what we do month in and month out -- () ~;f should that be improved to address this, and frankly, that's 3 how it got so detailed, is that attempts to address previous 4 -problems, they wrote specifics into the procedure, and now 5- we have a fairly long procedure. 6 But I think, somewhere, there is, you know, 7 looking at the events and then feeding it back into the 8 program. 9 DR. POWERS: George, just to remind you that it is 10 the lack of handle that the NRR feels they have on shut-down 11 events for monitoring and inspecting licensees that they_ 12 were advocating their shut-down-rule, They feel like they 13 don't have an adequate handle to get after_ things during 14 shutdown for enforcement.-
) 15 CHAIRMAN APOSTOLAKIS: But the question that 16- really is raised --
17 DR. POWERS. Now, we rejected that1 argument. 18- CHAIRMAN APOSTOLAKIS: But the question that is
+
19 ~ raised in my: mind is whether performance-based inspection is 20 capable of preventing something like that from happening. I 21 don't know what the performance measure would-be in that 22 case. Can we rely on performance alone, or is there 23 something else we should be doing to complement that? 24- MR. CARROLL: Well, one good example of why I 25 don't think you can rely on inspections entirely -- they're \/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
117 1 certainly going-to do some good -- is Chernobyl. That was () 2 one of the best operating plants in the FSU, and something 3 happened. 4 -CHAIRMAN APOSTOLAKIS: Performance is good, but I' : ' 5 don't know that -- 6 DR.- POWERS: They did a really lousy 50.59. 7 MR. MEYER: In a regional example,.that's what you 8 would tend to see, is different examples where they haven't-9' fully evaluated the consequences of something.that was being 10 done at the plant that you would raise and say, hey, where's 11 the review of what you're doing? 12 MR. MOSLEH: But also a way of relating the 13 . leading. indicators,-if you want to--use that terminology, to 1 14 .more serious events, and at the detail level, you're looking
) 15 for.those, and that's the idea behind_ going-to lower level ,
16 of detail, right?- 17 RMR. MEYER: True. 18 MR. MOSLEH: And if you' don't have a mechanism of 119 collecting that. evidence, that:information, and relating it, 20f then the predictive power of this-thing disappears. 21 CHAIRMAN APOSTOLAKIS: It doesn't cover 22 everything. 23 MR. MEYER: Right. I don't know that -- 24 CHAIRMAN APOSTOLAKIS: It doesn't -- 25 MR. MEYER: It doesn't disappear, but I can't [ ANN RILEY & ASSOCIATES, LTD. Court Reporters
-1250 I Street, N.W., Suite 300 Washington, D.C. 20005 j (202) 842-0034
118 1 assure you that it's going to be 100-percent effective. O) ( 2 It's the best that we have. We'll do it to the best of our 3 ability. 4 MR. DUDLEY: This is Noel Dudley. From my 5 experience, major events, when the agency has gone back to 6 look at whether there were precursors to those events, they 7 have found the precursors in inspection reports back a year 8 to 18 months previous to the event, 9 What happens is that there's so many -- so large 10 amount of data that, when you get into the PPR process, 11 determining what areas should be inspected, it's expert 12 judgement being used at that point. 13 CHAIRMAN APOSTOLAKIS: That, it seems to me, 14 argues for the prioritization of things. A ( _) 15 In other words, as we said earlier, if you look at 16 these -- you know, it says, you know, look at operator 17 knowledge and performance, operator training, cleanliness of 18 the facility, without any attempt to prioritize, and perhaps 19 as a result of that, you don't have this appreciation of, 20 you know, I have seen something that may be important, 21 because it disappears in an ocean of things that you have 22 seen, 23 But also, the question of whether performance can 24 really capture everything is really a significant one, and I 25 don't know how else you supplement it. I mean it's not an I \ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
119
.1 obvious thing, andLmaybe it's not the job of the inspectors.
() 2 Maybe it's the job-of'somebody else'to see that, especially 3 since most of this really refers to pre-initiating event 4 conditions, right? Because you don't have very many=of 5 those. 6 MR. MEYER: True, 7 CHAIRMAN APOSTOLAKIS: So, you have to extrapolate 8 .from this performance and say, gee, you know, they are doing 9 such a great job now that if there is, indeed, a medium-10 LOCA, they will also do the right thing. 11 MR. MEYER: But your point about is it beyond
-- 12 inspection -- it would have to be part of inspection, 13 because the events that we see -- those same -- Millstone --
14 I inspected Millstone in the early '80s, and they were a 15 very well-run place. 16 So, if it's look at the licensing and.the 17 structure of what they're doing-over a period of time, as it 18 degrades, if you're'not going there periodically to look at 19- what's happening, I don't know that you're going to find it. 20 CHAIRMAN APOSTOLAKIS: Somebody told me, actually, 21 that Chernobyl is called Prometheus.
~
The piant has been
.22 named after the Greek, you know, demi-god. Prometheus was 23 supposed to think about -- that's a very interesting name 24 for Chernobyl, isn't it?
25 I think we're asking a lot of questions that ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
120 1 really are not part of what you were intended to present, fi G 2 but you know, stuff happens. 3 MR. MEYER: I don't feel slighted. 4 CHAIRMAN APOSTOLAKIS: But I'm concerned about the 5 time, though. So, if you can skip some of your view-graphs 6 that you feel -- 7 MR. MEYER: How about if I skip the rest of my 8 presentation? 9 CHAIRMAN APOSTOLAKIS: No, no, no. I mean we'll 10 leave it up to you to decide. 11 MR. MEYER: Okay. Well, why don't just skip to 12 slide number 10, and just to finish up by saying these are 13 the procedures the inspection program provides where the top 14 three are ones that the resident, the people at the facility () 15 would be performing, and they're directed into operations, 16 maintenance, and engineering, and they're also people that 17 are regionally based that come out to look at licensed 18 operator re-qual, where we're looking at how they perform in 19 the simulator, we're looking at engineering to see how they 20 structured, what's the performance, the modifications, the 21 engineering reviews. 21 There's also an attempt -- this procedure is 23 probably one of the most recent -- of looking at the problem 24 identification, analysis, and corrective action program to 25 get a sense of are they identifying the problems and dealing ()
\~
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
121 1 with them? ( ) 2 And lastly, I've listed the -- this is the 3 standard for initial operator exams. It's a very good 4 performance measure. 5 A year ago, we went to Millstone and found that 6 six out of the seven peope: did not pass, and so, the 7 response was to basically say to the facility that this was 8 unacceptable and that they should analyze what the problems 9 were, and they did a reasonably credible job of doing that, 10 but it gives us a good performance result to use to evaluate 11 how the facility is performing. 12 Next slide. 13 And when problems are seen in events or in the 14 core procedures that we just talked about, here are a small A) (m 15 sample of the ones that we can use to address specific 16 issues. 17 One is a sustained control room observation, where 18 you go in to look at what's going on over a focused period 19 of time in the control room, or if training is the issue, 20 there's a procedure where you can-go and inspect and 21 evaluate the training program and how it's being performed, 22 the emergency procedures, plant procedures, and also, if 23 there's any sort of an event, there's the opportunity to 24 provide an augmented inspection team, give the team the 25 expertise they need to fully understand what happened during {G*-} ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
122 1 the event and what does it mean to the performance at that f( ) 2 plant? 3 So, with that, I think we have pretty much 4 addressed what I intended to cover, and I will turn the 5- presentation back to Stu. 6 DR. POWERS: Before you abandon the podium here, , 7 let me ask you this question. You have a wealth of 8 experience in this area of inspections and looking at human 9 performance in the --
,1 10 MR. MEYER: Am I being set up here?
11 DR. POWERS: -- operation -- this is an easy 12 question, and you may not have an answer right now, but I'd 13 appreciate it if you have an answer later. 14 MR. MEYER: All right. (_) 15 DR. POWERS: The question is, out of that wealth 16 of experience, do you see deficiencies in the tools made 17 available to you and other people involved in inspection 18 with respect to human performance, that are correctable 19 deficiencies in the tools made available? 20 MR. MEYER: I think there's a lot of progress 21 occurring. I don't think we're there. I think there is a 22 sensitivity to human performance. 23 What we've been limited by was how do you address 24 that? I mean so much in the past was licensing and 25 structure, and I spent the first two years with the agency O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
123
-1 looking at quality assurance programs, and my conclusion was -2'- -Ilwas wasting my time.
3 MR; CARROLL: And the licensees'. 4' MR. MEYER: Right. -And half the time, what I was 5 trying to get them to fix was what they had done in response 6 to the_ previous inspector,_who said, if only you would write - 7 'your: audit reports this way, it would be so much better,'and 8- it was simpler to do_that than to try and tell him,_-no, no, 9; no, you: don't understand. So, we've gotten-away from that. 10 - We talked in the previous session about the root 11 cause training, where initially we had a tool _that was one 12 day, and it was-limited, and now it's been expanded to four 13 days,-and now -- I checked in between -- it's a required 14 . course.for new-inspectors.
- 15. _Does that mean that.everybody's had it? No. -The 16 new people are getting it. It gives them an additional
_17 capability. 18 We're_trying to make this PPR process better, 119- _Like I said,'the PIM, the idea of providing the data on 12 0 ~ -which these assessments are based, is only two years old. _I
=21 mean we've been doing these kinds of assessments for-years.
22 So, I think the tools are being provided. We're 23 still not-there. A lot.of'it.is the gut subjectivity and 24- (experience of the people involved. 25 MR. CARROLL: You would advocate continuous ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300-Washington, D.C. 20005 r (202) 842-0034
124 1 support for the various activities that are going on in the () 2 human factors arena in the agency. - 3 MR.-MEYER: I would say that human-performance is 4 4 a crucial aspect of what's happening at the facilities, yes. 5 DR. SEALE: Dr. Powers earlier reminded us of the 6 agency's concern for enforceability of shut-down rule 7- concerns, and we've talked about that in another context, 8 but I have a similar kind of question. 9 Are you comfortable with the extent to which the 10 inspection process looks at the activities of non-licensed 11 operations personnel? i 12 MR. MEYER: I would say yes. Primarily, the 13 resident inspectors have the opportunity to observe the 14 different people. I think, in the procedure, there's even a
) 15 suggestion that, periodically, they should go with 16 non-licensed people it. the field and see what they're doing 17 and watch the rounds and do'they understand what's involved 18 and, you know, are they picking up -- it's part of -- '
19- DR. SEALE: So, the process is there. 20 MR. MEYER: I.think. 21 DR. SEALE: Perhaps we need to better identify - 221 some of the pitfalls out there to help the inspectors in - 23- focusing on certain areas. 24 MR. MEYER: And I have to admit the inspectors are 25 also real people -- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (2L2) 842-0034
125 1 DR. SEALE: Oh , sure, j ) 2 MR. MEYER: -- and_they respond to the forces 3 'outside, you know, hey, do more=of this, hey, write this up, 4 hey, how's that'PPR write-up going, and do they have the 5 time to do what they would like to do? Not always.. 6- DR. SEALE: Yes. , 7 CHAIRMAN APOSTOLAKIS: Thank you very much. 8 So, now, in a dazzling display of his oratorical 9 abilities, Mr. Rubin will condense 17 view-graphs into nine
.10 minutes. Let's go to the heart of the matter, Stu. What is 11 the message you want to send?
12 MR. RUBIN: Are there any questions? 13 CHAIRMAN APOSTOLAKIS: That's really the 14 objective.- ! 15 DR. SEALE: Wow. 4 16 CHAIRMAN APOSTOLAKIS: I told you he was going to 17 be dazzling. 4 18 MR. RUBIN: 'What I'd like to say on this slide is
'19 that the staff at headquarters conductsta daily morning-
- 20. review of events and reports from the inspection staff to 21 cull through that. We provide a human factors specialist
- 22. from the branch to watch over that, make sure that important 23- human factors issues are not kind of falling through the 24 cracks. The human factors specialist will help out, if 25 necessary, on the followup, including, perhaps, even ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
126 1 participating in any AIT that may be dispatched. () 2 The next two slides I really won't get into. 3 Suffice it to say that the staff, the branch in particular, 4 has developed some decision tree charts that allows it to 5 make some sort of a logical evaluation of when and where a 6 human factors-type inspection may be appropriate. 7 Those evolve with suggestions of management from 8 both the region and headquarters needs to buy in, but this 9 does provide a logic framework for deciding if it's a 10- training area, if it's a communications area, etcetera. 11 The second chart is actually part of our 12 inspection procedure for tra:c,ing, and it lays out the 13 evaluation in kind of a rigorous systematic formal manner in 14 terms of deciding if there's an immediate safety concern (, 15 related to training, if it's appropriate to send out a team 16 to look broadly at training, to focus in on specific areas 17 like maintenance or operations or what have you, or also, 18 for that matter, to conduct a for-cause operator licensing 4 19 re-qual examination that's prepared and issued -- 20 administered by the NRC staff. 21 Moving on to the training of inspectors, the 22 training really, if you look at it, is spelled out in the 23 manual chapter 1245 for inspector training and requirements. 24 The top half of this slide lists those trainings that are 25 required. The fundamentals of inspection contains rN ANN RILEY & ASSOCIATES, LTD. ('-} Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
127 1 information or learning's on how to evaluate effectiveness in 2 management, corrective action programs, root cause incident .- 3 investigation techniques, also has personnel reliability, 4 MORT, which, when you look at the details of-MORT, it really ~5 covers all of the factors that influence human performance, 6 -:PRA basics, includes HRA techniques. 7 Also, there's optional training that is available 8 within the TTD, some coursec better than others. You talked 9 about the human performance investigation process being one 10 of them, Not all inspectors take these courses. Those who 11 have, it would appear, are better positioned to look at 12- human performance on'AITs and so forth. 13- Turning to the next page, if you took a look at 14 the other courses-available-within the agency,- the PDC 15 offers courses that, although are not directly related to 16 understanding human factors themselves, do provide an 17 understanding of how to go' conduct an investigation of human 18 performance problems. 'Some, however, do contain a certain 19 context on it. 20 There are outside courses which the staff has 21 taken. The human factors course at-the University of 22 Michigan, for example, is two weeks, very intensive 23 introduction to human factors, and there are many others. 24 I would not say that inspectors typically take 25 these courses, however. I do believe that they're mostly ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
128 1 taken generally by the headquarters specialists, but they (~ ( 2 are available to the inspection core out in the regions and 3 the residents. 4 Turning to the next slide, what this does is it's 5 intended to show what kind of human factors expertise 6 resides within the human factors assessment branch. 7 The branch typically gets involved with very 8 specialized kinds of inspections that have a real human 9 factors content to it, things like human performance 10 inspections, training inspections, EOP procedure review 11 inspections, corrective actions, OSTIs, AITs, just to 12 mention a few. 13 As far as the specialists are concerned, 14 collectively they provide a very extensive formal university ( 15 training and work experience in evaluating human 16 performance, things like expertise in human factors 17 engineering, psychology, engineering psychology, human 18 system interface, staffing and qualifications, fatigue, and 19 so forth, just to mention a-few of the expertise that 20 resides within the branch, 21 MR. CARROLL: As a matter of curiosity, what does 22 an engineering psychologist do? 23 MR. RUBIN: What does an engineering psychologist 24 do? 25 CHAIRMAN APOSTOLAKIS: I think he said ANN RILEY & ASSOCIATRS, LTD. Court Report 6r' 1250 I Screet, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
129 1 psychologist engineer. (' 3) 2 MR. ECKENRODE: We have in the audience back here, 3 Would you like to have him describe it? 4 MR.-CARROLL: I'm just curious. 5 DR, MILLER: What's his educational preparation 6 for that position? 7 MR. BONGARRA: I'm Jim Bongarra, and I'm an 8 engineering psychologist. An engineering psychologist, in 9 effect, is sort of synonym, I guess, in a way, for a human 10 factors psychologist. I think the terminology is similar, 11 if not, indeed, the same. 12 What do we do? Wc basically, in effect, conduct 13 inspections that are human performance-based. We do reviews 14 of, for example, control room designs. We're involved in, O (_,/ 15 in effect, activities that are focused more on-16 design-related issues as opposed to training issues, for 17 example, or organizational development-type issues. 18 CHAIRMAN APOSTOLAKIS: But your degree is in
.19 engineering psychology.
20 MR. BONGARRA: Well, in actuality, my formal 21 training is not in engineering psychology, per se. I have 22 an experimental -- 23 CHAIRMAN APOSTOLAKIS: But there is such a
~24 discipline in colleges, engineering psychology?
25 MR. BONGARRA: There is, indeed. () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
130 1 . CHAIRMAN APOSTOLAKIS: And that's distinct from () 2 3 human factors. MR. BONGARRA: I don't think that the distinction i
- 4. is that clear. I think it depends upon where you -- you
- 5. know, where you obtain your degree from, whether it's -- you 6 get a degree-in engineering psychology per se or whether you 7 get a degree in human factors.
-8 I think the curricula.that support both of those 9- degrees are very similar.
10 CHAIRMAN APOSTOLAKIS: I think' Allen Swain was an
-11 engineering psychologist, that guy who wrote that handbook.
12 MR. CARROLL: Okay. 13 CHAIRMAN APOSTOLAKIS: Okay. Thank you. 14 MR. RUBIN: Thank you, Jim.
) 15: Of the non-professional or non-FTE within the 16 branch, fully a third are budgeted to_ support inspection 17 activities in the areas I talked about, and adding up the 18 numbers, we-came up with 17 inspections that we supported 19 last year..-
20 The next chart is-simply intended to indicate that-21 contractors are part of the cadre of expertise that we can- . 22 -draw upon for looking at human performance issues. To be 23 truthful, today, that support is generally limited to what I 24 would call M&O or management and organizational contractors, 25 and they typically are brought in to support diagnostic ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 1 Street, N.W., Suite 300 Washington, D.C. 20005 (202) 942-0034
131 1- evaluation team inspections and special evaluation teams. () 2 CHAIRMAN APOSTOLAKIS: Can you give us a natae or 3 two? I-mean who are these contractors? 4 MR RUBIN: The names of people? 5 CHAIRMAN APOSTOLAKIS: Yes, contractors. 6 MR. RUBIN: We have someone who has led those, Ron 7 Lloyd. 8 Do you have a few names you can give us? 9 MR. LLOYD: My name is Ron Lloyd, AEOD. 10 There are several different contractors that we 11 used on diagnostic types. Jonathan Wert has been on several 12 of these. In fact, he was on the Clinton effort that just 4
~13 happened and did the M&O effort. He'd also been on other 14 ones.
) 15- There's a couple of different ones we've used over 16 time, some within the agency that were NRC people, but-then 17 we also used some contractors on the outside. There was a 18 ' guy named Brian-Hoagenson. We used him two or three 19 -different times. He has an-operations and.a management-20- background.
21 Going farther back in time, there were several 22 other contractors. 23 CHAIRMAN APOSTOLAKIS: So, they are not' engineers. 24 MR. LLOYD: They're people that have -- some of 25 these are engineers, some of them are ex-operators, some of O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
132 1 them have been utility managers in operations departments () 2 and things like that. 3 Jonathan Wert, who was on the Clinton effort, was 4 a Ph.D. in some sort.of psychology sort of a background, and 5 that wan his expertise. 6 CHAIRMAN APOSTOLAKIS: Okay. Thank you. 7 MR. RUBIN: Okay. 8 Turning to the next slide, I would like to quickly > 9 get into how we process the findings that are documented in 10 inspection reports. Basically, we cull through every 11 inspection report and identify the human factors findings 12 and code them by these categories. We have six or seven, 13 I'm not sure. 14 MR. RUBIN: These in turn are broken out into a O \;,/ 15 total of 60 subcategories which appear on the following 16 page. They're also categorized by the plant department 17 where the human performance issue is associated and the kind 18 of work that was involved, such as things like 19 troubleshooting, routine maintenance, calibrations, et 20 cetera. 21 So just turning quickly over the next page-is the 22 fuller breakout of the cause categorization, moving on. 23 This page is a revisiting of the last page that we had for 24 Glenn with the cause categories assigned in the parentheses. i 25 So for that same item No. 1 for example would be categorized ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
133 1 as.11 no procedures. ,The last one would be -- well the () 2 next-to-last one would be labeling less than adequate, No. 3 61. 4 Moving on to the next page, what this represents 51 now is you'look at -- we've-done this now for all our inspection reports and compiled a sort, an analysis for a 7- full.12-month period. And for the 12-month period we just 8 completed there were 3,438 human factors hits that were
-9 found in inspection reports, and this particular sort sorts 10- out those hits by the basic cause categories. And so you 11 can see, for example, procedures accounted for about a third 12 of the findings. The second most prominent one was 13 . deficiencies in work factors. And n,anagement supervision 14 deficiencies were the third most common we saw.
15 Turning to the next -- 16- DR. MILLER: Let me stop here a minute. How many 17 of those were related to operations or the operators 18 directly? I know: procedures probably -- procedures are-not 7 19 .necessarily written by operators. 20 MR. RUEIN: Okay. Well, let me go to the next 21 slide. You might see a little breakout there. What we did 22 -is, we then took the most common category of procedures and 23 we further sorted that group by the department in which the 24 procedural deficiency existed. And then you can do it in 25 virtually any sort you want. But this is insightful and 4 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
134 1 perhaps not surprising that the largest contributor was () 2 operations. Some areas had virtually no contributions to 3 the procedural deficiency area. 4 DR. MILLER: Let's go to the operation now. What 5 fraction of those were errors or lack of human performance 6 by operators, not just operations? 7 MR. RUBIN: Don't have that information. We could 8 provide that through a sort. The intent here was just to 9 indicate what kind of sorting capabilities 'e available in 10 the code. That can be provided if you need it. 11 DR. MILLER: So let's go to one other category. 12 When you say mechanical, is that -- what kind of performance 13 errors are we looking at? 14 MR. RUBIN: This is basically -- after operations (O (_) 15 you have the three principal maintenance areas, instrument 16 controls, electrical maintenance, and mechanical maintenance 17 groups, chemistry. These -- 18 DR. MILLER: Engineering pretty high? 19 MR. RUBIN: Engineering is also included in there. , 20 This is a sort by department. So the procedures -- 21 DR. MILLER: So procedures of course could fall 22 under -- like before you had procedures was a fairly high 23 area. So procedures could fall into many of these areas. 24 MR. RUBIN: Yes. Yes. Right. So looking at all 25 the procedural deficiencies that ap; eared in inspection (#
\-
i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
135
-1 reports, the dominant area was operating procedures is what
( ) 2 this chart is trying to say. The next most were 3 deficiencies in I&C procedures, mechanical maintenance 4 prccedures. 5 MR. MOSLEH: The percentages. Could that be kind 6 of influenced at least by opportunities that such procedures 7 have to reveal potential errors in them like operations that 8 are used more often? 9 MR. RUBIN: That's true. There are more of them. 10 They're revised frequcntly. The staff looks at that area as 11 part of its course. So all these things together will 12 create reasons why that may be the most common area, 13 MR. CARROLL: But do you get a hit if you should 14 have had a procedure and didn't? Is that a procedure? () 15 MR. ECKENRODE: Yes, you do. 16 MR. RUBIN: Yes, it's a procedure. 17 MR. ECKENRODE: You can break procedures down into 18 four or five areas also, No procedure available, procedure 19 inadequate -- 20 DR. MILLER: Or procedure used incorrectly. 21 MR. ECKENRODE: That's right. And we have -- that 22 breakdown is within this. You'll see them in those 60 items 23 that you saw. If you look at procedures it tells you what 24 they are. 25 MR. RUBIN: Yes, if we can go back to that slide, ANN RILEY & ASSOCIATES, LTD. [\ - _ Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
136 1 i did quickly skip over it, it was the full breakdown of the h 2 major cause categories, and within the procedural breakout 3 we have four different subcategories, no procedures, 4 procedure technical content less than adequate, procedure 5 ,, contains human factors deficiencies, procedure reference 6 1 document development, and maintenance program less than 7 adequate. All those taken together is built into that one 8 spike at almost 50 percent of the hits. 9 DR. MILLER: So a poorly written procedure, you 10 would categorize that human factors? 11 MR. RUBIN: If it was from a human-factors 12 standpoint, that's true. 13 DR. MILLER: A poorly designed engineering upgrade 14 would be human factors? 15 MR. RUBIN: Um-hum. Um-hum. As opposed to the 16 technical content, you know, the wrong number in the 17 procedure, thing of that sort, the wrong valve being 18 identified. 19 DR. MILLER: So a poorly organized management 20 structure, is that a human factor? 21 MR. RUBIN: Um-hum. 22 DR. MILLER: So everything is human factors? 23 MR. RUBIN: Yes. 24 DR. MILLER: Okay. 25 CHAINMAN APOSTOLAKIS: Although human factors -- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
-137 1- DR. MILLER: It's like QA used to be'20 years ago.
2- Everything needs to be--- 3: CHAIRMAN APOSTOLAKIS: -How about if you call-it 4 human performance really, because factors means something 5 specific. 6 DR. MILLER: I think that's a good point. 7 MR. RUBIN: Okay. The final sort is.the sort that 8 we produco semiannually fer both the PPR. process input as well as for the senior manager screening meetings. What 10 this bar chart represents.ie the total number of human 11 factors hits-for the last 12 months by plant in a given 12 rehion. So I don't know what regions this is, but that will 13 be the sort. 14 We also provide a twice the national average, () 15f which is indicated by that starred line, and the purpose.of 1 16 that is to be abic to have some discriminator of outliers, L17 you might say. So there are two-plants in this case, Plant 18 P and Q, which.would be flagged as being twice the national 19 -average and perhaps should get a special look at the PPR J20 stage and at the prescreening stage. 21 DR. SEALE: Are they both in the same-region? 22 MR. RUBIN: This is all from one region. i 23 MR. MOSLEH: Are they normalized or can they be 24- normalized with the number.of inspections? 25 MR. RUBIN: Not at the present time, that I'm [ ANN RILEY & ASSOCIATES, LT D . Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
l 138 1 aware. () 2 DR. STUTZKE: Why is the national-average going 3 up?
-4 MR. RUBIN: This is-tha national -- in other 5- wcrds,_we took the total number of hi:s --
6- DR. STUTZKE: Oh, I understand that, but -- 7 MR. RUBIN: Divided by.the number of plants. 8 DR. STUTZKE: Oh, I see in the year 1994 the 9 national average was 70 hits per plant and now it's 98. 10- MR. ECKENRODE: One of the reasons is we're 11- becoming more attuned to human performance problens, and sie 12' are now reporting more human performance problems. 13 DR. MILLER: Does that mean the total number of 14 inspection hits? This is only human performance hito, s () 15 right? Io the total number of inspection hits going up, or 16 is the total number which are attributed to human-17- . performance going up? Or both? 18 MR. ECKENRODE: Primarily the human performance. 19 I don't think the total number of inspection hits are going 20 up '. i 21 DR. M~LLER: So they may be in fact going down 22 while the total number of human -- 23 MR. ECKENRODE: No , it's about --
-24 DR. MILLER: Or maybe they're staying even?
25 MR. ECKENRODE: I think that -- well, inspection
; ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
, . . . . . _ _ _ _ . _ . _ . _ - - - + - -
139 1 hits is -- I was thinking of LERs. LERs are going down a () 2 -bit, but inspection hits I'm not sure. 3 MR.-RUBIN: Okay. 4 DR.-MILLER: It seems like-that's A fairly 5 important number, at least to tao. Aru we getting better at 6 just identifying hits as being human performance, or are 7 we -- or is human performance degrading? 8 MR. ECKENRODE: No, we're getting better at 9 identifying human performance. That's definite. 10 DR. MILLER: So a hit 10 years ago might have been 11 attributed to something else. Today it may -- 12 MR. ECKhWRODE: It might not even have been a hit. 13 DR. MILLER:- It might not have been a hit, but 14 let's say 10 years ago you had a hit, but today it might be () 15 human performance.- Ten years ago it might have just been 16 engineering design. 17 MR. ECKENRODE: Carrect. 18 MR. RUBIN: There are confounding influences in i 19 all this, so one being the changes in procedures as was 20 mentioned, the plant operations procedure, and the most 21 recent revision had built in many, many more row human 22 factors contents. So that fact alone may cause inspectors 23 to be more sensitive to those which may increase the 24 numbers, things of that sort, as opposed to plant 25 performance actually having.a negative trend. But that ANN RILEY .1 ASSOCIATES, LTD. Court-Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0934 ,
140 1 procedural change, because it's across all plants it applied () 2-3 equally, that should normal out in the twice-the-national-average line. ___ 4 DR. MILLER: If you look at the highest-level 5 category, and that's plant performance, which has 6 dramatically improved over the last 10 years. 7 MR. RUBIN: Okay. Well,.let me just say the last 8 slide, the kind of cats and dogs really of other things we 9 do, we build into the operating plans of both headquarters 10 and the regions requirements that really bear on human 11 factors, such as the need for the regions to observe INPO 12 accreditation, board meetings and accreditation, team 13 visits. We are committed to that three FTE to support 14 inspections with our human factors staff. We have task ' () 15 interface agreements with inspectors who are not as 16 knowledgeable in a particular area. We'll ask che 17 specialists at headquarters -- 18 MR. CARROLL: What impact on nuclear plant' safety 19 would you have if you had six FTEs to support the regions? 20 MR. RUBIN: Well, clearly if we doubled the number 21 of specialists there would be additional insights I would 22 ' suspect that would be brought out by those inspections that 23 the additional participants were involved with. 24 MR. CARROLL: Would that be1 cost-effective? 25 MR RUBIN: Well, let me say that within NRR there
- ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
141 l 1 is a trend to have the staff of NRR be less responsible for 2 inspection activities and to focus more on the licensing 3- review area, and so I'm not sure that we're going to be able
-4 to --
5 MR. CARROLL: Okay. Let's transfer-these_ guys out 6 to the regions then. Would that be-cost-effective, to have 7 that many performance specialists to-support inspection 8 activities? 9' MR. RUBIN: Well, the reality is that they're not 10 all the same. There.is specialization across those nine FTEs, so in principle if you did scatter them to the 12 regions, you wouldn't be scattering a uniform expertise. We 13 have training specialists, et cetera. So -- go ahead.
-14 MR, ECKENRODE: One answer I think is that there 15 is a finite number of inspection hours per plant _per year,
-16 and you basically shift around the various things that are 17-. _important, and so it's not a matter of just putting people 18 onto_it now, it's a matter _of the number of hours you're 19- going to spend on human factors things versus the number of 20 hours on other items. So it's a tradeoff basically.
21- DR. MILLER: Has the number of hours on human 22 performance increased over the lasc ten years? 23 MR. ECKENRODE: Specifically on human performance? 24 DR. MILLER: Right, or the number of FTE devoted 25 to human performance. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
I 142 1 MR. ECKENRODE: Yes, I think you've seen that l 2 here. l 3 DR. MILLER: Right.
-4 MR. ECKENRODE: Just in the number of human l
l 5 performance hits that you're seeing. So it has increased, i t 6 but to do what Jay is saying I think would be -- wouldn't -- : 7 isn't a solution. l 8 DR. SEALE: In that regard, though, earlier you 1 9 made the comment that there's a tendency for the inspection l' 10 process or let's say for the NRR people to be less involved 11 in the plant kind of activities. I guess based on what 4 12 you've said about specialization and so forth that the human 13 performance specialista you have are a clear countercurrent l ; 14 to that trend, or ought to~be, anywny. ' ( 15 And I wonder, Mr. Chairman, if we want to 16 recognize the fact that the human performance specialists i 17 should not reduce their interest in plant activities, and 1 18 that indeed they need to continue to be involved in these 19 kinds of inspection activities. 20 MR. RUBIN: Well, I would say this, that in the 21 last six months,-which is the period that I've been involved . 22 with the branch, that there's been if anything an increase 23 in the frequency that our phone rings asking for support. 24 DR. SEALE: Okay. 25 MR.--RUBIN: At plants, whether they be for-cause ANN RILEY & ASSOCIATES, LTD. Court Reporters . 1250 I Street, N.W., Suite 300 ! Washington, D.C. 20005 (202) 842-0034 1 _ . _ - . . _ . _ . . , _ , . . . _ _ _ , . _ _ _ _ , _ _ . , . _ _ _ , _ _ _ _ . _ _ . _ _ . _ _ . _ _ _ - _, J
143 1- training inspections, inspections at Millstone, the () 2 Commonwealth plants where there are very significant human 3 performance-issues that we get asked a lot, and it's really 4 such that we can't really cover all the bases. But at this 5' point -- 6 DR. SEALE: But on the other hand the last thing 7 you need trJ dc ib to be trying to defend yourself when 8 someone says ney, these guys are running counter to the way 9 everybody else is. What's wrong with them? And the answer 10 is there's nothing wrong with you. 11 CRAIRMAN APOSTOLAKIS: Going back to the 12 inspection objective No. 3, safety-related and risk 13 significant systems are maintained in an operable condition. 14 Does that mean that you guys are using fossil vessel and () 15. risk-achievement worth to define the risk-significant 16 systems,_or is it a general kind of thing. 17 MR. MEYER: Maintenance rule wise, yes. The 18 maintenance rule, when they -- they risk-rank'-- well, they 19 start with what are the systems within the scope. And then 20 the next thing is to risk-rank it. And I know for a fact 21 that that's where they use that. So that when the inspector
-22 goes in you would say what are the risk-significant systems.
23 And that would have been a factor in deciding. 24 CHAIRMAN APOSTOLAKIS: Okay. Anything else? 25 -DR.-POWERS: I'd like to just close by posing Er - ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
144 1 similar question I posed earlier. You seem to have the () 2 capability to derive a tremendous-amount of information
-3 about how human factors-affect plant-performance and how it 4 shows-up in inspection reports. It would appear to me then -5 that from=all that you've presented that you have an 6 adequate complement of tools to address human factors 7 issues. Is that your impression?
8 MR. RUBIN: We feel that we have adequate 9 resources in terms of people and in terms of the tools we 10 have available to us at this time. 11 MR. CARROLL: So you have no pending user requests 12 with research? 13 MR. RUBIN: For what was that? 14 MR. CARROLL: No pending user requests with 15 research to develop more tools? 16 MR. RUBIN: Oh, in that sense we are seeing a 17 closure, if you will, to the last' user needs memo that we 18 sent to research, and we're working on one right now. That 19 needs to bubble up through NRR management to see if we can 20 receive support for those user needs. It's in kind of the 21 preliminary stage of pulling that together, but I anticipate 22- there will be a user needs memo for human ~ performance tool 23 development going to research. 24- MR ~. CARROLL: I would hope 1so. 25 CHAIRMAN APOSTOLAKIS: Thank you very much, () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
145 1 gentlemen. We will reconvene at 1:30. () 2. .(Whereupon, at 12:18 p.m., the meeting was 3 recessed, to reconvene at 1:30 p.m., this same day.) 4 5= 6 7 8 9 10 11
-12 13 14 15 16 17 18
-19 20 21 22 23 24 2 5 __. _
O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W,, Suite 300
-Washington, D.C. 20005 (202) 842-0034
146 1 AFTERNOON SESSION () 2 (1:31 p.m.) 3 CHAIRMAN APOSTOLAKIS: Okay. Let's start. The 4 next presentation from AEOD on the Results From Studies of 5 Human Performance Event Database by Mr. Eugene Trager. 6 MR. TRAGER: Good afternoon, my name is Gene 7 Trager. Actually, this presentation is an update of an 8 earlier presentation of the results that we have with the 9 Human Performance Event Database under development AEOD for 10 the NRC. 11 In the handouts that have been distributed, we 12 have included additional information that was given in an 13 earlier presentation, and this material is for the benefit 14 of people who weren't at earlier presentations and is a O x,,/ 15- refresher for people that have been. 16 In its most recent updated form, the database, as 17 you can see, contains 196 records on 199 events. It 18 consists of or includes AIT reports, IIT reports, Human 19 Performance studies, Special Team inspections, the LERs for 20 all of those events, and a new type that we have added, and 21 these are events for.which there was a high conditional core 22 damage probability in the ASP database, but for which there 23 was not an LER. In other words, events for which the NRC 24 did not inveutigars the event. 25 In recent years, there is some missing _ data, and () ANN RILEY & ASSOCIATES, LTD. Court Reporters
-1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
147 1 that is result of a certain amount of time lag for getting () 2 records into the database, and we are trying to improve 3 -that. For example, we do have some ASP events in -- to be 4 added in '95, '96 and '97, that will be added, hopefully, in o 5 the near future. 6- MR. CARROLL: Why the trend that we see here? 7 MR.-TRAGER: This trend is -- it is apparent 8 there's fewer investigations of events going on, and AITE, 9 which I guess is the major event studies, or most thorough 10 event studies, and, as you can see, the numbers of those 11 have decreased pretty significantly. 12 Theoretically, the number of Special Team 13 inspections should have increased to make up for those, but, _ 14 as I said, I am missing some data, too, in recent years, so 15 I can't say that those haven't occurred. 16 MR. ROSENTHAL: This is Jack Rosenthal, Gene, we 17 decided to mess up your presentation by chiming in freely. 18 MR. TRAGER: Sure. 19 MR. ROSENTRAL: We did, in a different context, go 20 back and look at events that had occurred to see if we were 21 missing events. And concluded with -- and the screening 22 criteria for IITs and AITs, there's written criteria, has 23 remained the same. The number of events -- of 24 investigations has increased -- has decreased, and part of 25 that -- or much of that is attributed to just plain better () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I' Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 , o
148 1 plan performance. () 2 MR. CARROLL: Now, the HPS, that's where -- 3 MR. TRAGER: AEOD did some studies in the early 4- '90s and -- 5 MR. CARROLL: And you have stopped doing those. 6 MR. TRAGER: We discontinued those because they 7 were going to be picked up by other groups in the agency. 8 MR. CARROLL: Okay. So that is the explanation of 9 that string of zeroes. 10 MR. TRAGER: Right. Right. 11 MR. CARROLL: And were they picked up? 12 MR. TRAGER: I think they have been. Hard to say. 13 These were -- 14 MR. CARROLL: If it is hard to say, why do you 15 think they have been? 16 MR. TRAGER: I would assume they have.been, let's 17 .put it that way. The -- 18 MR. CARROLL: -That's how one gets in big trouble, 19 is assuming things. i 20 MR. TRAGER: . Right. 21 DR. SEALE: I am intrigued by the inscrutability 22 of the NRC to itself. 23 (Laughter.] 24 MR. TRAGER: - Well, we know -- for example, we know 25- the number of SCRAMS have decreased. I mean there's lots () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250-I-Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
149 1 and lots of evidence that that is the case, and it happens () 2 across the industry. And when that happens, .there is just 3 fewer opportunities, I guess, for there to be problems. So, l 4 you know, that.might be one example where that would happen._ l 1 5 Also, it is----good plan performance is just good 6 business now. It's just tmo expensive to be off-line. 7 The -- we have a report that is being: prepared by 8 INEL but it is kind of an ongoing -- it is being updated as 9 we work and develop the database. But in the near future,-
- 10. we are going to be issuing a version ~which includes what we l
11 have learned so far, and things we are -- additional things 12 we are thinking of attempting. i i 13 CHAIRMAN APOSTOLAKIS: When do you think this will 14 probably come out? 15 MR. TRAGER: I could be within a couple of months. 16 CHAIRMAN APOSTOLAKIS: We would like to get a copy 17 lof thP". 18 MR.__ TRAGER: Sure. 19 CHAIRMAN APOSTOLAKIS: As soon as -- 20 LMR. TRAGER: Okay. 21 CHAIRMAN APOSTOLAKIS: -- you can send us one. 22 MR. TRAGER: The types of -- this is a breakdown i L 23 of.the factors contributing, or contributing factors as ( 24 determined by the different investigations or studies. And, 25 of= course, we have the total across the top for perspective \- - () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250'I Street, N.W., Suite 300 L Washington, D..C. 20005 (202) 842-0034
,m -. - . - . , - -.,_,.e- - - , , _ , , _ , . , . , , ,,---4., . , - . . . . . . . , , - , , , - . . , . _ . , . _ _ , , , , . _ , . , . , - . - ,
150 1 'with the numbers of a given report type, and then the () .2 breakdown and contributing factors found. 3 CHAIRMAN APOSTOLAKIS: Now,-again, management and 4 organization. Doesn't, for example, communications belong 5 there?-- What does management and organization include? 6 Because,-you know, some of us would think that
- *f communications, procedures, and so on, training are part of 8 managing the plant.
9 MR. TRAGER: One of the other things I hope to 10 have distributed was an Appendix B which contains the 11 definitions of the different data fields. 12 CHAIRMAN APOSTOLAKIS: Yeah. Okay. 13- MR. TRAGER: As you flip through the fields, there 14 will be -- I nced a copy myself, actually, to t, ell you. () 15- Field-No. 32, the question is whether con.munication was 16 involved, or communications were involved. Then if you look 17 at 33, it includes the kinds of things that would be -- 1
.8- would make that a contributing factor. Any communication 19 issues including any -- and so on and so-forth.
20 CHAIRMAN APOSTOLAKIS: So, basically, then what 21 .you have done is you have identified.four or five areas like 22' training, communications, so on, and everything else'from
-23 the nanagement and organization area is under the heading 24 Management and Organization. .Right? That seems to be the 25 case.
, ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
151 1 MR. TRAGER: The Management and Organization () 2 field, too, there is a certain definition here. 3 -CHAIRMAN APOSTOLAKIS: Yeah. 4 MR. TRAGER: Right. And it is -- it is a pretty 5 broad -- a pretty broad-definition, I think.- 6- What this is -- the basis in this database, these 7- different, I guess you could say bins or possible 8- contributors come from a -- the datt' se structure is based 9 on a Human Performance Investigation Process which was 10 developed for the NRC Office of Research. And that, in-11 turn, the HPIP, is based on the MORT, the MORT method for 12 analyzing events at DOE facilities. So that's where those 13 categories come from. And I -- 14 MR. CARROLL: The reason that I can't add up the
) 15 first column and get 73 is that there may have been more 16 than one factor?
17 MR. TRAGER: Yes. Yes. 18 MR. CARROLL: To those events. 19 1410. TRAGER: Right. And I have some more 20 statistics that we will get to in a minute, will: help 21 straighten this out a little bit. 22 MR. CARROLL: Okay.
- 2 3_ MR. TRAGER: Okay. This table presents the 24- management factors, or the various factors and then the 25 fraction out of the total. For example, in AITs, 51 out of
-=
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
152 1 66, or 77.3 percent, management organization was a factor, () 2 or found to be a factor in AIT reports. That's the same for 3 the rest -- the rest of the numbers. 4 By definition, the column of -- the fifth column 5 over, the column of LERs and percent LERs, those are reports 6 prepared for events that were investigated, and that in the 7 first four columns. The AITs, IITs, Human Performance 8 studies and STIs, and as I mentioned before, the ASPLERs are 9 events that had high, relatively high conditional core 10 damage probabilities but weren't investigated by the NRC. 11 CHAIRMAN APOSTOLAKIS: Now, of course, this 12 doesn't tell us anything about how important these things 13 were, right? 14 MR. TRAGER: It doesn't say much about importance.
) 15 The --
16 MR. CARROLL: Well, they ask if LERs -- 17 MR. TRAGER: The high -- 18 CRAIRMAN APOSTOLAKIS: Yeah, I mean that was the 19 only criteria. 20 MR. TRAGER: Yeah. 21 CRAIRMAN APOSTOLAKIS: Right? 22 MR. TRAGER: Well, in -- let me get that back on 23 there. 24 CHAIRMAN APOSTOLAKIS: Good. We -- there is also 25 information en just the ASPLERs, by definition, were LERs [\- \ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 ;
153 1 that had a conditional core damage probability greater than () 2 10 to the minus 5 and that was -- we just arbitrarily 3 selected that group. 4 So 54 percent can be attributed to system design. 5 MR. ROSENTHAL: Maybe -- why don't you just go one 6 more slide. 7 CHAIRMAN APOSTOLAKIS: Okay. 8 MR. TRAGER: Yeah. What I am going to do in the 9 next slide is cull out the percentages, because it is a 10 little easier to see it. 11 CHAIRMAN APOSTOLAKIS: Just before we go co that, 12 it is Human Factors Engineering. What does that mean, the 13 layout? 14 MR. TRAGER: Human Factors Engineering? () 15 CHAIRMAN APOSTOLAKIS: Welcome aboard and that 16 kind of thing? Or -- what is Human Factors Engineering? 17 MR. TRAGER: Human Factors Engineering would be 18 ' labeling the displays, controls. Again, you know, as 19 defined I guess in Element 25 here. 20 CHAIRMAN APOSTOLAKIS: Right. Now, one other 21, question that just came up. If you look at the various 22 models that have been developed for Human Roliability 23 starting with the NRC Handbook, they talk about performance ,
. 24 ' shaping factors, which presumably include what reason calls
~25 immediate psychological precursors. I don't see any of
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W.,. Suite 300 Washington, D.C. 20005 (202) 842-0034
154 1 those here. () 2 lt_mean, essentially, what you are saying here is 3 that the contributing factors are always management and 4- organization related. -I-don't see anP hing saying, yeah, 5 the guy was depressed that-day and he made a mistake. So, 6 should we -- 7 MR. TRAGER 4 Well, in -- 8 CHAIRMAN APOSTOLAKIS: -- drop this idea of PSPs 9 from the models then? I am not saying that PSPs have to 10 survive, but we can't say one thing in the model and then 11 for the data show something else. 12 MR. TRAGER: I think the PSPs are all in here, 13 contained within. The.-- 14 MR. MOSLEH: Gene, there are two types of PSPs, 15: the. external PSFs. 16- CHAIRMAN APOSTOLAKIS: Yeah. E17 MR. MOSLEH: Which seem to included in these. .The 18 internal or psychological ones. 19 MR. TRAGER: How could be -- that could be 20 included in the individual--- 21 .MR CARROLL: -Issues.
- 22. MR. TRAGER: Issues. And if-you read the 23 . definition of that, that is things like' tunnel vision, for 24 example. .You know, mindset. So those could be internal.
25 CHAIRMAN .'.POSTOLAKIS : Yeeh, they could be.. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite.300 Washington, D.C. .20005 (202) 842-0034
155 1 MR. TRAGER: So, anyway, from these percentages, () 2 3 we can have some graphical illustrations. MR. ROSENTHAL: Can you -- can you go back to the 4 prior slide? 5 MR. TRAGER: _ Sure, j l 6 MR. ROSENTHAL: Because I think we have time, and ' 7 we have anticipated, actually, some interest in this. If 8 you look at the ASP column, what -you see is that procedures 9 are a lessor contributor compared to M&O and system design. 10 In the 1 percent, you know, there is only 1 psrcent of all 11 LERs that make the ASP cut, and now he has moved it up. 12 That is a 10 to the minus 6, 10 to the minus 5. So we are 13 talking about really the important events. 14 This morning the graph that you saw from NRR was
) 15 on all inspections and all LERs and shows a lot of trouble 16 with procedures. But what we are saying is that in 17 important events, using ASP as a criteria, procedures aren't 18 as important. You end up with a. lot of procedural hits 19 across the board because there's a lot of procedures of the 20 plant, so there's lot of opportunities to find things vrong, 21 and it is easy for inspectors to find things wrong with the 22 procedures.
23 But when you look in the ASP column, it iL not as 1 24 important. 25
~
MR. MOSLEH: Is that because ASP hits are events O ANN RILEY & ASSOCIATES, LTD. Court Reporters
-1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
156 1 . involving operators and control room, and like I-saw this () 2 morning, they did -- that the ASP column involves control 3 room issues, as opposed to the database we saw in the 4 morning or in the other presentation? That included not: 5 only. control room, but also -- 6 MR. CARROLL: Maintenance. 7 MR. MOSLEH: Maintenance activities. 8 CHAIRMAN APOSTOLAKIS: Could be. Yeah. The 9- population is different. 10 MR MOSLEH: Yeah. 11 MR LLOYD: This is Ron Lloyd. I think what you 12 were looking at this morning was inspection report issues. 13 And so there, if the Region goes out and they do their 14 inspection modules, and the inspection module says go look () 15 at two of these, three of those, and write your report. A 16 lot of those have to do with looking at the procedure, 17 locking at the surveillance tests that went with it, looking 18 at this and so on. 19 So it is -- and there is a lot of bent on looking 20 at the programmatic side. So when you look at the
- 21 programmatic side, you-are going to find a lot of problems
- 22 with procedures. Either the procedure isn't good enough, it i
j 23 confused somebody. The procedure wasn't there. Then you 24 get into some of the engineering issues. .Maybe the is information, the procedures.aren't as good as they s'..ould be 1 b ANN RILEY & ASSOCIATES, LTD. Court Reporters
- - 1250 I Street, N.W., Suite 300 L - Washington, D.C. 20005
- (202) 842-0034 l
l'
.,,-, .,,._,.=,.,-4.m...._,,,_.--.=.--,% .....-~..,---....m- m.,.,- ~ -- r m m.6 _,__,m.-,,_._,www-y,,mm.-- ,.-+.3,. .
157 1 and so on. () 2 Here, this -- if you are looking at the ASP 3 events, you are looking at the impact the procedures had at 4 creating that particular' event.- So you are beyond looking 5 at conditions,-but you are also looking at actual events where things broke, things went bad. So it is considerably 7 beyond just what is happening in the control room itself. 8 You are looking at equipment that is not responding 9 properly. You are looking at people that don't do the right 10 thing at the wrong time. Things like that. Like Wolf 11 Creek, which was brought up this morning also. 12 The other point I think I wanted to make was each
.13 one of these inspection documents that you have got up here 14 or event documents on the right-hand side has a little bit
( 15 different charter, and AIT has a specific charter that they 16 may go out and look and this is what you are supposed to do 17 .as part of the AIT. 18 For an IIT there's a specific charter that says go 19 out and look at this.- You are looking at root causes in a. 20 lot of senses where.in some of these other kinds of things 21 like inspection reports, that is not the intent. to do that. 22 So each one of them has a little bit different 23- function and a little different charter, which tends to 24 explain the different kind of distribution you get from one 25 part to the next.
=
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
158 1 MR. TRAGER: Some of these things you might () 2 expect, c- some of the things if you consider for example 3- the human performance studies that were conducted, these i were conducted by AEOD. They took place over a couple days' 5 time, the on-site, the-interviewing and data collection, and- - 6 the--report was prepared-on that. 7 The people writing up the report tended to be-8 cautious and something like management and organization 9 being a contributing factor was -- a lot of things would be 10 clear but in an investigation for that length of time it is 11 very difficult to say too much about management an. 12 organization, so that's maybe the reason why it is as high 13 as you might gather from some of the other documents. 16 One of the things here, too, I think -- maybe it's k 15 obvious, maybe it's not obvious -- but the first four 16 columns here, these are all studies performed by the NRC, 17 reports of studies performed by the NRC. The fifth column,- 18 .LERs, are'the reports prepared for-licensees, by licensees 19~ for the studies that were investigated. 20 The numbers you will see in the percent column are 21 much lower than they are, say, in the AIT column, so that's 22 just the numbers, which is sort of interesting I think 23 because the people who are preparing the licensee-event 241 report have the benefit of the AIT report when they are 25- preparing their report, so they have all of the insights in ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
159 1 that AIT study. () 2 3 The ASP LERs are different in that they are events that were never investigated and all we have is what is in 4- the licensee event report. 5 _- MR. CARROLL: Does it bother you to have that many 6 events that-had-safety significance that were-not 7 investigated? 8 MR. TRAGER: Safety significance -- that's a 9 difficult question. 10 Let me try and give r* -- 11 MR. CARROLL: I wouldn't ask it if it ecy easy. 12 (Laughter.) 13 MR. TRAGER: Let me give you my -- 14 CHAIRMAN APOSTOLAKIS: Say yes.
) 15 MR. TRAGER: -- my understanding of why this
'16 happens.
17 This happens because the ASP program, there is a 18 time lag between-the time an event occurs and the time that 19 it is analyzed by the ASP program and put into a database, 20 and the lag may be as much as two years, so when -- for 21 example, you have a loss of off-site power, which is 22 something which automatically gives it kind of a hi h 23 conditional core damage probability, something which know 24 will be analyzed but if it is not put into a database or not determined to be significant for a year, year and a half,
- = = _
O ANN RILEY & ASSOCIATES, LTD. Cor Reporters I 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202)-842-0034 ........._..,,...,...._.i...... .
-u--.m__-. -- - -~-
160 1 it's already too late to do a detailed investigation or () 2 analysis of it, so it is not -- 3- MR. CARROLL: Well, I kept hearing this' morning 4 about all these experienced people with great intuition 5 about events and so forth and they ought to be able to pick 6 these things up.- 7- MR.-ROSENTRAL: Well, let's take a specific 8 ' example from last year. It's a high scoring ASP event. 9 It was at McGuire-or Catawba -- I don't remember 10 which one -- where they had a loss of offsite power in the 11 switch yard for whatever reason. 12 One diesel was out for maintenance and the'other 13 diesel started. The event is over in however many hundreds 14 of milliseconds it takes to open and close breakers and the () '15 few seconds to start the diesel, and there's little time for
-16 human intervention. That equipment all worked, but it ends 17 up being a high ASP event and aas not -- and for which we-H18 didn't have an investigation.
19 The plant was in conformance with its tech specs. 20 It just happened to have -- I think it was in the switch 21 yard'-- switch fault just when one of two diesels was out 22 for service, and it ends up scoring high, so I don't think 23 we have to apologize for that. 24 The other events -- 25' MR. CARROLL: Let's go to the other end of the O' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington,-D.C. 20005 (202) 842-0034
161 1 spectrum. () 2 MR. ROSENTHAL: Okay. Oh, you want me to give the 3 example? 4 MR. CARROLL: Yes. 5 [ Laughter.)
-MR. ROSENTHAL:--At Vogtle another loop-type" event,
-7 lln fact the operators, the operating organization struggled-8 to get a diesel started, and that is an example where --
9 actually there was an IIT of that one -- Gene was on that 10 IIT where, because the charter of the AIT was very hardware 11 oriented the richness of the human involvement in getting 12 the diesel that didn't want to start running was not fully 13 captured, so it's a mixed story. 14 In the ideal world these would be concentric
) 15 circles and they are far from it.
16 MR. CARROLL: So.take that case. Why didn't 17 somebody on the IIT team, was it, why didn't they come back 18 and say, hey, we ought to revise the charter because not 19 that we have gotten into this it looks like there's some R20 things that we ought to really look into? 21 MR TRAGER: But that.is a determination of the 22- team leader and whoever the team leader is reporting to and-23 the determination was made not to look into that. 24 MR. CARROLL: Okay. 25 MR. ROSENTHAL: Nevertheless, when you look at it
- = _=
0 ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
162 1 and you fully understand that the data is impure, for () 2 example an AIT will try to get on and off site in a week to 3 reduce the burden on the licensee. 4 That AIT will work many, many hours and they'll 5 work to the charter, so if the charter doesn't include human 6 factors, you are not going to get any human factor 7 observations, so we work very hard on the front-end to 8 ensure that the charter does include human performance. 9 If you get somebody from let's say Jay's group or 10 my group that's -- or even NRR Headquarters, it's very 11 attuned to these issues. There will be a richer write-up. 12 Many of these issues are beyond the scope of inspections, so 13 there's nothing citeabir Inspectors are reluctant to write 14 to it.
) 15 So we fully recognize that it is impure data. We 16 don't have good denominators because these are all problem 17 type issues or there wouldn't have been any sort of look, 18 and the denominator ought to include good as well as poor 19 performance.
20 Nevertheless, I think that there are some 21 interesting things that shine through and one is that the 22 M&O area seems to be an important contributor to important 23 events if you accept ASP as a criteria for important events 24 or the qualitative AIT or IIT criteria. 25 The performance shaping factors -- the traditional () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
i 163 1 ones like-let's say the bottom three for sure -- don't-seem 2 to'be as big of players. Well, good, Those are e,he 3 operators who trained. Now we know something and I think l 4 that those sorts of conclusions one could draw even ] 5 recognizing that the data is impure. 1 l 6 CHAIRMAN APOSTOLAKIS: As a matter of fact, you i l 7 know, if you exclude, say, individual issues perhaps, other 8 PSPs and maybe, I don't know, the human factors engineering, f. 9 the percentage of management and organization is much higher 10 than 46 percent. i ! 11- MR. ROSENTHAL: Yes, 1 12 CHAIRMAN APOSTOLAKIS: It's hard to tell because i
. 13 they overlap a lot, but it is certainly more than 50
- - 14 percent.
15 -MR. ROSENTHAL: Let me spend just a moment with 16 the system design. l . l 17- In order to be a high ASP event, how do you get 4
- 18 something like a real, a loop or a LOCA,,which drives it, or 19 you end up with disccveri:n that a system wouldn't have
- 20 performed its safety function. -
{ 21 For example, there's on ASP event that involves
- 22 inadequate MPSH, another ASP event that involves pressure '
' 23 - locking of the-valves -- they just couldn't get them open.
j: 24 Now when you analyze that you assume that the period was an 25 . entire year for which_it wouldn't have worked.- In fact, it ANN RILEY & ASSOCIATES, LTD. Court Reporters
- . 1250 I Street, N.W., Suite 300-
' Washington, D.C.-20005 (202) 842-0034 i
q w- -- y- , ,--Y-tr' -
--vv-*t-"- y *~ ~ * - - ' ' '
-em -e- +
e --m-ny,ww- rw-m +-r- -a -e - -
-r r- r
164 1 wouldn't have worked from Day One, and that sort of thing () 2 drives the high ASPS and that is the 54 percent, so I think 3 we understand that. 4 CHAIRMAN APOSTOLAKIS: And these are ASPS-for 5 power, right?
-6 MR. ROSENTHAL: Mostly of power.
7 MR. TRAGER: Well, let me clarify what these are. 8 These ASP-reports were not investigated by the 9 NkC. They had high conditional core damage probabilitieo 10 and they did involve human performance. 11 CHAIRMAN APOSTOLAKIS: What do you mean they were 12 not investigated by the NRC? I mean -- 13 MR. CARROLL: Not necessarily. 14 MR TRAGER: By definition. By definition if an ( 15 event has a high conditional core damage probability but it 16 wasn't investigated by the NRC then it is called an ASP LER. 17 MR. CARROLL: That's what i was asking. Why 18 weren't they investigated?. 19 CHAIRMAN APOSTOLAKIS: I'm sure that they have not 20 been_ investigated -- 21 MR. ROSENTHAL: I'm sorry, Gene. Most ASP events 22 are_from a reading of the LER and for which an independent 23 NRC investigation was not performed. 24 MR. TRAGER: That's correct. 25 MR. ROSENTHAL: But aren't there some for which we ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I' Street, N.W., Suite 300 Washington, D.C. 20005 (202)'842-0034 j
165 1 have both an ASP, an A3P LER, and an AIT or something?
) 2 MR. CARROLL:- There must be --
3 MR. TRAGER: No, that's impossible. It's either investigated or it's not investigated. 5 If it is investigated and an LER is written, then 6 that is called-an LER. 7 If it was not investigated but it had a high 8- conditional core damage probability, it's an ASP LER. 9 There is one other thing, too. It has to involve 10 human performance, and that is determined for events in the 1 11 1990 to 1992-timeframe in the ASP database there is a field-l 12 called " human error" and it is a logical field. It's either 13 yes or no, and the determination was made by the people w!,, }- 14- did the coding for the ASP-database.
) 15 They discontinued use of that code around 1992, so 16 we asked-INEL to go in and make determinations f.'om a 117 reading of the record in the~ ASP database whether it did 18 -involve human performance, so we may be-modifying -- the
-19 numbers may change a-little bit.
20 We_have to go back and review those again. 21 MR. ROSENTHAL: Can you go over the titles for 22 everybody, because I am confused. 23 An'AIT is an investigation by the NRC? 24 MR.= TRAGER: AIT is an investigation by the NRC. 25 -MR. ROSENTHAL: And an IIT?- O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C.' 20005 (202)-842-0034
166 1 MR. TRAGER: IIT is just a higher level. (d I 2 MR, ROSENTHAL: Right. HPS -- 3 MR. TRAGER: A performance study was a study that 4 was performed '90 through '93 by -- 5 MR. ROSENTHAL: By us. 6 MR. TRAGER: By AEOD. A special team inspection 7 is an inspection conducted by the NRC of a problem or 8 problems at a plant -- in other words, by definition it is 9 special. It is not covered by an AIT and IIT or routine 10 inspection module, 11 MR. ROSENTHAL: And that is a regional-based 12 inspection. 13 MR. TRAGER: Regional-based inspection. 14 MR. ROSENTHAL: And they may or may not have had
'(_) 15 assistance from Headquarters --
16 MR TRAGER: Correct. 17 MR. ROSENTITAL: -- in terms of term members, 18 Okay, now we probably should take just actually a 19 few minutes on percent LER and percent ASP LER. 20 MR. TRAGER: For any event that was investigated 21 [ by the NRC, there should be an LER and I think in 99 percent 22 of the cases chere ere, or 95 percent. 23 That is the meaning of this LER column. 24 If an event occurred at a high conditional core 25 damage probability but had not been investi, ed by the NRC, ( ) ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 i L .
= 167 1 and-involved human performance,2it is P- ASP LER. () 2- MR. CARROLL: Wait. Now intvsJaq3ated by . ;he NRC 3- means AIT, IIT or STI -- 4 MR. TRAGER: Or HPS. 5 MR. CARROLL. So -- or HSP.
-6 CHAIRMAN APOSTOLAKIS: But ASP is a separate 7 program,sright?
0- l 8 MR. CARROLL: Yes. 9 MR. ROSENTHAL: Yes. CHAIRMAN APOSTOLAKIS: Accident Sequence 11 Progression. 12_ MR. ROSENTIRL:__ Yes. 13 CHAIRMAN APOSTOLAKI5: These guys are looking at
- 04 an incident. They screen the~ events auJ the ones that.have 151 a likelihood orothe potential for having the probability of li6 -greater than 10 to the=minus 4 are analyzed further.
17 I don?t recall any criterion that you should not 18 do an ASP.if there has been an AIT so I am really confused.
~19- What does that mean,-that they are not
-20 investigated by the NRC? .
21 MR. TRAGER: The ASP program is totally 22 independent. 23- CHAIRMAN APOSTOLAKIS: Right.
-24 MR. TRAGER: And they have certain criteria --
25 -this is my understanding. There are people who know a lot p( ) ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
l 168 1 more about the ASP program than I do -- but an event with ( ) 2 certain characteristics will be investigated. 3 CHAIRMAN APOSTOLAKIS: Right. 4 MR. TRAGER: And as a result of the investigation, 5 that will produce a particular conditional core damage 6 probability for that event. 7 CHAIRMAN APOSTOLAKIS: Yes. 8 MR. TRAGER: So that is an independent database. 9 What we asked INEL to do was take the ASP database 10 and just give us in the 1990 and lator time period all 11 events with a conditional core damage probability of 10 to 12 the minus 5 or greater. 13 CHAIRMAN APOSTOLAKIS: Right. 14 MR. TRAGER: With that database, we matched those O (_,/- 15 against to see whether they were investigated or not 16 -investigated. 17 CHAIRMAN APOSTOLAKIS: And they were not? 18 MR. TRAGER: And many were not. 19 MR. CARROLL: Go to Table 1, George. That'll show 20 you what's happened. 21 CHAIRMAN APOSTOLAKIS: Yeah? What about it? 22 MR. CARROLL: Well, that'll show you how many ASP 23 LERs there were relative to other things that were done. 24 CHAIRMAN APOSTOLAKIS: The thing I trant to clear 25 up in my mind is just because there was an ASP analysis does ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
169 1- not mean-that there was no AIT if there was a human error () 2 involved._ _Because they're independent activities. 3 MR TRAGER:- Absolutely. And there are -- 4 CHAIRMAN APOSTOLAKIS: Okay. Okay. So a lot of
~5 these ASP LERs were in= fact investigated separately -- !
6 MR. CARROLL: No. No. 7 -MR. TRAGER I think you can have -- if you 8; perform an AIT of an event, you can have a high conditional 9 core damage probability. You will have an LER for that 10 eventbydefknition. 11 CHAIRMAN APOSTOLAKIS: Right.
-12 MR. TRAGER: Okay? If you have -- if:it's never 13- investigated it will be aul ASP LER. That's all. !
14 CHAIRMAN APOSTOLAKIS:- That's where you're -- 15 DR. STUTZKE: Now, ASP:LER's to the residue. .They 16 looked at all-ASP events and threw out the ones that had 5J 17 already been investigated, and this is what's left. 18 CHAIRMAN APOSTOLAKIS: That's not what Gene said. 19 DR. STUTZKE: I think that's what-he said. 20 CHAIRMAN APOSTOLAKIS: That's not' what he said. 21 He said that ASP is an independent activity. 22 DR. STUTZKE: That's true. 23- CHAIRMAN APOSTOLAKIS: So they asked INEL to 24 categorize all of those. 25 DR. STUTZKE: Right. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
170 1 CHAIRMAN APOSTOLAKIS: You also asked them to p) i s-2 separate out the ASP analysis that involves significant 3 human errors so that an A7" had been conducteC on them. 4 DR. STUTZKE: Un.- hum . Then they set this aside, 5 CHAIRMAN APOSTOLAKIS: No. Jay says no. 6 MR. CARROLL: Maybe I didn't understand you, 7 George. 8 CHAIRMAN APOSTOLAKIS: ASP is a separate problem. 9 MR. CARROLL: Yes. 10 CHAIRMAN APOSTOLAKIS: They don't care whether 11 other groups at the NRC investigate an event or not. They 12 just look at what happened and they make a determination 13 whether the conditional core damage frequency was greater 14 than 10 to the minus 5 per year. T:nt's what they do. A (,,) 15 MR. CARROLL: Yeah. 16 CHAIRMAN APOSTOLAKIS: Now from that group in some 17 of those events there were human errors that were 18 significa^1t enough for another group of the NRC to decide-to 19 do - - 20 MR. ROSS: No. There are lots of events for 21 which, better or worse, we don't have an AIT and -- special 22 investigation. 23 CHAIRMAN APOSTOLAKIS: Yes. 24 MR. ROSS: And that decision is made the day after 25 the event or the day of the event. O} ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
171 1 CRAIRMAN-APOSTOLAKIS: Sure. But there may be -- l 2 MR ROSS: Six months later -- 3 CHAIRMAN APOSTOLAKIS: Yeah? 4 MR. ROSS: You read an LER, you say -- and you 5 analyze in the ASP program, you decide that it was 6 important, and-then you kick yourself that you hadn't called 7 the team out six months previously. As Gene said, from a 8 human-performance standooint, the trail is cold and we don't 9 attempt to reconstruct -- 10 CHAIRMAN APOSTOLAKIS: Right. And I understand 11 that. That's one round that you may have not done the AIT, 12 but then ASP says it's important. 13 MR. TRAGER: Right. 14 CHAIRMAN APOSTOLAKIS: Let's take the other case
, - 15 now. You realized at the beginning that it was important 16 that you do an augmented investigation.
17 MR. TRAGER: Right. 18 CHAIRMAN APOSTOLAKIS: Six months later the other 19 group that does ASP -- 20 MR. TRAGER: Right. 21 CHAIRMAN APOSTOLAKIS: Also analyzes -- 22 MR. TRAGER: Right. 23 CHAIRMAN APOSTOLAKIS: That sequence of events. 24 MR. TRAGER: Right. 25 CHAIRMAN APOSTOLAKIS: And I thought you said no. (' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 2 84 I b34
172
-1 See, in other words, some of the AITs are-in the 2 ' ASP LERs.
3 MR. TRAGER: I believe-so. 4 CHAIRMAN APOSTOLAKIS: It can't be otherwise, 7 5 because the guys who are doing that are a-completely 6- different group. k 7 MR. TRAGER: -I think I have something a little { 8 farther down the road that might clarify it. 9 CHAIRMAN APOSTOLAKIS: No, but is it correct 10 understanding,. Gene,-what I just said,- that some of the AIT 11 events are also in the ASP-LER -- 12 MR.-TRAGER: No , it can't. Wall -- 13 CHAIRMAN APOSTOLAKIS: No, because -- I mean, the 14 guys at SAIC who are doing it, I don't think that they would 15 say oh, we're not going to do this because they have already 16 done an inspection report. 17 MR. CARROLL: No, that's not the. case. That's not
'18 what they're saying.
19 MR. TRAGER: Could we come back to this a little 20 later. I may have a diagramsthat-might explain this or 21 clarify the -- 22 MR. CARROLL: Okay. Well, regardless of all these 23 nuances we've been talking about, I still.think AEOD and-NRR 24 on the day the event occurred should do a better job of. 25 deciding whether an AIT or whatever is needed. ANN RILEY & ASSOCIATES, LTD. Ccurt Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
173 1 MR. LLOYD: This is Ron Lloyd again. Maybe one () 2 final comment on this, and then we'll let Gene roll on and 3 see if we can clarify some other stuff. 4 We had another table that is not here, and it 5
- tows ASP events at this many. Another column we've got --
6 here's our investigation by the NRC,-whether it was a 7 special inspection, AIT, IIT, whatever it was, those between 8 1990 and 1997 our ability as an agency to pick off the most 9 safety-significant events ranged anywhere from 20 percent to 10 about 85 percent, and it goes back to Jay's comment here, I 11 think we could maybe do a better job, because a lot of this 12 stuff we'll find out afterwards that yes, it was a 13 significant event, and it did not get formally investigated. 14 CHAIRMAN APOSTOLAKIS: Okay. But again the (_/ 15 question that I have is very, very simple. Forget about the 16 tape. Forget about what they told. What is happening out 17 there? The people who do the ASP analysis do not use as a 18 criterion of whett t to do it or not whether there has oeen 19 an AIT investigation. Okay. 20 MR. LLOYD: True. That's transparent. They don't 21 care. 22 DR. POWERS: Let me ask this. 23 I'm really afraid to ask questions -- 24 [ Laughter.] 25 DR. SEALE: I wonder why. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
174 1 DR. POWERS: But d.f in fact the definition as the h 2 speaker gave it that the ASP LERs did not include anything 3 that the NRC investigated, why is the column useful to me? 4 MR. TRAGER: What it does, I think the ASP LERs 5 are events that involve human performance, are significant 6 because -- and by definition having a high conditional core 7 damage probability -- were never investigated by the NRC, it 8 makes them no less important frcm a safety standpoint. I 9 mean, these are still important evel.t s . 10 DR. POWERS: But I can~t derive relative 11 importance of the categories on the left side of the table 12 from that column alone. 13 MR. TRAGER: One of the things you can do is you 14 can -- one of the reasons that this data base was set up was 15 to try to get an understanding of how good licensee reports 16 are, how good investigations are, what's the kind of 17
- information that ought to go into an event report, _
18 information on human performance, what would be most useful, 19 and this is a way of comparing what's being reported in 20 different types of reports, and I think from the 21 percentage -- 22 DR. POWERS: All the column says is that it seems 23 to me is on the whole those events that at come point prove 24 to be significant involving procedures that NRC typically 25 does not institute AITs or IITs or has a lowtr probability h ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
175
- 1. of instituting an AIT or an IIT. That's all it really says,
.( ) J2 MR. CALROLL: Well, now, wait a. minute. Remember, 3 an individual event may -- I'm sorry -- may have many of 4 these-contributing factors. 6- DR. POWERS: That's right. I agree with that. 6 But if it-has a strong procedural-component, there's a lower 7 likelihood that an IIT or an AIT gets-initiated. .That's all 8 it says. 9 CHAIRMAN APOSTOLAKIS: Which table now of all the 10 tables-we have? 11 DR. POWERS: 2B. 12 CHAIRMAN APOSTOLAKIS: 2B. 13- MR. CARROLL: The one he's got up there. 14' I don't know that I agree with that, because --
) -15 DR. POWERS: If I go look at the rest of the 16 tables there's a fair correlation on all the entries, the -
17 magnitudes of the entries, except for those on-the line 18 labeled procedures. There's a fair correlation. I mean, 19 it's not perfect by any means. 20 MR. CARROLL: I would expect for almost any event, 21 . knowing th; procedure happiness that inspectors have, that 22- they're going to find some problem with procedures, whether 23 it was a major contributing factor or not. 24 MR. TRAGER: I think that's the other thing, too, 25 contributing factor, and a lot of the human performance e
\
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
176 1 studies that were done at the time the dectsion was made to h 2 do the study, it appeared as though the plant had dealt very , 3 well with event, and it was only after investigation it was 4 found that not such a good job had been done, which is kind 5 of intercsting, but you'd expect it. If you look thoroughly 6 and you ask a lot of questions about every aspect of what 7 happened when that event occurred, you find that well, maybe . 8 things didn't go as well as everybody thought. It wasn't a 9 textbook case. }, 10 CHAIRMAN APOSTOLAKIS: Well, I think what's / 11 happening here, Gene, is that at least I am overwhelmed by 12 the amount of information that you are giving us in those 13 tables, so can you tell us what the top cwo conclusions are 14 from all these tables? What would you like me to take with 15 me after this? 16 DR. MILLER: Take two. 17 CHAIRMAN APOSTOLAKIS: I want to take only two. = 18 It's a psychological factor. 19 DR. MILLER: I think the amount of data is -- 20 CHAIRMAN APOSTOLAKIS: Well, because we have three 21 presentations times 12. i 22 What is the general message you are sending here? 23 Can you tell me without -- i 24 MR. TRAGER: These are ordered in highest 25 percentage, and I think the message is that manag3 ment and h ANN RILEY & ASSOCIATES, LTD.
. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (2021 842-0034
177 1 organization is most frequently found to be a contributor in
] ) 2 events investigated by augmented inspection.
3 CHAIRMAN APOSTOLAKIS: Only, or in general? 4 MR. ROSS: In general -- well, I came away with 5 three things, and we'll have to clear up this ASP LER. No. 6 1, that management and organization is an important aspect 7 in important events, however you measure. 8 CHAIRMAN APOSTOLAKIS: Okay. No. 2? 9 MR. ROSS: No. 2, althcugh we find a lot of 10 procedural problems, procedures don't seem as dominant a 11 thing in the ASP LERs, which are the most. No. 3, the 12 normal shaping factor issues that we worried about, like 13 work environment, workload, ergonomic-type things, don't 14 seem to be the big hitters. And so we did enough with paper () 15 tape and labeling and get on and worry about other things. 16 CHAIRMAN APOSTOLAKIS: .Okay. Good. Good. 17 DR, MILLER: And communications, I guess. 18 Communications is not as high as I would -- 19 CHAIRMAN APOSTOLAKIS: But
- bat's still --
20 MR. TRAGER: If you compare the AIT column, for 21 example, which correlates pretty much with the percent LER 22 -column, and you compare the numbers, it's just saying that 23 AITs are finding much higher numbers of contributors, which 24 you might expect. Reasonable people might disagree. But on 25 something like training, for example, where training is b
\"'
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
178 1 found to be a contributing factor of 15 percent of the LERs ( 2 for which investigations found the contributor to be 44 3 percent. There are differences. 4 CHAIRMAN APOSTOLAKIS: From the three conc'usions 5 that Jack just gave us I would say the thing that was , 6 relatively new to me is how insignificant the psychological 7 precursors of unsafe acts are, because I did expect 8 management and organization to be very high. And I didn't 9 really think that the procedures were the dominant thing. 10 It's nice that you confirm it. But this other thing about 11 the psychological precursors, I didn't expect it to be so 12 low, and perhaps all these efforts over the years to define 13 those things -- 14 MR. MOSLEH: But they're also more difficult to (__/ 15 identify. 16 CHAIRMAN APOSTOLAKIS: Because you don't really 17 have many incidents, l 18 MR. MOSLEH: Yeah. 19 CHAIRMAN APOSTOLAKIS: Real incidents. 20 MR. MOSLEH; And also those deal with personal, 21 you know, classification, characterization of, you know, 22 people's behavior on an individual basis, and it's not 23 difficult to -- 24 CHAIRMAN APOSTOLAKIS: They are more difficult, 25 but I think it's a fairly reasonable conclusion, that what O Q ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l i
179 1 really matters is everything else, which is consistent with' () 2; the idea of really looking *or the error reports in context. 3 I mean, you can't go all the way and say yeah, everything is 4 due to context. But I think tnis 11 consistent with that
- 5. effort that ATHEANA is undertaking. Right?
-6 And perhaps one of the reasons is if I were to 7 guess that the - _you usually have teams of people, so the 8 psychological precursor -- I mean,:all three of them, or all 9- four of them, have to have the same kind of psychological 10 problem that day, whereas the context is independent of how 11 many people you-have. -Right?
12 DR. POWERS: Surely you're not telling me that the 13 three-foot telephone cord has no bearing -- 14 CHAIRMAN APOSTOLAKIS: The which one? D Ts ,) 15 DR. POWERS: The three-foot telephone cord has no 16 bearing on reactor accidents.
-17 -[ Laughter.] "
1G MR. TRAGER: To go from this to a graphical ---
- 19. this is just an Excel representation of the percentage.
20 MR. CARROLL: We already did that mentally. 21 CHAIRMAN APOSTOLAKIS: Yeah. 22 MR. TRAGER: That's already been done? Okay. 23 CHAIRMAN APOSTOLAKIS: Now,-you know, in the 24- future I would find it at least useful, you know, when you 25 present statistics such as this one, these, to actually give () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
180 1 a couple of examples, you know, real incidents where, you () 2 know, this is more or less a typical thing where what we 3 feel is due to procedures or due to workload. Because I 4 went to the field that defines workload -- well, maybe it 5 doesn't define it, but it says fatigue / stress / workload the 6 contributing factor. It seems to me that a stress and 7 possibly the fatigue are PSFs, but the workload is probably 8 an organizational issue. Right? You don't know, I mean, 9 unless you see the -- 10 MR. CARROLL: It depends on how you want to slice 11 the baloney, 12 CHAIRMAN APOSTOLAKIS: Well, and the real 13 incident, what exactly is it that happened. 14 MR. MOSLEH: There are also interdependencies 15 among these factors. 16 CHAIRMAN APOSTOLAKIS: Well, of course. 17 MR MOSLEH: You know, workload causing stress. 18 CHAIRMAN APOSTOLAKIS: Yeah. Exactly. But I mean 19 that's why you need to see a few examples. For example, if 20 the guy panics, I don't think that's related to the 21 workload. If on the other hand he has to do too many 22 things, he's under extreme pressure, you know, then it's 23' related to the workload. 24 MR. MOSLEH: I think one step further that all 25 these factors need to be mapped to an understanding of how ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 1
181 1 these things interact, an initial model, and tnen you see, () 2 you .now, natural grouping or, you know, insights as to how 3 they interact. 4 CHAIRMAN APOST0uAKIS: No, there's no question 5 about that. 6 MR. LLOYD: This is Ron Lloyd again. What we 7 intend to do for the reports, we've got a couple different 8 reports that will be out. Gene mentioned one that should be 1 9 out within a couple of months. There should he a follow-up 10 report probably later on towards the end of summer. 11 What we intend to do there is look at some of the 12 various groupings and see what we can get from various data 13 sorts. One would include single-unit facilities, multi-unit 14 facilities, regional characteristics, is it a small utility, () 15 is it a large utility, those kinds of things, and we can do - 16 sorts against that. And also using the ASP data base we can 17 look at -- see where those are coming from, where the safety 18 significance might be, and draw whatever the appropriate 19 conclusions might be. 20 Those will be more event-specific and we'll hope 21 to have those in the report. 22 CHAIRMAN APOSTOLAKIS: Okay. 23 MR. CARROLL: That was a table that wasn't in our 24 package? 25 CHAIRMAN APOSTOLAKIS: fes. It's not there. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
l 181 1- these things interact, an initial nodel, and then you see, () 2 you-know, natural grouping or, you know, insights as to how 3 they' interact. 4 CHAIRMAN APOSTOLAKIS: No, there's no question 5 about that. 6 MR. LLOYD: 'This is Ron Lloyd again. What we 7 intend to do for the_roports, we've got a couple different 8 reports that will be out. Gene mentioned one unat should be 9 out within a couple of months. There should be a follow-up 10 report probably later:on towards the end of summer. 11 What we intend to do there is look at some of the 12 various groupings and see what we can get from various data 13 sorts. One would include single-unit facilities, multi-unit 14 facilities, regional characteristics, is it a small utility, () 15 is it.a large utility, those kinds of things, and we can do 16 sorts against that. And also using the ASP data base we can 17- look at - -see where-those are coming from, where the safety _ 18- significance might be, and draw-whatever the appropriate 19 conclusions might be. 20 Those will'be more event-specific and we'll hope
-21 to-have those in the report.-
22 CHAIRMAN APOSTOLAKIS: Okay.
- 23. MR. CARROLL: That was a table that wasn't in our 24 package?
25 CHAIRMAN APOSTOLAKIS: Yes. It's not there. ()_ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
{ 182 1 MR, TRAGER: That is correct. That was the one I () 2 think Ron referred to before. 3 CHAIRMAN APOSTOLAKIS: I don't think we need
)
4 another table, f 5 MR. TRAGER: It may just muddy the water. 6 (Laughter.) 7 CHAIRMAN APOSTOLAKIS: Heard enough. Will you go 8 to Slide 9? 9 Oh, I think you just described that, right? The 10 kinds of things here. 11 MR. ROSENTHAL: The important thing is that we'll 12 be publishing the database -- 13 CHAIRMAN APOSTOLAKIS: Yes. 14 MR. ROSENTHAL: -- and we'll either slip -- unlike 15 the other database that you heard of, this is a lot of text 16 fields in this so it explains what you are really talking 17 about. 18 CHAIRMAN APOSTOLAKIS: Right. 19 MR. ROSENTHAL: And either we'll slip a floppy 20 disk into the back cover of the NUREG or we'll put it on the 21 Internet and people can actually access it. 22 MR. TRAGER: I had made the offer to people at 23 various conferences and meetings that if they were 24 interested in getting a copy of the database, understand 25 that it is changing, continues to change, and it's in an fI\~ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
183 , 1 Access, Microsoft. Access format right now, so if you are (p) 2 interested, just let me know and I can e-mail copy,-attach
'3- it=to an e-mail message. We have had success doingsthat.
4 CHAIRMAN APOSTOLAKIS: This isLthe INEL report you 5 referred to earlier? 6 4 TRAGER: Yes. That's it, if there are no 7 other questions. Thank you. 8 CHAIRMAN APOSTOLAKIS: Thank you. 9 MR. CARROLL: I do have to comment that this is 10 _the third type of stapling-I have seen today.
.11' CHAIRMAN APOSTOLAKIS: Yes. There is only one 3 12 corner that is left.
13 We are hoping that ATHEANA will do that. 14 .Okay, so we will break for -- let's'see. We have 15 a break now between -- 16 DR. POWERS: No. 17 CHAIRMAN APOSTOLAKIS: No, we don't. You're 18 right, we don't. 19 Okay. ATHEANA. 20- DR. POWERS: George, let me ask you a question 21- about ATHEANA. Is that-one of_those= computer codes that has-22 been given to Sandia to work on?
-23 CHAIRMAN APOSTOLAKIS: It's not a computer code, 24 yes,-but Sandia is working on it.
25 DR. POWERS: Oh, okay, so I have to announce that ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
I l 184 1 I have some sort of conflict with something to do with r)- ( 2 whatever this is that we are going to hear about. 3 CHAIRMAN APOSTOLAKIS: Right. 4 DR. POWERS: I can assure you that I know nothing 5 about the subject, but members should bear in mind that 6 anything that I say about it is tainted by whatever conflict 7 I have. 8 CHAIRMAN APOSf0LAKIS: That you know nothing 9 about. 10 DR. POWERS: That I know nothing about. 11 CHAIRMAN APOSTOLAKIS: So there's a zero 12 correlation factor with everything you say? 13 DR. POWERS: You would be safest to do that, I 14 think. O V 15 CHAIRMAN APOSTOLAKIS: And this is Mr. Cunningham? 16 MR. CUNNINGHAM: Yes, sir. Good afternoon. 17 MR. CARROLL: Not here with the usual suspects. 18 CHAIRMAN APOSTOLAKIS: He is not here with the 19 usual suspects. 20 MR. CUNNINGHAM: I feel very lonely up here this 21 afternoon. 22 CHAIRMAN APOSTOLAKIS: He brought another group of 23 suspects. 24 MR. CUNNINGHAM: Yes, I have another set off to 25 the side but they turned down the opportunity to join .e .
\-
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
185
-1 DR. POWERS: Having learned from your other set of
() 2 peers, they sat to the side. 3 MR. CUNNINGHAM: Yes, sir.
-4 CHAIRMAN APOSTOLAKIS: Do we have a copy of what 5 you are about to present?
6 MR. CUNNINGHAM: No. We are distributing it now. 7 CHAIRMAN APOSTOLAKIS: Hand it out now. Okay -- 8 and I see the stapling is-in the right place. 9 Congratulations, Mark. 10 MR. CUNNINGHAM: Thank you. 11 Shall I go ahead? 12 CHAIRMAN APOSTOLAKIS: Yes. 13 MR. CUNNINGHAM: Okay. I am here by myself this 14 afternoon to give you a piece of what is going on in the
) 15 ATHEANA program and specifically the committee requested 16 information on a demonstration of the program that.we had 17 had at the Seabrook station over the last six or eight 18 months.
19 I am going to make the presentation but on Slide 2 20 I should note that basically the presentation I have got' 21 here is an adaptation of a presentation made by the real
- 22. people doing the work at a CS&I conference in Chattanooga 23 about two or three months ago.
12 4 I put this up.here mostly to indicate that there 25^ were a number of people involved in this, including O- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
186 1 contractor personnel to us and four people from the Seatrook () 2 3 Station. CHAIRMAN APOSTOLAKIS: That creates a question 4 right away. 5 MR. CUNNINGHAM: Okay. 6 CHAIRMAN APOSTOLAKIS: I think the reason why we 7 scheduled the meeting for today is that your staff told us 8 that by this time the exercises at Seabrook would have been 9 done, and that you would have something to present, but then 10 you say.they presented this three months ago? 11 This is three months' old? 12 MR. CUNNINGHAM. No. What I have got is an update 13 of what was presented three months ago. 14 CHAIRMAN APOSTOLAKIS: Okay. 3 (Q 15 MR. CUNNINGHAM: Again, most of this is to say 16 that there's many folks involved, including people from 17 Seabrook. 18 Ann Rigby Smith was scheduled to make this 19 presentation but she is in Albuquea ue with most of these 20 people writing up the documentation on ATHEANA in 21 anticipation of a peer review that we have got coming in a 22 few months. 23 I do have with us today Dennis Bley and Catherine 24 Thompson, whose names are on here, who have been at various 25 times involved in the program, as well as Gareth Parry is e-(' gj ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 I (202) 842-0034 l
\
i 187 1 hiding back there, who was. involved in the program early-on () 2 3 in the process. CHAIRMAN APOSTOLAKIS: Is there a reason why he is
'4 hiding?
5 MR. CUNNINGHAM: In his former life as a-
'6 contractor.
7 CHAIRMAN APOSTOLAKIS: You are presenting the
.8 results of the application --
9 MR. CUNNINGHAM: Of-the demonstration, yes. N 10 CHAIRMAN APOSTOLAKIS: We have never really-- - 11-- discussed the model itself, and at some point we will 12 probably have to do that. 13 MR. CUNNINGHAM: I was thinking we'had done-that-14- with the committee but it's been several years ago -- ( 15 CHAIRMAN APOSTOLAKIS: .That was before my time,-I 16 think. 17 MR. CUNNINGHAM: It could have been before your: 18 time, but that is kind of a= vague memory. 19 CHAIRMAN APOSTOLAKIS: Do you think we should do 20 this before you got your peer review or -- when do you plan 21 to have the peer review? 22 MR. CUNNINGHAM: The peer review will be in the-23 April-May timeframe. 24= What we were going to suggest is that when the 25 peer review happens that we would invite all of the members ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 __ 9
188 1 to come, because it is going to be a detailed explanation of () 2 what the model is, an explanation of.what the demonstration 3_ showed in detail, and that sort of thing, so it may be a 4 better opportunity to_see the breadth of what is going on in 5- - the program rather than trying to, if you will, force it _6 into a half a day or a day with the subcommittee or 7 something like that. 8 yCHAIRMAN AP0STOLAKIS: So the peer review-will be 4 9 _ what,'two, three days? 10 MR. CUNNINGHAM: It will probably be three days, 11 yes, 12 CHAIRMAN APOSTOLAKIS: Here in Washington, I hope?
-13 MR. CUNNINGHAM: I think so, yes, 14 CRAIRMAN APOSTOLAKIS: -What do you-think?- That 15 sounds like a good' idea -- so that is when we really get 16 down to the details ---
17 MR. CUNNINGHAM: Yes, that is when you would have 18 an opportunity to really see-the details of what is in the 19 model and how it has been implemented and that sort of 20- _ thing. 21 CHAIRMAN APOSTOLAKIS: What is-the mechanism, Mr. 22- Chairman, about these things? Since this will not be a 23 formal ACRS -- 24 DR. SEALE: But it is an NRC meeting. 25 CHAIRMAN APOSTOLAKIS: No, in terms of our ANN RILEY a ASSOCIATES, LTD. Court Reporters. 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
. _ - _ - _ - . _ l
189 1' committee than I should come back as the Chairman of the 2 Subcommittee and report to the full committee?
'3 MR. CARROLL: Or members that want to go.--
4 CHAIRMAN APOSTOLAKIS: Oh, sure.
-5 DR. MILLER: Invite us all to go, yes. Make sure 6 we're all. aware of the meeting.
7- CHAIRMAN APOSTOLAKIS: Yes, but the mechanism will-8 be what? Like Noel will be there as well and he will take 9 notes? 10 .DR. SEALE: Maybe or maybe not, depending on 11 whether you need the support. 12 -CHAIRMAN APOSTOLAKIS: Can we write a letter = based-13! on a peer review that the Staff has organized? 14 DR. SEALE: -No. 15 CHAIRMAN APOSTOLAKIS: That's.my point. So at 16_ some pointythere will be a' presentation to the full' f 17 committee? 18 MR. CUNNINGHAM: Yes. What we could_do -- 19- MR. CARROLL: I mean'if you come-back an say 20 everything is wonderful, there's probably no need for a 21- letter. 22 CHAIRMAN APOSTOLAKIS: They probably want a letter
- 23 if:I say that.
24- (Laughter.] 25 MR. CUNNINGHAM: Under the circumstances, I think i
.s ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
190 g 1' -we would insist. .5 .2 CHAIRMAN'APOSTOLAKIS: There will be a' lot of 3 pressure to' write a letter. 4 MR. CUNNINGHAM: Under those circumstances we 5- would be willing to make a presentation -- 6: [ Laughter.) 7 DR. POWERS: A less painful set-of circumstances, 8 'you'll be willing to make a presentation as well? 9 MR.-CUNNINGHAM: -Yes. I think it would be good' 10 .after the peer review one why or another. 11 CHAIRMAN APOSTOLAKIS: So you will come to the-12 -full' committee? 13 MR. CUNNINGHAM: To the subcommittee or a. full
'14 - committee after_the peer review and-say -- not give you-i 15 .three; days worth, but saylthis is what we discussed, this 16 is.what came out of the peer review.
17 CHAIRMAN APOSTOLAKIS: Now as far as you are-18' concerned, shrald the ACRS write a letter on ATHEANA at some 19 : point or do you think it's not necessary and it is up to us-20 to decide whether we want to do it?
-21 MR. CUNNINGHAM: I-guess =it would be up_to you
- 22. more-than --
23 CHAIRMAN APOSTOLAKIS: You will not request one? _24 MR. CUNNINGHAM: .No, we are not. requesting-a-25 letter and I know the committee'is interested in the more O ANN RILEY & ASSOCIATES, LTD. Court Reporters
~1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
n <=1 191 1 general subject of what is going on in research -- 2 CHAIRMAN APOSTOLAKIS: Right. 3 MR. CUNNINGHAM: -- in the agency. It may fit in. 4 that type of format as well. 5 CHAIRMAN APOSTOLAKIS: - Okay. That is my 6- understanding too. 7 -MR. CARROLL: Before moving ahead,-Mark, out of
-curiosity, the three, four actually, Seabrook people - _what 9' are their positions?-
10 MR. CUNNINGHAM: I will come tack to that, but two 111 of them are in the PRA group and two of them are in the 12 -- Training Staff at Seabrook, both, the two training people-13 are licensed operators at Seabrook. 14 CHAIRMAN APOSTOLAKIS: .Brookhaven is out? 15 MR. CUNNINGHAM: Brookhaven was not involved in 16 this demonstration. They are -- they were the principal-17 -authors of the Frame of Reference Manual which was used, but 18 'they weren't so much involved in this-demonstration. 19 By way-of background, ATHEANA has been something-
- 20. that has been evolving over the last.several years. Early 21 on, more with-Brookhaven. -More recently with Sandia and 22- subcontractors in both cases.
2 3 -- We have a another -- a number of documents that 24 have'come1out on kind of laying out-the conceptual
-25 background and the experience, operational experience that O ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l : 1
192 1 we have seen that led us to -- or guided us in the early () 2 parts of the program.. 3 The two key pieces of documentation that are out 1 4 now in draft form are the Frame of Reference Manual, which 5 is kind of a technical background document, and the 6 Implementation Guide, which more closer to a procedures 7 guide, that sort of thing. 8 The intents of the -- both documents, the Frame of 9 Reference and the Implementation Guide, are to provide the 10 basic information an analyst would need to identify and 11 characterize and quantify human reliability, this type of 12 human reliability modeling in a PRA.
-13 Before I get into the details, there's two or 14 three things that we talk about in the rest of the
(_.) 15 presentation, to kind of try and clarify what we mean by 16 those some of those things first. One is we have -- we talk 17 about human failure events, and these are what you can think 18 of as the basic events in fault tree or event tree, that 19 sort of thing. It is a representation of a system or a 20 component or a train failure that is the result of one or 21 more unsafe human actions. 22 An unsafe action then is something, an action that 23 is inappropriately taken or not taken when needed by an -- 24 by a member of the plant that results in a, somehow 25 comprising a piece of equipment or a train or a component. [,)
\--
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 2 (202) 842-0034
193 l 1 An erroreforcing context, I heard you talking 2- about before,-in the previous prebentation, basically, it is 3 a situation-that arises within a facility such that it_could _ 4 -- that could impact how the operator performs and the 5 lixelihood of success for performing certain acts, or could 6 lead them to perform acts that they shouldn't be. 7 CHAIRMAN APOSTOLAKIS: Now, this raises an 8 interesting question here which actually raised in my mind 9 earlier, and not only in the context of ATHEANA, but in , 10 other areas as well, like in software. I read the paper by 11 the same people that was presented at the OECD meeting last 1 October. It is actually very good, I mean a lot of insights 3 and it talks about the medium'LOCA at Seabrook. 14 So there is -- the reason I am raising that is 15 your first two lines there. What-they found was that the 16 _ human - the unsafe acts analy::ed, one of the unsafe acts in 17 operator-stop -- stopped the. pump. And they found that-the 18 operators may be lod to believe that RCS pressure is higher 19 than it actually'is. Okay. And the specific error-forcing 20 context associated with this HFE-UA deals with the failure 21- of wide range reactor coolant system RCS pressure 22 indications'in such a manner that the operators may be led 23 to believe that the prejoure is higher than it actually is. 24 Seabrook knows that now. Right? You have done 25 ft. They know it. , = ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) '842-0034
l l 194 1 MR, CUNNINGHAM: Yes. 2 CHAIRMAN APOSTOLAKIS: And they fix it. 3 MR. CUNNINGHAM- Yes. 4 CRAIRMAN APOSTOLAKIS: Nnw, what? In other words, 5 you are doing much m"te than jitst PRh heia. You are telling 6 the plant that you have discovered something that they may 7 or may not choose to correct. Right? To eliminate. 8 MR. CUNNINGHAM: Well, ar.d you could is they are 9 teaching then. delves that. 10 CHAIRMAN APOSTOLAKIS: Yeah. 11 MR. CUNNINGHAM: Because it was principally them 12 doing it. 13- CHAIRMAN APOSTOLAKIS: Right. But the point is 14 that -- and then it is the same thingi you know, () 15 inappropriate depletion of resourcen, another thing. And 16 then other " failures", in quotes, that will not necessarily 17 ;1ead to core damage but may have'the potential to cause the 18 operator problems is a small-generator tube leak and so on. 19 If a-plant-doet-ATHEANA and identifies these 20 things, the plant may choose'to do something about them. 21 MR. CUNNINGHAM: Yes. 22 CHAIRMAN APOSTOLAKIS: Now, where does that leave 23 the PRA? 'Then you have nothing to go with, because they 24 fixed them.__ _ 25 MR. CUNNINGHAM: ENo , they don't -- well -- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
195 1 MR. CARROLL: What do you mean -- 2 CHAIRMAN APOSTOLAKIS: Well, I'll tell you what I 3 mean. Either the plant says, yes, you discover these things 4 but they are of such low probability that we are willing to 5 live with them, then you can do your PRA stuff and say, 6 okay, the probability of this is this. 7 Or the plant can cm.._ back and say, gee, you know, 8 that is intercating. We will make sure that the operators 9 will not be led to believe that the RCS pressure is high, so 10 they eliminate that. 11 MR. CARROLL: No, I don't think-- 12 CHAIRMAN APOSTOLAKIS: Why? 13 MR. CARROLL: -- the eliminate it. I think they 14 reduce the probability. () 15 DR. POWERS: I think what tley do, George, in that 16 particular case, is they do some detailed thermohydraulic 17 studies and advance their instrumentation. 18 CHAIRMAN APOSTOLAKIS: No, but -- no, just a 10 second. Because this is not just your -- it is not an issue 20 just for ATHEANA. 21 MR. CUNNINGHAM: Yeah. 22 CHAIRMAN APOSTOLAKIS: The reason why a lot of 23 people in software reliability or analysis think that the 24 concept of a probability of failure of_ software is nonsense 25 is because they say if I find something wrong, I will fix 0 ANN RILEY & ASSOCIATES, L'ID. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 s
196 1 it. So I am saying -- I am_asking, are we finding ourselves () 2 in the same situation here? 3 Dennis. Identify yourself. 4 MR. BLEY: I am Dennis Bley, B-L-E-Y. I have ! 5 been a little concerned about that in the last year, 6 thinking very much along the lines you have just said. And 7 then it struck me that, almost 20 years ago, when PRA was 8 just beginning, exactly the same thi g was going on. The 9 first PRAs, and you can remember, Professor Apostolakis, 10 because you were involved in some of those, two or three 11 things stood out as the very largest contributors to risk, , 12 and the utilities fixed them immediately, before they 13 published the PRA. Isn't that a good thing? You know, I 14 think it is a good thing.
) 15 CHAIRMAN APOSTOLAKIS: No, I think it is great.
16 MR. BLEY: And I think it is a parallel situation. 17 After it has been applied some, just as after PRA was-18 applied more and more, the number of those drops off and now 19 you are seeing the risk that you are willing to live with. 20 But I think the first few times we do this, we will have 21 that-parallel situation.- 22 CHAIRMAN APOSTOLAKIS: No, but there is a 23- l difference.though, in a parallel -- they are_not exactly 24 parallel situations, because there you had, say, you know, a 25 hundred contributing, the top three were eliminated. You O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
197 f 1 still have the other 97. () 2 But here, you find a situation that may be, you 3 know, an error-forcing context, and they eliminate it. You 4 don't have_anything else. Unless they tell you, no, in-the l _'
-5 second case, inappropriate depletion of-resources, we 6 believe that such -- that is so-unlikely, we are not going 7 to do anything about it, and then you-have a probability.
'8 MR. BLEY: Well, in fact, in this demonstration, 9 there were a number of situations that were analyzed and 10 weren't presented in the results because they were rather 11 small. And, in fact, the demonstration, because of its time 12 and work constraints, had to try to elevate a few things 13 that they thought wou)d be most important and do those, 14 rather than trying to do a reasonably complete analysis. So
() 15 it was a very selective first look. 16 CHAIRMAN APOSTOLAKIS: No,-I understand what you
-17 .are saying, but I mean let's take inappropriate depletion of' 10 resources. Specific error-forcing context involves the 19 failure to receive an empty reactor water storage tank 20 alarm.
21 Let's say now that they make sure that that alarm 22 arrives. I don't.have to worry about this context'anymore, 23 do I? I have eliminated it. 24 _ MR. BLEY: -Yeah. 25- CHAIRMAN APOSTOLAKIS: See, that is the heart of ANN RILEY_& ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
198 1 the argument in software reliability, that is why I am () 2 bringing it up here. Because-Levison will tell-you, I find 3 it, I fix it, leave me alone, there isn't such a thing as a 4- probability-of failure of software. 5 MR. MOSLEH: Unless we represent it, take-those as 6 -cases representing a class of -- 7 CHAIRMAN APOSTOLAKIS: Yeah, right. And then we 8 are back to square one. 9 MR. MOSLEH: But we have similar issues. 10 CHAIRMAN APOSTOLAKIS: They are saying this is 11 everything we found, but we are so uncomfortable we wi.1.1 say 12 it is 10 to the minus 5. 13 MR. MOSLEH: I think, you know, that -- 14 CHAIRMAN APOSTOLAKIS: I mean that is not -- by 15 the way -- 16 MR. MOSLEH: We have the same issue in the 17 hardware as well. 18 CHAIRMAN APOSTOLAKIS: I don't want you to 19 misunderstand me here. This is not a criticism of ATHEANA. 20- This is a larger issue. 21 MR. MOSLEH: Yes. 22 CHAIRMAN APOSTOLAKIS: Okay. You find the context
'23 and you fix it and you have nothing else. What is the 24 _ probability of failure now? First of all, you have really 25 done some good, because the plant now has improved. Right.
ANN RILEY & ASSOCIATES, LTD. Court Reporters-1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
199 1 There-is no question about it. () 2 But in terms of the objectives e.' this project, 3 you may have a problem. 4 MR CUNNINGHAM: Well, yeah, first of-all,-you 5- don't eliminate it, you change the probability -- you reduce 6- the probability of it occurring. 7 CHAIRMAN'APOSTOLAKIS: In some instances. And 8 then I don't have that problem. 9 MR. CUNNINGHAM: Yeah, 10 . CHAIRMAN APOSTOLAKIS: That is the cass -- 11 MR. CARROLL: Well, you get an alarm, George, you 12 put in a second one. 13 MR. CUNNINGHAM: Yes. 14 MR. CARROLL:- Make them -- there is still -- 15 MR. CUNNINGHAM: That's right. There is still a 16 probability of failure there, with that sort of thing. But 17 then you'are -- again, you have got this background noise, 18 if you will, of other types of events that you choose not 19 to, 20 CHAIRMAN APOSTOLAKIS: Yeah. Like they come 21 across failure. 22 MR..MOSLEH: Yeah, it is similar to that 23 situatic.. 24 CHAIRMAN APOSTOLAKIS: What was it EPRI was 25 pfoposing, rectifiability? We find it, we fix it. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 ___ O
l 200 1 Therefore, the d' . abase -is zero events. - That was the O '2 argument in-the '70s regarding -- V 3 MR. BLEY: If I-may -- 4' CRAIRMAN APOSTOLAKIS: No, but that is 5 probably -- 6 MR. BLEY: Dennis Bley, once more. I do agree 7 with you, and we ran into the same thing at NASA when we did 8 work there. We fixed everything. But then you find you 9 can't fix everything. 10 CHAIRMAN APOSTOLAKIS: You can't fix - - #
- 11 -MR. BLEY: So you have those problems. There were 12 some conclusions from the Seabrook demo tnat are -- the 13 group is out in Albuquerque this week because we are writing 14 a detailed report on -- on the experience here at the
( 15- demonstration. 1<6 After it was all over, we began to look back at 17 everything we:had seen, and saw that our scheme fort-setting 18 priorities : 1r picking where we thought the most1important 19 places would be to look needed some-more work. And that 20 came about as a result of things we saw and how -- I would 21 call it how you could characterize the' crews at this 22 particular plant in terms- of a-variety of things that af fect 23 the way they operate, how they communicate, how they carry 1 24 out the work.
-25 We are still working on bringing that together.
O -ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
201 1 But I think what you will-find at the time of peer review is () '2 3 that will be pretty well worked out. And, now, along the lines Dr. Mosleh said, we identified some classes of 4 situations-that make certain kinds of unsafe acts and 5 certain kinds of contexts more troublesome for them, and 6 make others much less troublesome. 7 I think, you know, this is the first try, so I l 8 think we will -- 9 CHAIRMAN APOSTOLAKIS: No , I -- 10 MR. BLEY: -- begin to see those, and I think we 11 will end up with something that is a more sound basis for 12 eviduating the probability than just 10 to the minus 5. 13 CHAIRMAN APOSTOLAKISt. But I think that will be -- 14 that will be a major issue that you will have to deal with () 15 when it comes to actually quantifying things. What is -- I 16 mean if the -- if the owner comes back and says, but I am 17 going to fix this, then what exactly you put back in there. 18 Evidently, you guys have already started thinking about.it. 19 But I would add more there. It is not -- the 20 purpose, I guess, is not just to help PRA. I mean you are 21 actually doing comething that is very valuable to the plant, 22 right? 23 MR. CUNNINGHAM: Yeah. That's fair. That's -- 24 CHAIRMAN APOSTOLAKIS: Maybe the words have to -- 25-' -I didn't expect you to disagree with me, Mark. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202)-842-0034
202 1 MR. MOSLEH: But in that sense, it is not () 2 3 different from what we do in the hardware. CHAIRMAN APOSTOLAKIS: Can you imagine if he said 4 no? 5 (Laughter.) 6 MR. MOSLEH: In that sense -- 7 CHAIRMAN APOSTOLAKIS: Only I find this funny, and 8 Dennis. 9 Yes. Excuse me, Professor Mosleh. 10 MR. MOSLEH: I don't think that in that sense it 11 is different from the hardware case, because even there, 12 when they say, you know, we implemented certain design 13 change, do we give them full credit? 14 CRAIRMAN APOSTOLAKIS: Right. But -- O (s,/ 15 MR. MOSLEH: Unless there is a direct, like a 100 16 percent correlation between the specific cause and the 17 specific effect, and there is no uncertainty - 18 CRAIRMAN APOSTOLAKIS: But I gave you an example, 19 The reason why a group of people in the software reliability 20 area believe very strongly that the whole idea of putting a 21 probability of failure on the software is ridiculous, is 22 because they will tell you, if I find something, I fix it. 23 .And it either fajls or it doesn't. And these are pretty 24 reputable people. I mean they have done a lot of work. So 25 they think-it is-nonsense to try to. (-m
'~# ) 1001 RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
~ .. I
i 203 l 1 So I see some of that here. It may not ce exactly () 2 the same, becauss this is not as deterministic as a code, 3 right. It is not the same thing. 4 But I was struck by it when I was reading it. You 5 know, what if Seabrook said, well, I will fix this. Now, 6 what do you do? 7 MR. MOSLEH: That was my initial reaction, too, 8 when I thought about it. And so this is not really, you 9 know, different from the way that PRA has been used. 10 CHAIRMAN APOSTOLAKIS: No. It is very different. 11 Because for the hardware, it is a different thing. People 12 are more willing to accept -- 13 MR. MOSLEH: Because you see, they feel, there is 14 kind of a more direct physical evidence of change. Here you O k,) 15 are relying on analysis and a lot of subjective -- 16 MR. CARROLL- Or training. 17 MR. MOSLEH: And procedures and training So the 18 link may not be -- 19 CHAIRMAN APOSTOLAKIS: But that is not what they 20 identified. You see, I am willing to accept that for those 21 situations, you know, I am willing to. But it is a matter 22 -- if it is a context that is purely a matter of information 23 arriving in the control room, then I can do something about 24 it. It is not a matter of training and so. I didn't say 25 that all the situations are like that. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.N., Suite 300 WaGhington, D.C. 20005 (202) 842-0034
l 204 1- I mean if they come back and say, okay, we will () 2 train them better. Yeah, I know, I will put a probability 3 on it, you know, I'm no dummy. But if they tell me, no, you 4 really wanted-that alarm, Jay said to, I'll put a hundred 5 alarms. What is the probability of all those, all of them 6 failing? 7 MR. CARROLL: Old beta factor will get you, 8 George. 9 CHAIRMAN APOSTOLAKIS: Oh, and, you know, the 10 problem is I am heavy with the beta factor, Jay, weren't you 11 here when we had that meeting? Ali was here. 12 Anyway, it seems that you guys have already thought about 13 . it , so -- 14 MR. CUNNINGHAM: When we went into the.
) 15- demonstration project with Seabrook we had a number of 16 goals -- the project had a number of goals laid out.
17 Basically we wanted to test out the process as it's 18 described in the frame of reference in the implementation 19 uides. We had a training package that we could put 20- together that' introduced the people not involved with 21- ATilEANA w.'th what's basically in there, and we wanted to 22 -test out, see how successful that was.
-23 We wanted to look into the issue of our 24 capabilities and plant operators' capabilities to identify 25 _ air-forcing contexts that, you know, does it really-make
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 4 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
205 1 sense to go to an operating crew and see if -- and () 2 3 systematically define the more likely the more important air-forcing contexts and that sort of thing. And again, go 4 back and look at ways we could improve the documentation on 5 the process-that we've got. 6 MR. MOSLEH: Excuse me. Has ATHEANA then evolved 7 in the direction of mostly post-initiating-event kind of 8 analysis from the earlier focus which was shot down? 9 MR. CUNNINGHAM: The focus originally -- the 10 genesis of ATHEANA was in shutdown operations, in the sense 11 that we saw the types of human errors that we were seeing in 12 shutdown operations. There was a strong element of errors 13 of commission in that. The model today I don't think is 14 limited to shutdown operations only. I think it's broader 15 than that. 16 CHAIRMAN APOSTOLAKIS: Does anyone -- I think this 17 is the third time I'm asking this question, not today, but 18 does anyone else feel uncomfortable with the word " forcing"?
- 19. Error-forcing context, es opposed to error -- I don't 20 know -- enhancing or inducing. I don't know. You guys said 21 no, that you really wanted it to be forcing.
22 MR. CARROLL: Causing? 23 CHAIRMAN APOSTOLAKIS: Causing? I don't know. 24 It's not cause and effect really, is it? 25 MR. BLEY: If_you return to the word in physics, () SJRJ RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
206 l 1 where most of us put real definitions to it, you know, it [] ) 2 pushes you in the direction of failure, so I don't have a 3 problem with it. I know some people have said it means it 4 absolutely causes an error to occur, but that hasn't 5 bothered any of us involved in the project. 6 CHAIRMAN APOSTOLAKIS: Does it bother you, Tom? 7 DR. KRESS: No. Force is equal to mass times 8 acceleration. 9 CHAIRMAN APOSTOLAKIS: Oh, then -- 10 [ Laughter.] 11 CHAIRMAN APOSTOLAKIS: That must be a new law. 12 DR. KRESS: There's a variable force-and how fast 13 you get to the -- 14 CHAIRMAN APOSTOLAKIS: Now that's what bothers me, O (_) 15 because, you see, if I have F and M I get A. I'm not sure R1:6 if I have context I have error. 17 DR. KRESS: It doesn't imply that F is absolute. 18 It's a variable. 19 CHAIRMAN APOSTOLAKIS: Okay. If you guys feel
'o comfortable about it.
21 Next time I will ask the same question. 22 MR. CUNNINGHAM: We talked about this a little bit 23 before, but in terms of the project -- 24 CHAIRMAN APOSTOLAKIS: No, no, no, no, no, no. 25 Wait, wait. The success criteria is important. Again, this
/~
t ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 ,
._______J
207 (_ 1 is a success -- you have not quantified any probabilities () 2 yet. 3 MR. CUNNINGHAM: Yes, we have. I wasn't going to 4 talk about them.
- 5. CHAIRMAN APOSTOLAKIS: Oh, you have?
6 MR.'CUNNINGHAM: But that's being done now. -Yes. 7 This is -- 8 CHAIRMAN APOFTOLAKIS: And the success criteria 9 refers to that as well? 10 MR. CUNNINGHAM: This is success criteria in the 11 terms of accomplishing the goals of the demonstration. If 12 in order to successfully say we've demonstrated the 13 application of the method, we would say that we needed to 14 accomplish these things in a sense. I'll-come back a little 15 bit later to say what's our sense of what we've 16 accomplished. 17 CHAIRMAN APOSTOLAKIS: So it's -- but still --- 18 DR.-SEALE: Does it'get-you to a place where you-19 want to go?
-- 2 0 CHAIRMAN APOSTOLAKIS: All right.
21 MR.-CUNNINGHAM: In terms of some of the mechanics 22 of what the demonstration -- how the demonstration worked 23 was basically we had a team leader who was a PRA person 24l experienced in HRA but not somebody who was heavily involved
=_
;25) 'n the development of the ATHEANA.
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
208 1 CHAIRMAN APOSTOLAKIS: Can you name these people,
- 2. or --
3 MR. CUNNINGilAM: That was Donnie Whitehead from l 4 Sandia National Laboratories. We had'then the bulk of the 5' team whare che four individuals from Seabrook that we talked 3 6 a little bit about_before, two from the PRA staff, I guess
.7 the names are on the second slide, and then two from the 8 training staff, and these were people who were licensed 9 operators. One was'an SRO, I believe, as well.
10 MR. BLEY: Both are licensed SROs. One has been 11- on the training staff for many years and is probably the key 12 guy in setcing up simulator exercises. The other one was on 13- shift up until sometime in the last year, and he's been on 14- training now. Both work with SROs primarily and with 15 setting up and running the simulator examinations and 16 drills. 17 CHAIRMAN APOSTOLAKIS: So the only person from 18 your team that_was there was Whitehead? 19 MR. BLEY: Ho, And he was not part of the 20 development team. We wanted a third party.to do this 21- demonstration. 22 CHAIRMAN APOSTOLAKIS: So you, Dennis, for 23 example, were not involved in.the demonstration?
._2 4 MR. CUNNINGHAM: We get to the third bullet there -25 that we had support consulting and that sort of thing, and ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l o
I 209 1 that's where Dennis and the other members of the ATHEAHA () 2 team came in. But-it certainly wasn't -- they weren't there 3- full-time or anything like that. 4 CHAIRMAN APOSTOLAKISt- So the guys who did it were 5- -the Seabrook guys with the leadership-of Donnie. 6 MR. CUNNINGHAM: -Donnie Whitehead. 7 _ CHAIRMAN APOSTOLAKIS: Okay. 8- MR. CUNNINGHAM: Yes, that's right. But with the 9 consulting'and things of -- 10 CHAIRMAN APOSTOLAKIS: Sure. 11 MR. CUNNINGHAM: I think you were -- the team was 12 up there maybe -- 13 MR. BLEY: At least one of was up there at every 14 session to do some training to help out when they had 15 questions and to observe. Sometimes more of us were there. 16 MR. CUNNINGHAM: Yeah. When you say session, it 17 was like once -- we were up there like once a month or-18 something.like that? 19 MR. BLEY: Once to twice a month for two to four 20- days. 21 MR, CUNNINGHAM: Okay. Again, this ran we had 22 a training session, a three-day training session in July, 23' and then ran over about a few months. 24- . Within the demonstration itself-the first thing is 25 what sequences, what scenarios, what initiators we want to ANN RILEY &" ASSOCIATES, LTD.
. Court Reporters 1250 I Street, N.-W., Suits 300 Washington, D.C. 20005
-(202) 842-0034
I 1 1 210 i i work from. We've already talked about the one that seemed () 2 to get the most focus, which is the medium pipe break LOCA. I 3 Also looked at loss-of-offsite-power blackout and at some-d 4 ATWS things. 5 Step 2 we went in and looked for -- 6 HMR.-CARROLL: Why did you look at ATWS if you're
- 7 risk-informed?-
[ 8 MR. BLEY: .The reason was because the team from 9 Seabrook had -- their trainers had observed a surprising t 10 -event, and they were very interested in seeing if delving ) i 11' into that they could find anything more significant than 12 they'd already seen. .They had an event where the rods were 13 '- supposed to go in and the operator heard the click, click, 14 click and didn't check visually and actually the rods were () 151 coming out, and it led them to think they really wanted to - 16- look at that a bit with this process. So it wasn't picked 17 because it was thought to be-high-priority from risk, but it , 18 -- 'was when they~were somewhat concerned about and wanted to 19 look at a little more. 20 MR._ CARROLL: And what really happentd? Why did : L 21- that occur?- 22' MR. BLEY: I have to go back and look at it. I l-23 don't --
-24 MR CARROLL: It sounds like an interesting event.
25 MR. BLEY:- Yeah. i O ANN RILEY & ASSOCIATES, LTD. Court-Reporters 1250 I Street,-N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 _ , ._ _ . . ,. _ , -.u ._ -. _ . - . _.u ,- .. _ _ _ _ , _ . . . _ _ ...._ _ _ _ .. _ _ __. _ . _ . _ .
211 1- CHAIRMAN APOSTOLAKIS: Also the medium LOCA, why () 2 medium and not small? 3 MR. CUNNINGHAM: I'll come back to that in a
-4 second, but I was going to step through -- go through the 5_ steps and things like that and we can come back for some 6 more of the details.
7 Again, using the two documents that we gave them 8 we tried to identify what were the HFEs and unsafe actr.-that 9 would be associated with pipe breaks, medium LOCAs, and 10 things like that such that you could have core damage. So
-11 we'll get to I guess the next --
CHAIRMAN APOSTOLAKIS: Clarify something here on 13 this slide. You say for medium LOCA there are three 14 critical functionn, and these are makeup, heat removal, and () 15 long-term heat removal. 16
- MR CUNNINGHAM: Yeah.
17 CHAIRMAN APOSTOLAKIS: When you say.six 18 human-failure events identified, you mean_the'PRA has 19 identified -- 20 NR CUNNINGHAM: These would be -- 21 CHAIRMAN APOSTOLAKIS: You didn't identify those.
-22 MR. BLEY: The beginning of the ATHEANA search 23 process starts with looking at the event tree model in the 24- existing PRA, breaking it apart into-key functions, and then 3
25 for each function asking the question for each functional ( ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
212 1 failure is there a way a human could induce this. Those () 2 were the six HFEs here, not all analyzed in detail, but that 3 were found just by, you know, starting with the event trees 4 and -- 5- CHAIMSW APOSTOLAKIS: But_the important point is 6 that you did not observe six human-failure events. 7 MR. BLEY: No. Only two were chosen for modeling 8 5 the simulation. 9 MR. CUNNINGHAM: Step 3 then was to look at the -- 10 try to systematically identify the potential error-forcing 11 context that would be associated with these unsafe acts. We
-12 had simulated exercises that were performed to-look at the, 13 -you know, did the error-forcing context make any sense and 14 how did the operators perform under those circumstances.
15 Had a quantification step, then a documentation and getting 16 ready for a peer review which is here we're talking about 17- April or it's April-May time frame. 18- Go back now to the medium LOCA and the question-19 you just asked of why a medium LOCA. Again I think this 20 came from the experience of talking to the people at 21 Seabrook. They usually don't get much training on that type 22 of an event, and'it's less clear to them how their EOPs and 23 things would work having:a medium-sized break rather than a 24 omaller or large or something like that. So it's a little 25' more"fuzty to them. () IJM RILEY & ASSOCIATES, LTD. Court Reporters
-1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
213 l 1 The scenarios we looked at included basically the () 2 initiator and the two human failure events that we've got on 3 this page of the next one. We talked about some already. 4 One of them is inappropriate termination of makeup, or ECCS. 5 It gets at the operator chooses, makes a choice to stop the 6 pumps, one error-forcing context associated with that being 7 that he has false indication on reactor coolant system 8 pressure that would lead him to terminate. 9 DR. STUTZKE: Ma .*k , I have some questions on this 10 one. The error-forcing context here. 11 What basically you have here is pressure 12 indication has failed high,-and yet we're claiming the 13 operator han erred because he actually believed that 14 indication, which is what his training and his procedure O
\m ,/ 15 tells him to do. So I have a problem with calling this a 16 human error in general. It's always confused me. Isn't the 17 error the failure to recover, to restart af ter this hae:
18 failed? 19 MR. CUNNINGHAM: I don't think we're calling it an 20 error here. We're calling it -- 21 DR. STUTZKE: It's a human failure event. 22 MR. CUNNINGHAM: Onsafe act. An unsafe act. 23 MR. BLEY: I think we've defined those terms 24 pretty carefully, The human-failure event means it's 25 modeled as a human event in the PRA. The unsafe act means ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
214 1 it's an action by a person. It doesn't say it's an error. ( 2 It might well be --' 3 DR._ STUTZKE: It says inappropriately taken.- l 4 MR. BLEY: It might be that it-becomes more. ! 5 reasonable to them because of the context, and I guess
'6 something Mark didn't say, one of the reasons the trainers 7 lean toward the medium LOCA was because they felt in-itself IB the medium LOCA was something of an error-forcing context.
9 It led to some timing issues and places were decisions occur
- 10 in the procedures that they thought could in fact cause some
, 11 confusion. 12 It wouldn't be as fast as the large LOCA, waere-13 they had lots of training and things were clear. It 14 wouldn't be the slow case of the small LOCA that would get is into kind of a horse race of whether if you go onto RHR or 16- go onto open-loop cooling. There were several other places-
. 17 where they felt it just put the operators in spots they. -t 18 hadn't been before, which in and of itself is considered an -
19 error-forcing context.
. 20 CHAIRMAN.APOSTOLAKIS: Well, that's an interesting '-
21 point that you made. You said the timing of events. 22 MR. BLEY: Yeah. c 23 CHAIRMAN APOSTOLAKIS . Now-as I-understand it, 24 ATHEANA does.not deal with time. Is that a correct-25 understanding? AN!( RILEY & ASSOCIATES, LTD. Court Reporters , 1250 I Street, N.W., Suite 300 Washington, D.C. 20005' (202) 842-0034. .
., mm.w,- y9 %. . . -mw.. .w . h .- g,m., - , r.y, ,,,,..,,. . y, , , , -.m. .__,._7'
_.,.,..g , . _ , _ , . , . , - , ._ ,,-my v w.. ,,,-,y,,,,mn,y.,
i 215 1 MR. BLEY: -I don't -- 2 CHAIRMAN APOSTOLAKIS: I mean, you are not 3 simulating the evolution of an accident.
-4 MR. BLEY: ATHEANA's not a simulation model.
5 CHAIRMAN APOSTOLAKIS: It's not. 6 MR. BLEY: But timing can certainly be an 7 error-forcing context. 8 CHAIRMAN APOSTOLAKIS: In the context. 9 MR. BLEY: Certainly. 10 CHAIRMAN APOSTOLAKIS: I guess I'm a bit confused. 11 Maybe it's a matter of factual information that I don't. If 12 I read that page from the paper, and I think you have the 13 same thing here, you have the MLOCA, the human-fa.Ture 14 events, and Dennis told us earlier these were really in the 15 PRA, they were identified in the PRA, right, the 16; human-failure events. 17 MR.-BLEY: No. The functional failures were 18 identified in the PRA, the fact that they could be 19 human-failure events, human-caused events, was not 20 identified in the model. 21 CRAIRMAN-APOSTOLAKIS: It was done with the 22 ATHEANA. It was done using ATHEANA. 23 MR. BLEY: -Yeah, but ATHEANA asked for every 24 functional failure could a human cause this. 25 . CHAIRMAN APOSTOLAKIS: Well, again, let me ask
=
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
216 1- this. Inappropriate termination of makeup. When was this () 2 identified? Before you went to Seabrook? 3 MR. BLEY: No. The ATHEANA team, which was-Donnie 4 Whitehead and the four Seabrook participants, looked through 5 all-their initiating events from the PRA and picked two or 6 three -- actually-talked through about ten of them - -picked 7 three or four that they wanted to look at in more detail, 8 and for each of those they went through the event trees and 9 asked if they should add any additional functions to those 10 in the event trees, and then they went through and posited 11 human failures at these cases, then they went to the next 12 step to look at associated unsafe acts that could cause 13 those, and then they started setting priorities on those 14 things. 15 CHAIRMAN APOSTOLAKIS: Okay. So -- 16 MR. BLEY: io they picked this. 17 CHAIRMAN APOSTOLAKIS: That team, by applying _ 18 ATHEANA,.was led to identify, to uncover these or discover 19 these error-forcing contexts. 20 MR. BLEY; Yes. 21- CRAIRMAN APOSTOLAKIS: Okay. They could have done
.22 that in Washington. They didn't have to be at Seabrook.
12 3 At which -- I mean I thought you used a simulator 24 there. 25 MR. BLEY: After -- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
217 1 MR. CUNNINGHAM: Yes. () 2 MR. BLEY: -- streets were sel.ected. 1 CHAIRMAN APOSTOLAKIS: And what did the simulator 4 reveal, that this indeed could happen? That the unsafe 5 acts, well, the human failure events were identified that
-6 way.- The unsafe acts were also part of this analysis?
7 MR. CUNNINGHAM: Oh, yes. 8- CHAIRMAN APOSTOLAKIS: Okay, so all this is now 9 analysis. The only thing that Seabrook is contributing is 10 four guys. 11- MR. CUNNINGHAM: Four guys with a lot of knowledge 12 about their facility and operations. 13 CHAIRMAN APOSTOLAKIS: Of course, yes. 14 MR.-CUNNINGHAM: Crew performance and all those 15 things. 16 CHAIRMAN APOSTOLAKIS: Right, but the plant 17 itself, I.mean I am trying to understand what does it mean 18 to apply to Seabrook. Did you actually do anything there, 19 using the simulator --
.20 MR. CUNNINGHAM: Yes.
21 CHAIRMAN APOSTOLAKIS: -- that confirmed'or found 22 something? 23 MR. CARROLL: Using rb= plant design. 24 MR. CUNNINGHAM: When we say we applied it at 25 Seabrook, a lot of it is given the design, operation, __. = _ = -- () IJM RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
I 218 1 procedures, and training of Seabrook. It is also the fac. () 2 that they havs the simulator there -- 3 CHAIRMAN APOSTOLAKIS: Well, did you actually 4- simulate a medium LOCA on the simulator to find somothing, 5 to_see something? 6 MR,-CUNNINGHAM: Yes. 7 MR. BLEY: Yes. 8 CHAIRMAN APOSTOLAKIS: You did? 9 MR. CUNNINGHAM: Yes. 10 CHAIRMAN APOSTOLAKIS: You will talk about it. 11 Okay. 12 MR. CUNNINGHAM: Somewhere along the line, yes. 13 CHAIRMAN APOS70LAKIS: Okay -- I am just trying to 14 understand what -- 15 MR. MOSLEH: But is that the principal tool or
- 16. approach to simulate things, using a simulator and you 17 identify or the thing is actually an exercise more like an 18 analysis where the experts, plant experienced people and the 19_ PRA analysts will have to go through the scenarios and kind 20 -of basically imagine those based on --
21 MR. CUNNINGHAM: There is a strategy for coming up 22 with error-forcing contexts and things like that.
=23- How the simulator-obviously helps is let's take 24 some operators and put'them into that context and see how 25 they perform and I think that is what they were trying to
==
ANN RILEY & ASSOCIATES,-LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
+ - - _ _ _ _ -
219 4 1 do, is to see is this context they have been put in a I- 2 --reasonable one? 3 MR. MOSLEH: As a validation, not as the.first 4 step, right? Which one goes -- I think that is what Georgt j 5' is asking. I
.6 CHAIRMAN APOSTOliKIS: Well, I'm~trying to J
7 understand what happened. I don't have a question or 8- anything. I'm iist trying to understand what happened, but 9 that -- yes, that is the cause of the whole question. . 10 MR. BLEY: Just two or three points. 11 The first one is -- let me start with the , 12 plant-specific and being at Seabrook. 13 When we do a plant-specific PRA, how much do we 14 get from the plant? You know, we can do that in Washington 15 too, but you better go to the plant and make sure it is the 16 way-you said it is and you better get some people there to 17 doublo-check what you have done, so that is why it is-Seabrook-specific ', 19 ' The - simulator . cor ributed in two or three ways. r- 20 The way we never intended it to contribute is we 21 never thought we would put our error-forcing context in an 22 initiating event on the simulator and then we'd score how 23 often they passed or something like that. 24 We-don't think that has any real validity for lots 25 of reasons, some of which-is the whole context can't really
~ ANN RILEY & ASSOCIATES, LTD.
Court Reporters- , i 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
220 1 be simulated, but these are rare events anyway. () 2 What we were looking for in the simulator was to 3 see if in fact a scenario that wae g? aerated using ATHEANA 4 created a cognitively-demanding si.uation for the operators, 5 so part of-it was to observe them in the simulator, but moro 6 importantly it was to talk with them afterward and see what 7 they thought about-the-situation they were in, see where-8 .they thought it could be more difficult or easy, where their 9 training helped, where something etse might help more. 10_ It was.really as a way to get more and deeper 11 input from the crews. 12 While we were there, we observed other simulator 13 exercises and I.think going in I didn't expect we would 14 learn much from that because there wasn't an especially () 15 interesting context for thoso, but in fact we did. We 16 learned a' lot about how their particular crews operate and 17 it is different from how other crews operate, so we learned 18 a lot from that and in the end maybe that's some of the most 19 important things we learned and that was verified again. 20 Now when we ran the simulator on the medium LOCA 21 in fact, although there were other ways to check on the 22 situation, the trainers and other people at the plant were 23 rather surprised that in fact the operators did go as far as
-24 they.did towards saturation without. realizing what was 25 there.
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
I 221 1 They also decided that given what they saw and I) V 2 what they anticipated earlier, they really needed some 3 training in this area, so the simulator served many purposes 4 and going in we were not sure where it would be effective 5 and where it wouldn't but we thought it would be a shame to 6 miss the opportunity to learn from it. 7 I think we learned a lot and we learned way that I 8 think we would recommend that you use it in the future. 9 CHAIRMAN APOSTOLAKIS: I agree that in general you i 10 cannot really replicate the context and so on, but for 11 example, if I look at the inappropriate depletion of 12 resources again, it very clearly states the specific EFC 13 involves "the failure to receive an empty reactor water 14 storage tank alarm. Since the operators are directed by O) (. 15 procedure to stop pumps that take suction from the reactor 16 water storage tank upon receipt of an empty alarm. The 17 failure to receive this alarm may delay operator action." 18 This is very specific 19 Did you create that on the simulator and observe 20 the operators, whether they would actual do this, delay 21 action? 22 MR. BLEY: Yes. J 23 CHAIRMAN APOSTOLAKIS: And they did? 24 MR. BLEY: That was an interesting one, because 25 the way the simulator folks decided to set that up was to O\/ ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
l 222 1 fail one of the IRTUs, part of the computer system, and one h 2 of the key folks on the crew, as they were getting close to 3 this point, said hey, wait a minute, and it wasn't the boss 4 of the crew, said wait a minute, everybody, with IRTU number 5 whatever-it-was out, we may not get these two or three 6 different alarms, so let's watch those things very closely, 7 and he said that within 10 minutes of when we got to this 8 spot, so they in fact covered that one very well themselves, 9 because he was very alert to it. 10 MR. CARROLL: Suppose he hadn't been on the crew? 11 MR. BLEY: Don't know for sure. 12 MR. CARROLL: That is one of the problems. 13 MR. BLEY: The trainers picked that particular 14 area because they felt their people really did rely on that 15 alarm rather than watching the levels, and about half of 16 their crews in goina through a large LOCA, and the timing at 17 that point is reasonably close, about half of them end up 18 getting the alarm half-way through the switchover procedure 19 and have to trip all the pumps first and then start them and 20 so on. 21 They thought that was an area where they were 22 likely to see people have.a problem. 23 MR. CARROLL: Yes, and there is always some guy 24 that takes additional interest in things like how the 25 computer could cause this failure. Not everybody does that. h ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
223 1 DR. STUTZKE: Have you identified an error-forcing
-2 contexts that are not due to instrumentation failure?
3 MR. BLEY: Well, we have certainly identified -- . 4 DR. STUTZKE: That are risk-significant. 5 MR. BLEY: existing events. 6 - DR. STUTZKE: What do you mean by existing evencs? 7 MR. BLEY: Real events that have occurred 8 historically at power plants. 9 DR. STUTZKE. Can you give me un example of one? 10 -MR. BLEY: Particular -- I-did give you some 11 -- error-forcing contexts about the medium LOCA that was 12 -identified and turned ~out. to be = quite true, and that was the 13 ; timing issues of *.he-medium LOCA compared to the way the 14- ' procedures were-laid out and the way lart and small LOCAs-- 15- had progressed.
.16 Communications policies and practices is another 17- one. It tu.ns out on that' issue Seabrook's exceptionally 18 good. They do things I:hadn't really seen elsewhere and do 19 follow very .nnal communications procedures but they also 20- have something, and I don't know how widespread this is in 21 .the industry now -- they call ohem BAGS, standing for 22 Before, At, Going -- and they're just getting up to speed on 23 this. -24 Not all the crews are especially good at it, but 25 at various points along the way, they say wait a minute, b
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.F., Suite 300 Washington, D.C. 20005 (202) 842-0034
224 1 let's stop everything we are doing and have a BAG, and they () 2 talk about what's all happened, have I missed anything, 3 everybody checks everything that has happened up to r. hat 4 point, could we be on the wrong track, could anything else 5 have been going on. 6 They are really checking their situation 7 assessment pretty thoroughly. They g. over where they are 8 right now, and " going" -- where they expect things to go, so
-9 that kind of process if it works well can really help in a 10 situation assessment phase and can help identify when things 11 aren't going the way you expect.
IN However, it is new to them and some of the crews 13 aren't nearly as good at it as others, and in fact the crew 14 we observed had another rule of that sort, which is "Think () 15 it, say it" -- you know, no dumb questions. If something is 16 bothering you, speak up and we'll all talk about it. We 17 won't castigate you for not knowing for sure what is going 18 on. 19 In fact, one person had observed the way the 20 pressure was behaving and thought something was funny and he 21 didn't follow the rule. He didn't get it on the table and 22 if he had, they may have well figured out pretty easily what 23 problem they had. He had taken it far enough to check on a 24 saturation table but he didn't put his concerns out so that 25 other people realized they co; to be looking for another ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 1
225 1 pressure source -- so they have got some things that would 2 real.'y help them out and they almost got this but they 3 didn't. 4 That is how crews do that is another error-forcing 5 context in which these things occur. There are many more. 6 CHAIRMAN APOSTOLAKIS: So what you learned from 7 this application then was that the error-forcing contexts 8 and the unsafe acts that the analysts specify by applying 9 ATHEANA made sense when you went to the simulator, in some 10 sense -- that they were not unreasonable? 11 MR. BLEY: Yes. 12 MR. CUNNINGHAM: Yes. 13 CHAIRMAN APOSTOLAKIS: Okay. Would it make sense 14 at some future time to also see whether by running some 15 cleverly-designed simulator exercises one can laentify 16 additional error-forcing contexts or most likely unsafe 17 acts, or do we always have to rely on the analysts first and 18 then? -- I don't know. I am just asking. 19 Would something unexpected perhaps come from the 20 simulator? 21 MR. BLEY: The one thing I learned in our 22 experience up there was spending some time watching many 23 crews at a plant perform in the simulator will lead you to 24 understand more about the way that they operate such that 25 you will be able to see which kinds of cognitive problems ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
2261 1 are likely to be more severe for these. crews than others or () 2 less severe, so when you compare-them against real
=3 historical events you-can-see why a particular event would 4 be more likely or less likely at this plant, and you would 5 learn a lot from that.
6 I-think the-people doing HRA and PRA at the plants 7 -would learn a lot from sitting in~on the sessions -- 8~ CHAIRMAN APOSTOLAKIS: Right. 9 MR. BLEY: ---and'I am not sure that is done 10 overywhere. 11 It in done for the PRA but not on a continuing. 12 bauis I think. 13- ' CHAIRMAN;APOSTOLAKIS: For example, one other
- 14 thing that might be useful would b. actually create these
() 15 error-forcing contexts,-the same one with different crews 16 perhaps or different-times and so on,-and see what the-17' response will be to the same context. 18 -MR.'BLEY:- Yes.
' 1'9 CHAIRMAN APOSTOLAKIS: This was not done this 20 time, right?-
21 MR. BLEY: No. 22- CHAIRMAN APOSTOLAKIS: Okay. But that would be a 23 useful input. 24- I am also trying to -- you see, one of the things 25 that I think we have to at some point lay to rest, so to O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 1 l l
227 1 speak, is the issue of whether ATHEANA is good enough at 4 ll 2 this time, because there are other models that are out 3 there. 4 Some have emphasized simulation, real time 5 simulation. Other models have emphasized the cognitive =- 6 aspects to operators and they develop all sorts of rules how , 7 the operator might make a decision and so on, and I mean 8 they all have merits. 9 The question is is ATHEANA good c. lough at this 10 time? 11 For example, it would be nice to have some 12 simulation capability, but that probably can wait. Maybe 13 five years from now we can do that if we feel the need. 14 Judging frem what Denny said regarding time, you 15 seem to believe that it is good enough, that we don't need 16 to worry about actual simulation and that we will not 17 discover any error-forcing context that will be a surprise 18 to us just by simulating the sequence. 19 I don't know, but it seems to me that at some 20 point we'll have to start converging. No model is perfect. 21 I mean ATHEANA has certain been scrutinized more than the 22 other models, but what elements in other words that other 23 people have made the central piece of their work really 24 deserve some attention from the ATHEANA people? That is 25 really the question I am posing. l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
l 228 g 1 MR. CUNNINGHAM: We have done one demonstration 2 now for a couple of scenarios and that sort of thing. And I 3 think_we -- we certainly see the need to try this out in 4- other -- other contexts,.if you will, not just necessarily
- immediate LCCA at Seabrook, but other contexts.
6 We have talked about it and maybe applying it to a _7 low -- low power and shutdown conditions, maybe to a 8 con'.ition where you have'a fire, a fire initiate or 9 something. Which may bring a whole different set of demands 10 on the operators and on the model to -- to simulate or to 11- predict estimate what the people are going to do. 12 The other part of it is that while we are doing_ this, we are -- we are again_ jumping to my last slide in 13 14 that sense that we have been talking internationally of what 15 is going -- trying to get a better feel _of what_is going on 16 around the world in what we call second generation HRA
- 17. models.
18 CHAIRMAN APOSTOLAKIS: This is-a second 19 generation.
.20 MR. CUNNINGHAM: Yes, this is one. There are ,
21 others showing up in different places. And we -- through 22 CS&I, we are trying to find what else is going on and get a 23 better understanding and have those people talking to.each
-24 other, if you will.
25 There's a variety of things going on. Nothing ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
p
~
229- [1- . probably of theLocale,-quite, of-ATHEANA,.but certainly 2 there is other work that is going _on, and that is happening 3 in parallel. 4 CHAIRMAN APOSTOLAKIS: I believe -- that'is great. 5 I have no objection to that. I believe that it would be 6- ;useful for you, Mark, for your group, to actually set aside 7 some money and ask some of your people to spend some time,- 8 perhaps with a-helper from outside, for a relatively modest 9- effort, asking that question. 10 Take so-and-so's model that appears to be-11- fundamentally different'from your approach, from_the ATHEANA
'12 approach. Why is it different? And what-elements does this 13- model have that we don't have? And then the next question,-
14 should we have it? 115 Because if you expect CS&I to do that, I mean -- 16 MR. CUNNINGHAM: Well, but -- 17 CHAIRMAN APOSTOLAKIS: I think it will-be much 18 easier, ittwill be a matter of maybe three or four months 19' effort, one'of your staff, because,-you guys, you know, a
'20 . lot of;your people are not only familiar with this, they are 21 f amiliar with other raodels as well.
22 But I would like to see something like that, 23 because I see that as an important step-towards building
, -24 consensus. Because we do need that. We can't afford to 25 spend all this money and develop this, and then have people
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., 3uite 300 Washington, D.C. 20005 (202) 842-0034
230 1- say, gee, you-know,_but they left this out. 2 MR. CUNNINGHAM: Yeah. Just to-be clear on the 3 CS&I. thing, that's - my staff are the ones that are running , 4- -that program with CS&I and Remy Smith is the - .whatever-5 they are,- task gtoup, working group, chairperson on-that, 6 and -- 7 CHAIRMAN APOSTOLAKIS: So-it may happen there, 8- yeah. 9 MR. CUNNINGHAM: Yeah. Yeah. 10 CHAIRMAN APOSTOLAKIS: I don't know what they are 11 doing,:but that_is the kind of thing that-I think would be 12 -useful. 13 MR.. CUNNINGHAM: Yes. 4 14 CHAIRMAN-APOSTOLAKIS: You know, this model did ; I 15- this and.they -- you know, they simulated thht. 16 -MR. CUNNINGHAM: Yes.
- 17. lCHAIRMAN APOSTOLAKIS: Now,_it-is not in our s
.18 -model, but. You know, you may conclude that;you may need to 19 -do that, you know, five years from.now because we have other 20 buyers to take care of-now. Or that it is not-that
-21 important, whatever.
22 MR. CUNNINGHAM: Yeah. Okay. 23- CHAIRMAN APOSTOLAKIS: But it would be nice to 24 have something like that, because I really think then the 25- discussion will be more focused. O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
231l J 1 MR. CUNNINGHAM: Well, that's --- (f 2- CHAIRMAN APOSTOLAKIS: People will not find it 3 easy to dismiss ATHEANA anymore,_or if they have something. 4 serious,--then we will know -- we will'have a context within-5- which-disagrees can be discussed. o 6 MR. CUNNINGHAM: ~Yes. Okay. That's'a --
-7 CHAIRMAN APOSTOLAKIS: Okay.
8 MR. CUNNINGHAM: Okay. Thank you. 9 CHAIRMAN APOSTOLAKIS: So did we destroy , 10 completely your presentation, or shall we try a-little_more? 11 MR. CUNNINGHAM: We can try-and get it back.on 12 track. 13 . CHAIRMAN.APOSTOLAKIS: Yeah, we discussed, I 14 think; a lot of the stuff, 15 MR. CUNNINGHAM: The_ main point on-the -- on slide
- 16. 11 is we talked about the second-HFE. The other part_of it 17 is,_ again,. shaping the context of what is being introduced 18 to the operators and their potential-for unsafe acts and 19 things like that. We build'in other failures that are
'20- occurring 11n:the place that have the -- that have the-21- potential for compromising your ability to think about what 22 is going on and understanding what the right problems are 23 and the wrong problems _are, and that sort of thing.
24 Okay. So we -- okay. Now, that we have gone 25 through the exercise, the question is what have we learned ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
232 1 from .t and that sort of thing. So I am going back to the () 2 basic questions that we were asking. How well do the Frame 3 of Reference Manual and the Implementation Guide work? It 4 seems like the basic search process seems to be working 5 reasonably well, that we are identifying what seem to be 6 reasonable error-forcing contexts, that sort of thing. We
? need to improve how we document and set up the thing in the 8 documentation.
9 MR. MOSLEH: Mark, can you say somethsng ? bout how 10 comprehensive that coverage is? When you march through an 11 event tree, is that basically relying on the expertise and - 12 the judgment of the analyst or -- 13 MR. BLEY: I would say you should not attempt this 14 process without having the person from the utilities PRA
) 15 team who knows that model best, being a member of the team.
\ 16 MR, MOSLEH: PRA model? 17 MR. BLEY: Yeah. 18 MR. MOSLEH: Basic event tree. / 19 MR. BLEY: Ken Kiper from Seabrook is leader of 6 20 the PRA group and he was key guy on our team, and he was 21- able to continually bring out information from the PRA and 22 put it on the table for everybody else. 23 MR. MOSLEH: Then, basically, it is as 24 comprehensive as a group of experts who is familiar with the 25 PRA can make it, right? ( ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Nashington, D.C. 20005 (202) 842-0034
233 1 MR. BLEY: Yes. 2 MR. CUNNINGHAM: With respect to the training,-I 3 think there were some things that -- there was an overall 4 positive reaction from the licensee that we may look at 5 different: ways of implementing the training, in more a 6 piecemeal-fashion, or that sort of thing. 7 MR. CARROLL: Your cryptic statement, more
'8 extensive initial overview directed at plant management,;
9 what is that ---what is behind that? 10 MR. BLEY: The Director of Training at Seabrook
-11 suggested to us that it would have been very useful to have -
12- had a two to four hour presentation in the beginning of the 13 training that was aimed-at management at the plant, and to-14 bring them in, in addition to himself. That's where it came 15' from. 16- CHAIRMAN APOSTOLAKIS: Pretty soon they will come 17 here with laminated cards.
-18 MR. BLEY: They had those.
19 CHAIRMAN APOSTOLAKIS: The dominant error-forcing 20 context. Remember, we had -- 21- MR. CARROLL: Yes. Yes, I do. 22 CHAIRMAN APOSTOLAKIS: You want your context? 23 MR. CUNNINGHAM: I seem to have lost all my 24 laminated cards in these days, so it is -- 25 MR. CARROLL: They are very proud of those. ANN RILEY & ASSOCIATES, LTD. ~ Court Reporters 1230 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
234 1 MR. CUNNINGHAM: Yes. 2_ CHAIRMAN APOSTOLAKIS: Dominant -- 3 MR..CUNNINGHAM: Yes. Getting in the sense 4 ~ of --
-5 MR. CARROLL: It can also_get you into big trouble 6 to over-simplify.
7 CHAIRMAN APOSTOLAKIS: I think we read the paper, 8 Mark, so this -- 9 MR.- CUNNINGHAM: Okay. Well, let's -- 10 CHAIRMAN APOSTOLAKIS: We covered the important 11 stuff. , 12 MR.-CUNNINGHAM: Okay. 13 CHAIRMAN APOSTOLAKIS: You may suggest 14 improvements. Probably they.did. V 15- MR.- CUNNINGHAM: Yeah. Of_ course, they did, 16 and --
- 17. CHAIRMAN APOSTOLAKIS: So'let?s go to the plans 18 for-the future, if you don't mind.
19 MR. CUNNINGHAM: No .' ; 20 CHAIRMAN APOSTOLAKIS: Is there anything that-you 21 really want to tell us?
-22 MR. CUNNINGHAM: -Several things. One is the main 23 -- the big piece, come in the spring, April, May time frame.
24 It is intended to have a second peer review of ATHEANA. We 25 want to bring together what we have -- what we have-in terms ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
235 l1 of the conceptual model, as well.as the demonstration,-and () 2 3 bring it to the table and have a fairly broad peer review of it. 4L DR. POWERS: Mark, when you speak of peer reviews, 5 who are the peers? Are they the usual suspects that show up 6 on all these things? Or is this a different peer review 7 group?L 8 MR. CUNNINGHAM: This could be a little broader. 9 I think we envisioned it as a little broader than the usual 10 suspects. That, you know, we have seen a lot of interest-11 internationally in this, we have - Ethat may bring some 12 'other perspectives.to bear. The other people developing 13- second generation-models may-be right people, good people to 14- bring into this. () 15 DR. POWERS: Let me ask you a philosophical
-16 question about peer reviews. It is my impression,-and 17 correct me if I am wrong, that NRC staff has been very good E18 about soliciting their European partners to participate in 11 9 peer reviews-of what they view as critical computer codes or 20 activities. Something -- and have gotten g..od participation 21 on that. ' ~
22 But there has been limited reciprocity in that. 23- We find lots of European codes being-developed without 24 documented peer review of the type the NRC does, and without 25 asking NRC to contribute to those peer reviews that do O ANN RILEY a ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
236 1 occur. 2 CHAIPMAN APOSTOhAKIS: I think that is true. 3 MR. CUNNINGHAM: It is hard for me to say because, 4 in terms of codes and things like that -- 5 CHAIRMAN APOSTOLAKIS: No, but in general, Mark. 6 DR. POWERS: Not restricted just to codes, but to 3 7 anything that -- NRC has a very aggressive peer review, very 8 devoted to peer review. 9 MR. CUNNINGHAM: I would -- I guess I would -- 10 CHAIRMAN APOSTOLT > I don't think-it is the 11 NRC, Dana. I think it is L.~ way people in the United 12 States do their business. We tend to do that here in a 13 formal way. Look at this Committee. Does any other country 14 have anything like that? We have visitors, you know. It is p/ (_, 15 much formal. Conflicts of interest laws are, -- 16 [ Laughter] 17 -CHAIRMAN APOSTOLAKIS: -- you know, something to 18 tclk about over beer. I think it is a unique American 19 thing, it is not NRC. I mean we have these -- I mean I 20 don't know of any PRA from Germany or anywhere else that 21 went through what 1150 did, or what 1400. 22 MR. CUNNINGHAM: Certainly not with the visibility 23 that that did. You know, many of these PRAs have an IPERS 24 type of review from IAEA, but that is more -- 25 CHAIRMAN APOSTOLAKIS: And in a public forum. ( ANN RILEY & ASSOCIATES, LTD.
\-
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
237 1 MR. CUNNINGHAM: That'is --'yes. That's -- 2 CHAIRMAN APOSTOLAl'IS: But I am sure they review
-3' them internally, but not'in a public forum.
4 MR. CUNNINGHAM: Yes.- Yes, that is right. 5 DR. KRESS: Size will beat it. 6 CHAIRMAN APOSTOLAKIS: What? 7 DR. KRESS: Size -- 8- DR. POWERS: Yeah, at a public inquiry. So it'got 9 a very visible -- 10 CRAIRMAN APOSTOLAKIS: American attitude. 11 MR. CUNNINGHAM: You might extend -- 12 DR. MILLER: You might extend it to an island.
-13 CHAIRMAN APOSTOLAKIS: I might. extend-it to the
='
14 first island that is not part of Europe. 15 DR. POWERS: That's Iceland. 16 CHAIRMAN APOSTOLAKIS: No, really, I mean this is 17[ uniquely here. I was in Europe, spending a sabbatical [ half 18 a year, when Challenger happened and', you1know, there was 11 9 ' the-President's Commission and Feynman said a few things. _20 And I was telling them, you know, what Feynman said and all 21 that. And instead of' commenting on that, one very senior
- 22. -manager at Eastboro turned to me and said, you know, we 23 -really don't have things-like that-here, and-I am sorry for 24_ you.
25 DR. POWERS: Things like Feynman. () ,9NN RILEY & ASSOCIATES, LTD. Court ReporteJs 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
238' 1 CHAIRMAN APOSTOLAKIS: Things like public () 2' concittees, you know, reviewing things and speaking their 3 -peace. ' 4 DR.' MILLER: Saying he was sorry for us? 5 CHAIRMAN APOSTOLAKIS: Yeah. No, for them, 6 DR. MILLER: For them. Okay. 7 CHAIRMAN APOSTOLAKIS: He would rather -- he would 8 prefer to-have that over there to. 9 DR. POWERS: Well, let me -- back to the - -does 10 it-bother you that there is lack of reciprocity, since 11 things, I am not so knowledgeable-about, the criticism that 12 has leveled by.the Europeans at ATHEANA, but-I certainly 13 know about criticisms they:have labeled -- leveled at ' 14 consequence analysis codes, deficiencies they have brought-
) 15= up. Yet, without reciprocity, you can't find out what they' 16- are doing so easily in detail.
17 MR. CUNNINGHAM: Yeah. I guess -- I can think of 18 examples where we have worked very well with the-Europeans and-I can think of others where you just -- you have a very-
~20 difficult time in getting information out of them and things 21 -like that, so it -- maybe it is more case-specific than 22 . general. But I have had both experiences.
23 DR. POWERS: I think what you-are communicating to 24 me is the benefit you get from them in participating in your 25 peer review outweighs any. frustrations you might have of the ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 6 (202) 842-0034
239 1 lack of reci t rocity. 2 Mr., CUNNINGHAM: Okay. 3 DR. SEALE: That's what I meant. That sounds 4 statesman like. 5 MR. CUGINGHAM: Yes. 6 CHAIRMAN APOSTOLAKIS: In fact, in your peer 7 review, you should have at least one guy from over there. 8 MR. CUNNINGHAM: I'm sorry. 9 CHAIRMAN APOSTOLAKIS: You should have at ler.st 10 one person from other there 11 MR. CUNNINGHAM: Yes. That's -- our approach will 12 be go through the working group that we have with PNG-58, 13 with CS&I to try to get the right people into this. 14 CHAIRMAN APOSTOLAKIS: Since we are coming -- 15 first, I hTve two comments on that. First of all, the kind 16 of review you are planning to have will require a lot of 17 planning. I suggest that you choose candidate dates 18 immediately. I am willing to bet you will have to push it 19 to May. 20 MR. CUNNINGHAM: Yeah. I think that is part of 21 our -- we recognize that, that April is just almost 22 unattainable at this point. 23 CHAIRMAN APOSTOLAKIS: I mean even by telephone,
- 24 if you are going to esk people, you know, please mark your g 25 calendar.
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
240 1 MR. CUNNINGHAM: Yes. 2 CHAIRMAN APOSTOLAKIS: We will talk about it
-3 later.
4 MR. CUNNINGHAM: Yes. 5 CHAIRMAN APOSTOLAKIS: Second, since you-are 6 inviting the ACRS to be there, at least this-Subcommittee, 7 would you give us also an opportunity to have an input as to 8 what we consider peers, so we can suggest a few names? 9 MR. CUNNINGHAM: Certainly. 10 CHAIRMAN APOSTOLAKIS: And you are free to turn us l 1.1 down. 2- MR. CUNNINGHAM: Yes, But we are -- okay. 13 CHAIRMAN APOSTOLAKIS: Because, you know, th'is is 14 going to be a serious review and -- 15 MR. CUNNINGHAM: Yes. 16 CHAIRMAN APOSTOLAKIS: And,.in fact, 'I ---I.think 17- that is a great idea. I think it is very nice, it is good !
.18 .that you are doing this.
19 MR..CUNNINGHAM: Okay. 20 CHAIRMAN APOSTOLAKIS: But we have to be careful 21- . who=the peers are and -- 22 MR. CUNNINGHAM: Okay. Some of the other things 23 we have planned down the road, I have mentioned some other 24 demonstrations, either in the -- perhaps in the context of 25 low power shutdown, urban fire scenarios. We have the -- ANN RILEY & ASSOCIATES, LTD.
- /
Court Reporters 1250-I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0031
241 1 trying to link together what is going on with ATHEANA with () 2 what is going on, or trying to analyze management and 3 organizational influences on risk. CS&I work on second 4 generation? 5 MR. CARROLL: Have we heard about that, George? 6 CHAIRIiAN APOSTOLAKIS: About what? I am sorry. 7 MR. CUNNINGHAM: No. (l 8 MR. CARROLL: INEL, M&O activity? 9 MR. CUNNINGHAM: No. That's -- 10 CHAIRMAN APOSTOLAKIS: I have heard some, but 11 not -- 12 MR. CUNNINGHAM: Heard of it a little bit. It is 13 in its infancy, I think it is fair to say. Not nearly in the 14 shape thr.t ATHEANA, not -- 15 CHAIRMAN APOSTOLAKIS: But it has been conceived. 16 MR. CUNNINGHAM: Yes. That's right. 17 CHAIRMAN APOSTOLAKIS: Nothing that is worth 18 ' talking about yet. 19 DR. POWERS: Despite the fact that everyone that 20 has tried to do that before has found no mechanism to make 21 any advancement. 22 CHAIRMAN APOSTOLAKIS: Has been what? I didn't 23 hear you. 24 DR. POWERS: That everyone that has tried to look 25 at management and organization has found no way to proceed, ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
242 1 even looking at it. ()j > 2 MR. MOSLEH: No, some people did. 3 CHAIRMAN APOSTOLAKIS: Don't be too interested. 4 This is serious. ' 5 MR. CUNNINGHAM: We are also trying to continue to 6 look at operating experience, not just in the nuclear 7 business, but in a broad set of complex technologies to see, 8 look for the types of errors that are occurring in bad 9 accidents, be it aircraft or chemical, or that sort of 10 thing, and what do they -- what can that tell us about the 11 ATHEANA model and our ability to model errors of commission. 12 And then a longer-term thing, trying to look a 13 more computerized way of handling and letting -- 14 computerizing the ATHEANA process.
/O
(_)- 15 MR. MOSLEH: I have a question for Mark. This is 16 a question that I am asking when I talk-about our activity 17 on Human Reliability and immediately from the Human Factors 18 and Human Reliability and psychologists, this question comes 19 back. 20 CHAIRMAN APOSTOLAKIS: Engineering psychologists?. 21 MR. MOSLEH: No, pure. Pure psychologists. That 22 what are your plans for validation? And they have all kinds 23 of interpretation. I really don't have a good answer for 24 that myself. And I am wondering if you are giving it -- 25 MR. CUNNINGHAM: I look -- a lot of it comes -- in n ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 i _ - - - - _ _ _
243 1 a sense, is the next to the last bullet, in my mind, and I /~' (T). 2 am being a poor dumb engineer about this. Does -- if you 3 look at events that occur that are bad accidents, does your 4 .model seem to provide some sort of a reasonable match with 5 it? That you could look at -- Bhopal or Chernobyl are the 6 classic ones, and that sort thing, and what is occurring in 7 those accidents in terms of the human performance? Is it, 8 you know, how -- it is like we expected to see? Is it 9 different than we expect to see? And that sort of thing, 10 and help that -- allow that to shape what the model looks
=11 like.
12 It is -- it is tough because we don't have many 13 things like that, but that is why you have to go out well 14 beyond the nuclear business and try and see. Because, to 15 me, an aircraft accident or NTSB types of things I think are 16 very relevant to what we are studying here in terms of how 17 humans perform in complex systems in catastrophes, or bad 18 accidents, or whatever. 19 CHAIRMAN APOSTOLAKIS: I think part of what Dr. 20 Powers said earlier about the peer reviews from Europe 21 applies also to other industries here. I don't-know that 22 any other industry is really doing this kind of thing. You 23 guys are way ahead of everybody else, in my opinion. They 24 may be doing other things, but not what you are doing. 25 DR. POWERS: I think the Navy has tried some O) (
' ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
E 244 1L things in this area. [/l w 2 CHAIRMAN APOSTOLAKIS: The Navy has done a lot of 3 work on teams and for submarines especially and so on. 4 MR. CUNNINGHAM: Yes. 5 CHAIRMAN APOSTOLAKIS: So there may be something 6 there, yeah. 7 DR. POWERS: Flight opece; ions. 8 MR. CUNNINGHAM: Yes. 9 CHAIRMAN APOSTOLAKIS: One other thing before I 10 forget. What, in your opinion, will be the time between the 11 issuance of NUREG 1624 and the peer review? How much time 12 will the peers have to review the document before they walk 13 into the room? 14 MR. CUNNINGHAM: I don't know exactly but -- well, (_) 15 to back up. The peer review, I think, is going to be -- may 16 have -- may have two pieces to it - One is kind of a here is 17 what we are worrying about, and then in a sense come back a 18 little bit later to give us your thoughts on it. But, 19 clearly, we have to have the documentation. 20 CHAIRMAN APOSTOLAKIS: I think that is asking too 21 much of the peers. I mean if you -- 22 MR. CUNNINGHAM: May be. 23 CHAIRMAN APOSTOLAKIS: If you send it to them, 24 say, a month before the meeting. -- 25 MR. CUNNINGHAM: Yeah. ( ( ) ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
l- [ 245
.1 CHAIRMAN-APOSTOLAKIS: -- then you would expect 2- some real input when they show up.
3- MR. CUNNINGHAM: I kind of look that -- 4 CHAIRMAN APOSTOLAKIS: Because that doesn't mean-5 they are going to spend.a whole month reading it, because, 6- you know, the:reasonfwhy they are experts is they are doing 1 7- a lot of other things as well, 8 liR. CUNNINGHAM: I would expect a minimum of a 9 month. 10- CHAIRMAN APOSTOLAKIS: A minimum of a month. ~ 11 Okay. So we are talking about late May or-June for the 12 actual-review. 13 MR. CUNNINGHAM: Maybe,
-14 CRAIRMAN APOSTOLAKIS: Which is fine.
) 15 MR. CUNNINGHAM: . Yeah.
16- CHAIRMAN APOSTOLAKIS: Okay. Anything-else for 17 Mr. Cunningham? 18 [No response.] 19 CHAIRMAN APOSTOLAKIS: Thank you very much. We L20 : will take a break and we will come.back maybe at a few
- 21 minutes'before-4:00, and.it will be a free discussion. So 22 please come back. Okay.
23' Mark.-
-24 MR. CUNNINGHAM: Yes, sir.
25 CHAIRMAN APOSTOLAKIS: Thank you. i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
{ 246 1 [ Recess.) l 2 CHAIRMAN APOSTOLAKIS: We are back. We -- we have 3 asked our invited experts, before we go, to a discussion of 4 what we have today. To make presentations, if they wish, 5 and, evidently, th(v do wish, give us some thoughts. And, 6 of course, you know, prepared this before you heard the 7 presentationc today. So feel free to change things as you 8 find appropriate. So maybe we can give each one of you a 9 few minutes to tell us -- and then you can also give us your 10 general impressions of today. You can do it at the same 11 time, if you wish, or separately. It is completely up to 12 yta. 13 We do appreciate your presence here, realizing, of 14 course, that we are not compensating you for this. 15 So who wants to go first? 16 MR. MOSLEH: I will. 17 CHAIRMAN APOSTOLAKIS: Ali, why don't you go 18 first. 19 MR. MOSLEH: Yeah, I think based on -- 20 CHAIRMAN APOSTOLAKIS: Wai t , wait, wait. You get 21 the microphone and -- 22 MR. MOSLEH: Is this going to work? 23 CHA RMAN APOSTOLAKIS: Nobody knows how. 24 MR. MOSLEH: I think, based on the -- based on the 25 various prosentations today, I think I would like to make a ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
9 247 1 few comments that basically are directed toward the way I () 2 think that different activities at NRC are planned, are 3 being formulated or planned, and then maybe offer some 4- euggestions and ideas. 5 In all,these areas where there is inspection 6 procedures or human reliability -- 7 CHAIRMAN APOSTOLAKIS: -~Maybe -- excuse me for 8 interrupting. But maybe you can as background, because not 9 everybody knows that. Ali and his collaborators at the 10 University of Maryland have developed a model for human 11 reliability analysis. Right? 12 MR. MOSLEH: .Yes. 13 CHAIRMAN APOSTOLAKIS: Not everybody knows that. 14 That's why-I wantedito say that. So -- and it takes a 15 different approach, I think, to the issue than ATHEANA 16' takes, 17- MR. MOSLEH: ATHEANA. 4 18- CHAIRMAN APOSTOLAKIS: I think you tend to 11 9 emphasisithe cognitive aspects of the operators and trying
'20 to model those. You call it IDA?
-21 MR. MOSLEH:' Right.
22 CHAIRMAN APOSTOLAKIS: That stands for 23 Information -- 24 MR. MOSLEH: And that stands for Information,
- 25 Decision and Action, it kind of is a serial model.
ANN RILEY & ASSOCIATES, LTD. Court Reporters i 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 _ -_____1
248 1 CHAIRMAN APOSTOLAKIS: Right. And they have ( 2 . published a few papers. If anyone is interested to read 3 more about it, you know, the papers are in the public domain i 4= and I am sure Ali would be happy to send you reprints. 5- DR. POWERS: Well, I guess would be interested. I 6 would also be interested in what they have done in the 7 nature of peer review and what they have done in the nature 8; of validation. 9 CHAIRMAN APOSTOLAKIS: Sure. 10 MR. MOSLEH: Yes. 11 CHAIRMAN APOSTOLAKIS: Now, you have to remember, 12 this is just -- 13 DR. POWERS: Yeah, that is just between us. 14 CHAIRMAN APOSTOLAKIS: That's the project. ILmean 15 'he can't afford to do the -- but, yeah, that is a legitimate 16- question. 17 DR. POWERS: . Well, yeah, I guess I. understand that 18 it is a university project. But I -- what I want to 19 understand are ".he standards of peer review here as well.
.20 CHAIRMAN APOSTOLAKIS: Okay.
21 MR. MOSLEH: Okay. I will be glad to share that
~
i 22 information in part of my -- 23 CHAIRMAN APOSTOLAKIS: So maybe we can -- why 24 don't we have -- is it really that one paper that says IDA 25- in the title that you -- and don't send five different () ANN RILEY & ASSOCIATES, LTD, Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
249 1 papers. If there is one paper that you would like the 2 members to see. 3~ MR. MOSLEH: I think there are two papers.- One is 4 the basic concept and the model, elements of-the model. The 5 other one is an application-of it on the data, like -- it is 6 the sort of validation that Mark was mentioning. .Like 7 looking at data and correlating to the' data to the model, d CHAIRMAN APOSTOLAKIS: Okay. So why -- 9 MR. MOSLEH: Some -- 10 CHAIRMAN APOSTOLAKIS: -- don't we say then that 11 when you go back home, you will send copies of these papers 12' to Mr. Dudley. 13 MR MOSLEH: Yeah. 14 CHAIRMAN APOSTOLAKIS: And he will make sure that 15 everyone gets copies. 16 MR. MOSLEH: Sure. 17 DR. SEALE: That way we don't burn your-Xerox
-18 machine up.
19 CHAIRMAN APOSTOLAKIS: . Yeah. 20 MR. MOSLEH:- Okay. Yeah. I think'the comments 21- here again relate to the various projects and activities 22 . within different NRC offices, and I think something that is 23 common to all are these four questions. They are all interested in one way or another in factors affect human ~ 25 ' performance, and human performance, in various forms, O ANN RILEY & ASSOCIATES, LTD. Court Reporters
-1250 I Street, N.W., Suite 300-Washington, D.C. 20005 (202) 842-0034
. . . . .__mu----m--- --
250 1 control' room, outside en" trol room, as elements of 2 organization factors, elements of interaction of operators _3 with systems and hardware. So it is kind of a very broad 4 sense. 5 And then we would like to know in which way these 6 performance, the performance will be affected by these -- 7 these factors. So the factors, the causes and the way that 8 the factors affect the performance and the issue whether 9 they are risks or safety significant, should we care about 10 those, in general. And then the next step is prevention and 11 mitigation of their impact. This is kind of common to all. 12 DR. POWERS: Ali, let me ask you a question. I 13 agree with you, those questions certainly get asked 14 frequently. The question that I never seem to get an answer 15 to is have people looked at the equivalent of the risk-worth 16 of the human in these systems and somehow concluded that 17 human performance is unacceptably bad? Or that the 18 uncertainty in human performance is sufficiently large that 19 we need to do something in this field? 20- MR. MOSLEH: I think it is a little bit of both. 21 Although my personal opinion is that it has been mostly the '22 uncertainty about the significance of it, that we really 23 don't have -- our-numbers tell us that the human activities
- 24 are significant contributors to risk. But we don't have 25 cont 1.dence in those numbers.
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
251 1 So those-who kind of don't go back to the roots of () 2 those numbers and take the results at face value, tend to 3 say, well, most PRAs are dominant and many scenarios are-4 dominated by human activity, so we ought to do something
-5 about that. But fundamental question is are they dominant, 6 and are they not coming basically from our n 'ers which, ,
7 for which we really don't have a basis? Yc know, the same 8 basis that we find in other aspects of PRA. 9 DR. PO*AERS: I have people tell me frequently when 10 risk issues come up, that we have done a vast number of PRAs 11 and it shows that the plants are safe enough, even relative 12 to standards like the safe goals, they are safe enough. 13 Even within that kind of context and that boundary 14 condition, do we still think that human performance is () 15 sufficiently bad that we need to do something? 16- MR MOSLEH: Again, I think if we look at the 17 operating experience, for instance, as an indicator, you 18 find most accidents, events that have been recognized as 19- ' significant in the operating history have elements or pieces 20 of some human interaction; sometimes actually on both, on 21 positive and negative side. 22 And I think that probably is contrirating to the 23- concern that people express about the importance of human 14 activities or human element in the risk. -
'25 I-Fc..t think that the argument based on numbers O ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
252 l 1 is -- can hold the test of, you know, closer scrutiny. 2 CHAIRMAN APOSTOLAKIS: Maybe it is an unfair 3 question, but to pursue it a little further, there is a 4 widespread belief that the PRAs have strongly indicated that 5 the quantitative health objectives have been met by the 6 plants. Granted that, you know, shutdown risk and all that 7 maybe has not been analyzed in as detailed manner as the 8 power operations. But do you think that if we eventually 9 get a model like ATHEANA or some other model, that will make 10 us have confidence in the numbers? That this conclurion 11 might be out of turn? 12 I mean are we talking about something that is as 13 significant as that? Or maybe if we go down to the core 14 damage frequency and LERF, the large early release () 15 frequency, we will see a more dramatic impact there, but the 16 QHos perhaps will not be affected as much? Or is something 17 we can.not answer right now? 18 MR. MOSLEH: Well, probably we cannot answer it, 19 you know, with -- based on evidence and then it really is a 20 strong basis. But I think, again, personal opinion, if 21 anything, you probably are over-estimating, even in the 22 current PRAs, the impact of human contribution to -- I mean 23 the contribution of human op<3rators into the instruments. 24 . _ CHAIRMAN APOSTOLAKIS: So you think the trend kill 25 be -- p~ ' t-- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
253 l 1 MR . 110SLEH : I think so. 2 CHAIRMAN APOSTOL7 SKIS: Down. .Marty, do you agree 3 with that? 4 DR. STUTZKE - Yeah, I do. Very much. 5 CHAIRMAN APOSTOLAKIS: That's interesting. 6 DR. STUTZKE: Operators are not the dumb idiots 7 that HRA analysts typically treat them as. 8 CHAIRMAN APOSTOLAKIS: Well, but it is not a 9 matter of only the operators. I mean if you look at 10 incidents -- 11 DR. STUTZKE: Even organizetions are not this 12 stupid, 13 CHAIRMAN APOSTOLAKIS: But look at what happens 14 out there. 15 DR. STUTZKE: Absolutely. 16 MR. MOSLEH: Maybe I can make a comment about the 17 events that Gene and AEOD, and the first time I -- at the 18 beginning of my -- my activity in human reliability, I was 19 . asked by Gene to review'his 20 events that have been 20 analyzed in detail. And when I look at it -- one of my
-21 first impressions was like these operators are heros. They 22 recover under very -- you know, the accident and the 5- 23 situation, under very, very difficult situations. And we 24_ see so many of those. We see some errors too.
25 So I think there is a mixed -- mixed bag. You () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
254 1 -know, it is not that when we have a human element-and () 2 interacting during an accident and all that, that the -- 3 MP.. RUBIN: You have to agree, though,--that the 4 -operators at TNI didn't exactly cover themselves with glory. 5 MR. MOSLEH: Yes. And I think that is precisely, G I think -- 7 DR. STUTZKE: And we fixed that problem.- 8 CHAIRMAN APOSTOLAKIS: What did you say, harty? 9 DR. STUTZKE: And we fixed that problem at TNI. 10 CHAIRMAN APOSTOLAKIS: It is 20 years ago, 11 DR. STUTZKE: We made it go away. 12 DR. MILLER: The training programs are 13 dramatically different. 14 DR. STUTZKE: Yeah, I have got a viewgraph. 15 CHAIRMAN APOSTOLAKIS: It is it a fair impression 16- then that I have, that the result of all these efforts 17 perhaps will shift our attention to other things? Maybe the 18 operators are doing fine, but like the context, trying to 19 fix that a little bit? Most importantly, look at 20 organization and management issues that have led to 21 incidents that, you know, have embarrassed the industry or
<22 : some real incidentc. It will be a shifting, in other words, 23 of attention and resources, rather than having --
24- demonstrating that we are not meeting the criteria.
- 25. DR.'STUTZKE: I tend to agree. I don't think the ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
( 255 1 risk now is in the control room or even in the operators () 2 3 outside-the control room. I think it's in the management and it's in the regulutors. 4 CHAIRMAN APOSTOLAKIS: Regulators? 5 DR. STUTZKE: Well, if I see a $2.1 million fine 6 against Millstone, I wonder why was it allowed to persist. 7 Why did it escalate to that magnitude? This question was 8 asked earlier today. There's something wrong here. It's an 9 organization or it's an industry. There's something wrong 10 here. 11 DR. MILLER: You're looking at the big system. 12 DR. STUTZKE: Very big system. Even bigger than a 13 single utility, the industry. 14 DR MILLER: The system includes the industry and 15 the NRC. 16 DR. STUTZKE: Sure. 17 CHAIRMAN APOSTOLAKIS: That's-what I meant when I= 18 said that the emphasis would shift. 19 DR. SEALE: The capacity to screw things up is 20 generally vested in -- 21 DR. STUTZKE: Oh, and all the support groups like 22 INFO, NEI, EPRI. We all have our roles to play. 23 CHAIRMAN APOSTOLAKIS: So that will be really the 24- benefit of all these efforts.- And I personally think that 25 this emphasis-now on context is a healthy step. It's a good ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1250 I Street, N.W., Suite 300
-Washington, D.C. 20005 (202) 8d2-0034
256 1 step forward for everybody, not just, you know, the PRA () 2 analysts, but the industry and the regulators. 3 Okay. Good. So why don't we let you go. 4 MR. MOSLEH: So this is common to all, and I like 5 to make an observation that given the-questions I want to 6 say that this particular problem, human performance 7 predicting and assessing irrespective of the application is 8 a very difficult problem. And the point is that we should 9 not.take it lightly, that with some effort maybe big in the 10 context of an organization resources but modest compared to 11 the size of the problem, that that can be handled and 12 addressed to a level that would bring the level of 13 confidence that we want to have in these models that an 14 organization can really individually and alone address that. 15 That it takes more resources and an effort and maybe a more 16 active participation in collaborative work to address this _ .17 thing in a fundamental way. And whatever expectation that
-18 we have from our activities should really be seen in the 19 context of the resources we are willing to commit to this.
20 I don't expect, didn't expect at the beginning of 21 my work or ATHEANA's work or the next generation that some 22 of the fundamental questions that need to be answered before 23 this thing is raised to the level of a thermohydraulic kind 24 of analysis -- 25: CHAIRMAN APOSTOLAKIS: Raised to a level? () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
. - - . - . - .- . - - . .- - . - - . - - . - . - . _ - . . - ~ - . _ . . -
1 l i 257 1 1 -MR. MOSLEH: Reduced. Of a scientific kind of and l () 2 a technical confidence based on the science and engineering i 3 that needs to go into this to make it a predictive model to i 4 ha,e a better understanding of why mistakes are made and j 5 how -- and that's a direction as you indicated earlier we 6 have been taking, try to go a little bit deeper, to have a 7 better understanding of.the fundamentals, although one can 8 always have a shortcut, have an engineering approach and 9 have results that would be good enough for certain i
- 10 decisions. And I think that's probably where ATHEANA and 11 similar efforts-are leading.
12 DR. POWERS: I_look at the questions you've posed [ 13 here and I certainly think they're accurate questions. It 14 seems to me that whenever we discuss risk assessment that-() 15 another question arises in connection with human performance 16 usually prompts a soliloquy from our Chairman about how j 17= unfair life is that PRA get to ask all these. questions but
- l. 18 deterministic analyses aren't. That question always.
19 revolves around the deficiencies of PRA and its inability to 20 .now treat errors of commission. And isn't that what this I l 21 ATHEANA program'is attempting to address, that it's-trying , 22 _to find out if it's possible to inject within the PRA some 23 formalism for handling errors of' commission? 24 MR. MOSLEH: Primarily, I think, yes. 25- DR, STUTZKE: Sure. 4 () ANN RILEY & ASSOCIATES, LTD. Court Reporters i 1250 I Street, N.W., Suite 300 L Washington, D.C. 20005 (202) 842-0034 L - .
i 258 1 CHAIRMAN APOSTOLAKIS: Yeah. Nodding that -- we () 2 have a record to say yes. 3 DR. STUTZKE: Yes, that's correct. 4 CHAIRMAN APOSTOLAKIS: If you agree, of course. 5 MR. MOSLEH: How, related to the questions, the 6 HRA needs are and have been for some time from the beginning 7 that if they want to have a model or a methodology that can 8 be used in practice, in application, we need to have a 9 systematic method of identifying errors and integrating them 10 into the risk models, and we need to have a credible -- and 11 we can interpret the word " credible" in many different ways, 12 model for quantifying the errors. 13 And my sense is that in terms of practical applied 14 methods we still have some work ahead of us. We still have 15 some work ahead of us to get to that level of credible 16 quantification and a systematic method of identifying and 17 then modeling the human errors within the framework of 18 current PRAs. And I think that's also a key point. 19 And the reasons -- some of the reasons for this is 20 that the current PRA modeling framework, because of some of 21 the factors that I've listed there, is static, passive in 22 nature, hardware-oriented. They're not really suitable for 23 really a comprehensive context modeling. 24- CHAIRMAN APOSTOLAKIS: .Well, that's an interesting 25 statement, AIi. ATHEANA is static. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
259 1 MR. MOSLEH: Yes.
,,- ~ .
2 CHAIRMA11 APOSTOLAKIS: And the context, and I 3 believe Marty asked-that question earlier, the definition of 4 the context ceems to be hardware-oriented. 5 MR. MOSLEH: Yes. 6 DR. STUTZKE: Yes. That's my concern. 7 MR. MOSLEH: And that's in our opinion, in my 8 opinion, it's not sufficient. 9 CHAIRMAN APOSTOLAKIS: Now what else would you add 10 to it? 11 MR. MOSLEH: The context is a combination of the 12 plant, physical response of the plant, the hardware 13 condition, operators' immediate history and their activity 14 during the accident. These together cannot be addressed in (3 ( ,/ 15 a comprehensive way by just marching through the event 16 trees. That's the type of exercise that Garreth Parry and I 17 did for the Borsela nuclear powerplant about five, four 18 years ago, and it's basically this. A manual implementation 19 of a simulation that need to take place if you really want 20 to have the right picture or adequate picture of the 21 context. . 22- And that's why I was aFking Mark and I think 23 Dennis to what extent the simulator exercises are an 24 integral part-of implementing ATHEANA. Is it a validation 25 or it's an integral part of it, because personally I think O- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
-1
260 1 it's -- you can do a -- you can identify scenarica, .. !ew
/~'
(Aj 2 scenarios, but there is no way of assuring a meaningful 3 level of completeness, and it becomes an exercise of expert 4 judgment basically. And I think we do a lot of that in PRA. ; 5 But in some areas we are guided and helped by the 6 eagineering tools. In thermohydraulics we also rely on 7 expert judgment, as does the apply-to-success criteria, but 8 we are aided ay the engineering and the science of 9 thermohydraulics. We don't have that here. 10 CHAIRMAN APOSTOLAKIS: So are you saying then that , 11 the simulator should be used also to try to define some 12 context that perhaps the analysts by looking at the event 13 trees would not be able to identify? 14 MR. MOSLEH: I say it would be very difficult for O ( ,/ 15 the analysts to think about all possible combinations and 16 even go through a screening process of finding the 17 combination of error-forcing conditions, id(ntify them, and 18 then consider the combination together in a systematic way. 19 CHAIRMAN APOSiOLAKIS: So you are agreeing then 20 that a simulator should be a tool to do that? 21 MR. MOSLEH: Yes. Yes. 22 CHAIRMAN APOSTOLAKIS: Because that's not how they
- 23 used it this time.
24- MR. MOSLEH: Yes, I understood it that way, yeah, 25 that I think it's essential again -- this is a significant O ANN RILEY & ASSOCIATES, LTD. Court Reporters
- 1250 I Street, N.W., Suite 300 L Washington, D.C. 20005 l (202) 842-0034
261 1 step forward already, I want to state that. On the other () 2 hand, again I want to make sure that at least it is seen as 3 an evolutionary step toward something that in my sensing is 4 inevitable, that at some point we really need to bring these 5 error-forcing conditions in as an integral part of the 6 modeling activity, and so that the model generates and 7 guides us in the direction of identifying them. 8 C11 AIRMAN APOSTOLAKIS: What does passive mean? 9 MR. MOSLEH: Passive in a sense that basically 10 analyst sits down and then, you know, it's a reflection of
-11 the analyst's understanding of the system. The model itself 12 doesn't generate anything. The event tree when you draw it,
- 13 it's what you cooked up, basically. That's your model of i
14 the accident scenario, in contrast to a simulation program
) 15 that can present you with a lot of surprises. Of course, l
l 16 that's also subject to the state of the art and the 17 engineering, you know, level of engineering and science. 18 But given that -- 19 CHAIRMAN APOSTOLAKIS: Well, your approach is not l-l 20 simulation, is it? 21 MR. MOSLEH: It is simulation. It's tied to a 22 simulation. I don't know of other ways of really capturing 23 the broad spectrum of possible error-forcing functions. I 24 CHAIRMAN APOSTOLAKIS: Yes, let me put Nathan on 25 the spot here. You have done work on simulating accident I ANN RILEY & ASSOCIATES, LTD. j Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 i I
262 1 sequences. Do you agree with what Ali just said? That was () 2 before he joined the agency,-by the way. 3 MR. SIU: I agree with Ali that what we're seeing 4 is an important step forward, that ATHEANA does take you 5 part of the way there. I agree also that it doesn't take us 6 all of the way there. What I don't know is the answer to 7 your earlier question, how much do we really need to make 8 che decisions we need to maka? 9 I mean, what we're -- what Ali's saying is 10 conceptually correct as far as I understand _it, that there 11 are lots of things that you have to worry.about if you're-12 really worried about modeling as deterministically as 13- possible. Obviously there's some variability in this 14 process. But if you're trying to put in as much of the 15 phenomena of f ailure into your model as possible, you have 16 to do things at the much more detailed level, and you have 17 to worry about things like the history, how did the 18 operators get to a particular state of mind regarding where 19 the system is. 20 Do we really need that for the decision making? I 21 don't know. You know, in that sense I'd suggest, you.know, 22 we continue in the current vein, and as we learn more about 2? it, maybe we'll find out that there are places where we're 24 weak and where we need to bolster.
-CHAIRMAN APOSTOLAKIS: I remember you published a ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
263 1 paper a few years ago where you analyzed one sequence and () 2 you had the operators as a-part of it and so on. Woulduit 3 be useful now for you to go back and ---because at that time 4 1 don't think we were talking about context that much. I 5 mean, that's already several years. 6 MR. SIU: We were talking of context. We didn't 7 use the term necessarily. But go on. 8 CHAIRMAN APOSTOLAKIS: Okay. Maybe we were not 9 paying attention. Let's put it that way. But if you went 10 back, do you think you would be able to identify contexts 11 from that exercise that perhaps ATHEANA would not identify 12 following the current thinking that goes into it? 13 In other words -- 14 MR. SIU: We had more things-in our models than 15 what's covered in ATHEANA, that's true. So in principle 16 maybe. I mean, it was also in a way a structuring of expert 17 -judgment. We had -- essentially the paper you're referring 18 to I believe is human factors experts working through a 19 dynamic version of an event tree where you're stepping 20 through in time and.asking how the operator's model of the 21 plant is changing over time, and other issues as well, and 22 these are conditions that are not being asked by ATHEANA.
-23 Tney are being asked I'think in an expert-opinion sort of 24 manner.
25 You're trying to get the trainers, the operator () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
264 , 1 examiners-to put together all their ideas of what they've () 2 seen and come up with a nominal' scenario that tests the l 3 operators under certain conditions, and that's fine,-that's ; i . 4 very useful, but like Ali says, it doesn't test a13 possible ' 5 combinations. And we don't know JLf all possible i 6 combinations need to be addressed. But if there is a lot of l l i 7 sensitivity to your assumed nominal scenario, then of course l 8 you have to go farther than what we're seeing right now. I 9 CHAIRMAN APOSTOLAKIS: Why don'i you send a copy 10 of that to Noel? 11 MR. SIU: Sure. 12 CHAIRi APOSTOLAKIS: He will distribute it. 13 Thank you. 14 DR. POWERS: Before he gets away, maybe I need a 15 little more background here. Nathan,=you-said that ATHEANA 16 is a step to there, but it doesn't get us all the way there, ' 17- and I don't understand what "there" is. 18 MR. SIU: What Ali was referring to, which -- his 1 4 19 inevitable development of the full integrated plant operator 20 model. That's my understanding of what you're talking o e 21 about. o 22 MR. MOSLEH: 'Yes. Yeah, yeah. 23 MR. SIU: Dynamic, probabilistic, the whole bit.
-24 CHAIRMAN APOSTOLAKIS: But is there any reason why_
25 _you think -- not just you, Nathan, but Ali and Marty --- that () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250-I Street, N.W., Suite 300 Washington, D.C. 20005 _(202) 842-0034
265 1 ATHEANA could not accommodate that say a year from now? Is . 2 there anything-that -- 3 MR. MOSLEH: The problem is too big, George, to 4 consider all these -- S CHAIRMAN APOSTOLAKIS: Oh,-I understand that it's 6 too big, but if it's big for them, it's big for you. 7- MR. MOCLEH: No, but you rely on computers and 8 simulators to run the scenarios for you. You put the basic 9 principles there and let the machine run. 10 CHAIRMAN APOSTOLAKIS: No, but my question is 11- this. Is thera something thht's so fundamental and missing 12 from-ATHEANt that they will never get-there, or is it a 13- matter of, you know, finishing with this phase and then 14 maybe a year from now to go back and say now let's look at 15 simulation accommodated in our model?- 16 MR. SIU: It's poJsible'you could work-with it in 17 the.same way. I guess you kind of -- people had to work 18- with,1you know, multiple dimensional problems with, say, 19 single-dimensional models, and the computing capabilities 20 were limited. One could envision some sortHof quasi-static 21 approach.- But basically ATHEANA does not address time-22 explicitly, doesn't address thermohydraulic process
-23 variables explicitly. And that'- che sort of thing that you 24 would be'using-to establish the context, the dynamic context 25' for the operators. Again, what it's doing_is it's making O ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
266 1 the expert integ' a information internally and then () -2 come up with an assessment as to what's a likely scenario. 3 DR. SEALE: But isn't the expert the simulation? 4 MR. SIU: Well -- 5 DR. SEALE: On the system behavior. 6 MR. MOSLEH: But this is more than just system 7 behavior. 8 MR. SIU: How operators respond to the system. 9 DR. SEALE: I understan that. 10 MR. MOSLEH: You're right that the expert's 11 playing the role of the simulator or the simulation. You're o 12 absolutely right. You know, that has to be done that way. 13 You're imagining scenarios, and you're guided by some model, 14 model of the plant, and your understanding of the system.
) 15 DR. SEALE: What you're really relying on in part 16 in your warm feeling that you were talking about earlier is 17 that so'somebody screws up and you've got an initiator 18 you've kicked into the system now. One of the real 19 questions that strikes me is whether or not you adequately 20 represent the consequences of that initiator in ways that 21 provide the opportunity for subsequent intervention. That 22 is, is the data there that tells you what the plant state is 23 and what the inputs are and makes those inputs available to 24 you so that omeone can successfully intervene to overcome 25 the consequences of that initiator?
O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
s.________.._..--__.m._ - 267 1 I think you asking for a level of representation () 2 from the simult. tor that is probably higher than we have in 3 come cases now because you really want to be able to look at j 4- pretty fundamental consequences of that initiator to see-if l 5 there is place you ccn go to grab-the plant back, if you 6 will.- 7 MR, SIU: Honestly'I don't know if you need a real 8 high resolution simulator. Again, you are trying to provide 9' a couple of things. 10 You are trying to provide set points for operators 11 to take actions or things that they will be keying on and 12 you are also trying to provide information that the-13 operation will be using in situation assessment. 14 DR. SEALE - Yes. 15 MR. SIU: And that information may not have to be I 16 very highly detailed if you accept that there are
-17 uncertainties in your models and there are uncertainties in 18 your input parameters in the initial conditions'that you -
19 . start fuzzifying things always, again I don't know that you 20 really need a really high fidelity simulator.
"21 - MR. MOSLEH: For instance, one of the 22 error-forcing conditions would~be the trend and the 23 acceleration of a changing parameter, not the exact value.
24 - MR. SIU: - People have postulated using qualitative 25 physics models, for example, in these kinds of situations, O- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
268 1 and I don't know if that is a reality when it comes down to () 2 working with the kinds of problems we have, but again, it's 3 like I always say -- if you got trends and you can talk 4 about being in different states without having to say the 5 temperature is precisely this. That may be good enough. 6 CHAIRMAN APOSTOLAKIS: I never took a course in 7 qualitstive physics, by the way.- Qualitative physics? 8 MR. SIU: Qualitative modelling of physical 3 9 processes, you know, digraphs. 10 DR. POWERS: There is an excellent book called 11 " Physics for the Non-Mathematician" -- 12 CHAIRMAN APOSTOLAKIS: That's not for me then. 13 (Laughter.) 14 11R. CARROLL: Al Lewis taught a Freshman course 15 like that that you probably should have attended. 16 CHAIRMAN APOSTOLAKIS: But he doesn't use any 17 equations, you know. The problem with that is while you are 1E reading it you think you understand it but at the end I am 19 sure how much you understand. 20 So basically.what you guys are saying is that the 21 thermalhydraulic response and neuronic response of the plant i 22 in the simulation -- 23 MR. MOSLEH: And the operating response -- l 24 CHAIRMAN APOSTOLAKIS: And the operator reaction 25 to that may define additional contexts that right now we ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
269 1 don't have, but I don't know that -- I mean after ATHEANA () 2 does its thing, you can always supplement. it by something 3 like that, right? Or yo" hink it's so fundamentally 4 different -- 5 MR. MOSLEH: No No , I think it's actually 6 fundamenta31y the same. 7 ATHEANA is going in fundamentally the same 8 direction as these simulation or dynamic, except that the 9 simulator is the expert. 10 CHAIRMAN APOSTOLAKIS: Right now it is the expert, 11 yes, 12 MR. CARROLL: Well, I guess I haven't kept up with 13 simulators in the last two or three years, but my impression 14 is that they don't do very well in severe accident space. () 15 MR. MOSLEH: I think of course in all these cases 16 we are still limited by how good our model of the plant 17 behavior is and how good our model of the operator we have 18 really is -- 19 CHAIRMAN APOSTOLAKIS: I guess Ali's argument is 20 that_they may not do it very well, but they are still better 21 than an expert -- 22 MR. CARROLL: I don't think they do it at all. 23 DR. MILLER: They are not designed to do it. 24 CHAIRMAN APOSTOLAKIS: Really? So what kind of 25 training are we giving -- () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 i
270 1 MR. DUDLEY The requirements for the simulator is () 2 to model design basis acciden'.s -- 3 DR. MILLER: How Sell did they do on those severe 4 desiqn basis accidents? 5 DR. KRESS: Thy ' ve. not that bad. 6 CHAIRMAN APOSTOLAKIS: Like what? 7 DR. KRESS: Well, they input source terms, for 8 example, rather than calculate them. 9 DR. MILLER: How well did you determine hydraulics 10 and neuronics? 11 DR. KRESS: As long as you don't have to deal with 12 containment -- 13 CHAIRMAN APOSTOLAKIS: Could they model the medium 14 LOCA in the sequence that Mark and his colleagues talked to () 15 us about earlier? 1 16 DR. KRESS: Yes, they do. 17 MR. CARROLL: As long as you recover from it. 18 DR. SEALE: Mark has a comment. 19 MR. CUNNINGHAM: They could the medium LOCA as an 20 initiating event. Where you may get into trouble is 21 multiple failures of equipment and as the core conditions 22 would start to degrado. That is where you would run into 23 trouble. 24 MR. CARROLL: Rignt. 25 DR. SEALE: Crew recovery, they could handle that LI ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300
""""*"Si"64?: 63
271 1- kind of thing. () 2 MR.. CARROLL: Now most utilities use_their 3 simulator for emergency plan exercises up to the point 4 that -- 5 DR. SEALE:__The_ wheels come-off -- 6 MR. CARROLL: -- the wheels come off and then you 7' sort of play games. 8- DR. SEALE: Yes. 9 MR. MOSLEH: We are not going to be able to puch 10 ~ the PRA beyond our btsic capability of simulating and-11 understanding the physical processes, fundamental ones. 12 If we don't have basic understanding or we have no 13 way of predicting that behavior, then our PRA models also 14 stop there. () 15 If you can control the uncertainty and can think 16 of approximating the behavior, that is what is done. 17 CHAIRMAN APOSTOLAKIS: Okay. Back to your -- 18 MR. MOSLEH: Here what_I was saying is that you 19 still have --.we do not have a scientific, theoretical.or 20 empirical basis for identifying and relating context to 21 performance. The link between cause or environmental 22 factors to performance is still a subjectlof -- 23- CHAIRMAN APr.rOLAKIS: That is what bothers me 24' with the word " forcing" --.you guys can turn me down. 25- When we say " error-forcing context" and then~I
# \
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite-300 Washington, D.C. 20005 (202) 842-0034
)
272 1 -look at this second bullet there -- () 2 DR. MILLER: You are saying that requires a
-3 scientific --
4 CRAIRMAN APOSTOLAKIS: No, what I am saying is 5 that when I hear " error-forcing context" I understand that 6 if this context is-there, then there is a very high 7 probability that the operators will do something, but it is 8 predetermined, and now Ali in telling me no, there is no 9 real e*idence to connect the actual behavior with the 10 context. 11 Is that what you.are saying? 12 MR. MOSLEH: Yes. 13 CHAIRMAN APOSTOLAKIS: So why is it " forcing" 14 then? O
's ,/ 15 MR. MOSLEH: Enough so that we can have, we say 16 okay, now 4.t's enough for our modelling and predictive model 17 of operator behavior. That connection, that link is still 18 based on again expert judgment, some human factor results, 19 isolated exercises, sometimes extensive exercises, on the 20 impact of one parameter on behavior, sometimes run through a 21- .model of cognition but often just a correlation.
22- DR. MILLER: So there is some empirical basis. 23 MR. MOSLEH: There's a lot of empirical 24~ correlations. There's some modelling activities. Some of 25 these models are down to details of our understanding of the () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Wash'ngton, D.C. 20005 (202) 842-0034
273 1 mathematical operations, you know,-how kids learn and things () 2 like that. 3 CHAIRMAN APOSTOLAKIS: If I look at the example _ 4 that ATHEANA presented, that the operators do not receive a ' 5 signal that the tank is empty, you can't really say -- 6 DR. STUTZKE: To me that is forcing. That is 7- exactly forcing. : 8 CHAIRMAN APOSTOLAKIS:- It's a pretty strong 9 -context. 10 DR. STUTZKE: Sure, it is. 11 MR. MOSLEH: Sure, sure. 12 DR. STUTZKE: They should behave exactly as they 13 have been trained, as their procedures tell them to do. 14 Turn the pump off.
) 15 CHAIRMAN APOSTOLAKIS:- So that is not the kind of 16 -context you-are referring to?
17 MR MOSLEH: No, but that is an example of a case 18 where you say there is a direct relationship between what 19 the operator is observing or not observing, and what he is 20 trained to do, and of course in most cases operators do 21 .what -- a footnote there. 22 One of the values and benefits of going through ,
- 23 the exercise of looking at generic classes of contributing 24 factors, abstractions from the actual detailed context in.
25 the context of scenarios, when you have a model to take O- , ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 1
274 L 1 these various situations to a higher level of abstraction is () 2 that when you analyze events then you see new things that 3 you will not 133 able to see through just a linear 4 relationship between or a one-to-one relationship between 5 context and performance. 6- We saw-that in the application we had in applying 7 the item model to the AEOD events. We saw a lot or 8 surprises there. 9- CHAIRMAN APOSTOLAKIS: But you are not arguing 10 that the emphasis, the new emphasis on context, is 11 misplaced? 12 MR. MOSLEH: No, not at all. No. Absolutely not. 13 - In fact, if there is any hope in coming up with a 14 meaningful quantification of error probabilities it is in 15 the direction of making conditional probability assessment 16 conditioned on the context and in those cases that that-type 17 of assessment is a lot more reliable, more robust and easier 18 to make. 19 It might still be subjective but in a specific 20 ' situation, in the case that you, you know, the ATHEANA 21- example is a case where the conditional probability uf error 22 or the acticn is almost one -- because the context is so 23 well-defined that the operator would be crazy, a normal
, 24; . operator would not do anything but that, so it's earlier to 25 assess the probability and all you need to do is the
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
275 1 aasessment, estimation of the probability of the context or () 2 condition, and I think that is a major step forward in 3 getting us closer to a more meaningful quantification and 4 the data-gathering for that. 5 CRAIRMAN APOSTOLAKIS: That again, speaking of 6 time, that does not include the possibility that three 7 minutes later somebody will realize from the indications 8 that what they are getting is not the right context, that 9 something is wrong. 10 DR. STUTZKE: Excctly. 11 CHAIRMAN APOSTOLAKIS: So that's the thing. 12 DR. STUTZKE: That's the point I was trying to 13 make earlier. The recovery is the human failure -- the 14 failure to recover that -- and quite honestly I don't see 15 why it is an error of commission because you are still 16 operating under symptom oriented EOPs. 17 You will get symptoms, cues to the operator 18 something is wrong, and if they fail to react to is, okay, 19 the core melts down, but in my mind that is nothing more 20 than a recovery error or maybe an error of omission. 21 Now I mean part of the problem here is that HRA 22 analysts play games with the words. The terms mean 23 different things to the different individuals and it is a 24 very slippery subject, I think. 25 The other point I'll toss out is ATHEANA didn't () ANN RlL2V & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
i 276 1 invent the context as a concept. It'has always been around () 2 in HRA. What they have done is to formalize it and make it 3 more tractable, more practical -- what do you really mean by 4 that? -- make it visible, and that is a worthwhile step. 5 MR. MOSLEH: Absolutely, absolutely, yes. 6 CHAIRMAN APOSTOLAKIS: You want to wrap it up? 7 MR. MOSLEH: Yes. 8 CHAIRMAN APOSTOLAKIS: Oh, I'm sorry -- 9 MR. PARRY: George? Garreth Parry, NRR. 10 Just as a matter of clarification, I believe that 11 the definition of human failure events in ATHEANA mean that 12 there is an initial unsafe action and that it remains 13 uncorrected sufficiently long that an undesirable 14 consequence occurs, so that the recovery that Marty was () 15 talking about or the failure ta recover is embedded in the 16 definition of the event. 17 Now I don't know how that was taken into account 18 in the demonstration exercise. I can't comment on that but 19 in the original document, that was the intention of the 20 definition. 21 CHAIRMAN APOSTOLAKIS: Right, but -- do you assume that the original context remains the same and they just
.; failed to recover, or the context may change with time?
24 MR. PARRY: It could be -- logically it could be s 4 25 bota. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Stre'Jt, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
t 277 1 DR. STUTZKE: That's the hard part, the context -- (v ') 2 MR. PARRY: Oh, yes, it would be hard to do it, 3 but you are supposed to think through the scenario through 4 to its conclusion and if the context is an evolving context 5 I think that becomes part of the problem, so if there are 6 new signals that come in, they should be considered in 7 defining the human failure event. 8 It is not easy to do. > 9 MR. MOSLEH: You see how quickly it becomes 10 intractable because there are so many branching points at 11 the points where contexts may change -- this way, this way, 12 this way -- 13 CHAIRMAN APOSTOLAKIS: I think what may happen A 14 here is that you start out with, say, the way ATHEANA is km ,)- 15 structured now and you screen out a' number of contexts for 16 which.you do not need a detailed simulation. 17 MR. MOSLEH: Yes. 18 CHAIRMAN APOSTOLAKIS: And perhaps at the end you 19 have a few scenarios, a few contexts where you feel that 20 things might change in ways that you cannot really predict, 21 the way Ali says, you know, by just sitting in a room, and 22 au need more information, because to do a detailed 23 simulation is very costly so you really don't want to do 24 that for every -- so some sort of screening. 25 I can see ATHEANA, say a year or two from now, () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 642-0034 _---------a l
278 1 evolving that way, identifying contexts and criteria for h 2 identification of these contexts where you need a more 3 detailed analysis. 4 MR. MOSLEH: I was actually hoping that I would 5 hear that the simulation or simulator exercise would become 6 more of an integral part of ATHEANA even if it means like 7 basically toward the end where you have screened out event 8 scenarios. 9 CHAIRMAN APOSTOLAKIS: Okay. Yes, you want to -- 10 MR. MOSLEH: Very quickly, there is nothing new in 11 here except that some of these basic concepts I did not see 12 being really implemented or taken seriously in some of the 13 activities we heard today, that when you deal with 14 observation it has to sit in a context of a model and there 15 is an interplay in the role between observation, an 16 evolutionary role of r.iodel development, observation, model 17 development, observation, et cetera, you know, that we see 18 some activities that are essentially observation-oriented 19 and some that are essentially model development effort, and 20 I think we need to have a more meaningful interaction 21 between these two pieces needed for solving a particular 2? problem. 23 'fe saw many dif ferent definitions that are 24 classification of the same set cf issues between different 25 offices of NRC and I think at some point it would be nice to ll ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
279 1 have a consolidation of thase and I think what was a very lh 2 successful '.>rmula that was applied in the common cause 3 failure analysis case and I think could be applied here, 4 where you try from the beginning or during the process to 5 bring the broadest possible participation of the main 6 experts and I think you indicated that earlier today, and I 7 think that is essential for ATHEANA and then similar other 8 projects at NRC to reach a level of acceptability that they 9 might desire. 10 DR. STUTZKE: We are looking for the handouts like 11 , that, and I will make the formal disclaimer about possible 12 conflicts of interest since my company is involved in the 13 development of ATHEANA, but not me. 14 CHAIRMAN APOSTOLAKIS: Your company is very large. 15 DR. STUTZKE: Yes, there are 30,000 of us now, 16 so -- 17 CHAIRMAN APOSTOLAKIS: Two or three authors that 18 Mark talked about are from SAIC -- Susan Cooper -- 19 MR. CUNNINGHAM: Susan Cooper and Alan 20 Kolczkowski. 21 MR. CARROLL: I was just -- what I thought was 22 impressive was the 30,000. 23 DR. STUTZKE: Oh. We just Bellcore. 24 DR. SEALE: It's hard to contemplate that many 25 Beltway Bandits. f ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 f (202) 342-0034
280
- 1. DR. MILLER: Not that many beltways.
() 2
-3 DR. STUTZKE:
live here in Washington. I am just grateful they don't all 4 ' [ Laughter . ] ~ 5 DR. STUTZKE: What I tried to do was to think
-6_ through how human reliCaility research relates back to the 7 regulatory process a little bit. 3 8 It's a little difficult because I am not really a V- 9 regulator and I havc worked in a licensing department but-10 -not as a licensing engineer.
11 But let's take the obvious approach here and say a 12 regulation exists for the purpose of protecting the public 13 health and safety and NRC now finds itself in a 14- risk-informed regulat.ory philosophy,.so we are interested in L/h-i _f s 15 how risk is related to regulhtion in the context of human 16 . reliability.
.17 The first question, which is one that-I haven't 18 seen answered yet, is, is there really a problem with the 19 existing level of human reliability.
20 What we have -- I think Ali talked about it 21 before -- is PRA results indicate that human action is a 22 dominant contributor to risk and most of those numbers are 23 driven by a very limited set of actual data. Normally in 24 the post-trip world there were some simulator exercises 25 conducted by EPRI a long time ago that's the basis for O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
281 1 virtually all of that type of quantification. h 2 You have some data pre-trip and a lot of expert 3 opinion. o 4 So my question is more fundamental -- is there any 5 sort of evidence that can be produced that says you actually 6 have a problem with human performance in your power plants 7 now? 8 Second of all is how can you explain influencing 9 factors, and I put in the vord " quantitatively" -- getting 10 back to the risk. 11 I have always had a funny concern. Whenever I 12 speak to PRA or HRA analysts in the PRAs I've done and they 13 are off doing their thing and they are telling me, gee, 14 Marty, you know this plant is riskv because I don't like the 15 way they are trained -- and I go, well, suppose we change 16 the training program. Tell me how that probability of error 17 changes. It can't be done. It can be done subjectively. 18 But this is the model that Ali is talking about, 19 okay? So we can identify influencing factors. We don't 20 really know how they affect the rick equation per se. 21 That is the basic of the last one -- how can you 22 actually predict the reliability here? 23 CHAIRMAN APOSTOLAKIS: But you seem to believe 24 with your first question there that maybe the problem is 25 exaggerated -- is there really a problem? l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
?>
282
- 1 DR. STUTZKE: I have said that before. I think it
() 2 is. Well, you realize that in the current view of risk only 3 the operator is considered. We haven't really incorporated 4 the M and O sorts of-activities into a PRA. 5 -CHAIRMAN APOSTOLAKIS: So when you say human 6 reliability you are referring-to the operator? 7 DR. STUTZKE: Specifically operator reliability. 8- CHAIRMAN APOSTOLAKIS: Because, you know,-I would 9 say the presentation b; Mr. Trager earlier answers that 10 question partly. 11 DR. STUTZKE: Right. 12- CHAIRMAN APOSTOLAKIS: Humans were the major 13 contributor to accidents. 14 DR. STUTZKE: .Well, I'll play the devil's advocate
) 15 and throw the other slide up here ailittle bit. Let's see 16 if I can lead.into this.
17 As a prelude to this, I will note that a lot of 18 HRA efforts are focused-on the development.of models, 19 -frameworks, the real fun, esoteric mathematical stuff that 20 we all like to play with, not the actual collection, 21 measurement, statistical crunching of-data. 22 I state that because most of my work in PRA in the 23 past has been as a component reliability data analyst. I 24 read lots of_ maintenance records like that, and it is not 25 very pleasant work. It is difficult work. s ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
283 1 My question is, there seems to be a lot of data h 2 the NRC hau collected over the years, AEOD in particular, 3 LERs, whatever, yet I see a human performance database with 4 only 198 events in it and I wonder. This seems to be 5 inadequate. Nobody wants to do the hard work here. 6 MR. CARROLL: Well, I think the problem in that 7 regard is what I mentioned this morning. Nobody has really 8 taken the bull by the horns and straightened out the 9 reporting regulations so that people do put into LERs the 10 involvement of humans. 11 DR. STUTZKE: Right, and you know, the problem is 12 if there were truly an issue with human reliability then it 13 would be captured in the LERs. 14 How can you claim there is a problem when you 15 can't prove it is my point. Okay? 16 In other words, if there is a problem the 17 reporting requirements should have already changed. In 18 order to be sensitive to this, the inspection process would 19 have been changed. 20 MR. CARROLL: I don't think so. These guys, you 21 know, are jusc terrified by 50.109, the backfit rule. 22 DR. STUTZKE: Maybe it is. Maybe it is. Okay. 23 Anyway -- 24 MR. CARROLL: That really drives a lot of things 25 around here. h ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
284
-1 DR. STUTZKE: Anyway,-I would encourage NRC to-() 2 really go after data collection sort of e ing. I don't 3- believe it is ever possible to get a good statistical, sound 4 database to use those sorts of techniques to predict or i
'S estimate probabilities of human error, but; surely we can.do 6' a lot better than what we have now.
7 I will throw in, you know, you have got the actual 8' events going on. There's a lot of~ simulator training that 9 goes on in this country and people make mistakes there and 10- those are all recorded or they should be recorded > 11 someplace -- another wealth of information.
-12 To follow up_on this, I realized today-most of the 13 data is collected by one branch,-AEOD, but it is needed by 14 Research, and that interface doesn't seem to be happening 15 ve-y effectively at times.
16 It seems like we= reinvent the wheel a lot. 17 By the way, I do believe'it is possible to do some 18 sort of summary stat 3stics. We may not ell believe that is 19- the right number but we can get very close. 20- Okay. One of my pet peeves, and we talked about 21 this a-little bit before, is every HRA method I've ever seen 22 re-analyzes TMI and tells me all the= sins-that happened 23 there. Why? Because it is so well-documented that it is 24 easy to analyze. It has a wealth of information like this. 25 The problem is that as a result of TMI numerous ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
285 1- modifications have been mandated and actually-implemented, a () i 2 few-of.them -- the symptom-oriented operating procedures, 3 new training, the-STA, the technical support center, so 4- forth and so on like that, but I have never really seen any 5 sort of~a study that says as the result of all of this, core 6 damage risk has gone down, and I can look numerically at the 7 results of WASH-1400, compare them to the results of the IPE 8 and.they are virtually all the same. Nothing has changed. 9 CHAIRMAN APOSTOLAKIS: Well, that-comes back to 10 the comments that you and Ali made earlier, that how you 11 connect these.PSFs,'the performance, is something that is up t 12 in the air. 13 HDR . POWERS: I don't think -- 14 -CHAIRMAN APOSTOLAKIS: "on't you think? 15 DR. POWERS: I don' .nink comparing-the numbers
- in WASH-1400 to a current PRA is quite fair.
17 DR. STUTZKE: It's not. 18 DR. POWERS: I think ifLyou asked for a PRA to 19~ look-before and'after the TMI improvements that they would 20 in fact identify some substantial improvements that came out 21- of that effort. 22 We know absolutely that it would also identify 23 some that have proved to have less risk worth than others.
'24 DR. STUTZKE: Yes.
25 DR. POWERS: And I think the agency has an effort O ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I' Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
286 1 underway now to try to purge some of those less worthy r'3 -2 i j changes as opportunities afford themselves.
-3 DR. STUTZKE: Right.
4 DR. POWERS: So to say there's been no change is 5 to take an overly strict -- just comparing the two numbers 6 and not the methods behind them -- which I think is a flaw 7 when you look at PRAs. 8 DR. STUTZKE: Yes, I would tend to agree. 9 MR. CARROLL: One place which we feel is lacking, 10 you can find some indication of this, is through the ASP 11 program. 12 That certainly seems to indicate that things have
-13.- gotten better since TMI.
14 DR. STUTZKE: Right. O(s,/ 15 DR. POWERS: Yes, I think they show some 16 interesting plots that show that in the five years following 17 TMI that there's a fairly dramatic drop in the number of the 18 measures that they track. 19 DR. STUTZKE: Well, the point behind mentioning 20 thece, the original reason why I wrote them down in fact, 21 was maybe you have reached the point of diminishing returns. 22 If ATHEANA goes through and says, gee, we do have 23 a problem with error-forcing contexts and whatever, what 24 additional things can you do to improve the safety of the 25 power plant? -- you have already done all of these, how can 1% ANN RILEY & ASSOCIATES, LTD. ('} Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
- - -- ~ -
- . - l 287 1 you make it better? Maybe you just have to live with that
() '2 : risk. 3 DR. POWERS: --And I-think the way that people have 4 looked at and characterized the post-TMI activities-is they-5- had a certain blunderbuss quality to them, that in fact, to-6 quote some people, the opportunities afforded by-the TMI
- 7. ' action plan was for everybody to open_up their desk and pull 8- out all those things they wanted to impose on the licensees 9 in the past and couldn't find a justification for it.
10 Maybe one of the virtues of romething like -- I 11 don't know what'you call your code =-- IDA? 12 MR. MOSLEH: IDA. 13 DR. POWERS: Or something like that is a 14 . sharp-shooter approach, rather than-the blunderbuss 15 approach, that, yes, we have done many things --'many of 16 them we know are not very good ---and some of '. hem we know 17 have been very useful. Maybe there=is another thing that 18 could be done that-replaces both of those and doe ~ something 19 much, much better. 20 CHAIRMAN APOSTOLAKIS: That would be the third 4 21 generation, right? 22 DR. POWERS: I mean I think the second generation ^ 23 :has -- perhaps it is demonstrated, I don't know -- but has 24 the potential of addressing that kind of a problem. 25- DR, STUTZKE: Okay. The mosaic of HRA research -- () IJW RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 1
288_ l 1 -I-used the word " mosaic" mainly because I cleverly embedded 2 'my company's acronym in there -- 3 (Laughter.] i, 4- DR. STUTZKE: But it was stated earlier today t'd. _ 5 I-knew when.I made up the slide that the history of ATHEANA
-6 -was-shut down,. low power risk, and then it rapidly jumped to 7 errors-of commission. The solution to that technique is now -
8 . error-forcing context. 9 ~I see it being used to analyze premature 10 termination and safety system bypae'3 events. 11 I am not certain where - the - focus :i s here, you know? It seems like the research goes where people want it L 13' _to-go and not where-it-is needed necessarily. 14 We debated this this morning a lot, _ w hich leads to 15 the second bullet -- some sort of an integrated plan is 16 needed. 17 I came here-today expecting to see the plan. What. 18- I saw was a plan to make a-plan. 19 DR.-POWERS: You are not alone in your 20 -expectations. 21 .MR. CARROLL: Or if you had been here for three-or
~ 22; _four meetings prior to-this you would'have --
23- DR. STUTZKE: Yes. I had the privilege of reading 3 E 24 the last meeting minutes, but I don't know where NRC thinks 25 it is going, probably because-they don't know where they are ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l 1
289 1 really going. /~5\ (x_) 2 I guest "HEANA maybe could be adapted to handled 3 all of these things perhaps. Perhaps not. So, you know, 4 maybe we need some really let's get back and look at the 5 data and try to tell us well, what sorts of human-related 6 events actually occur in powerplants. We have statistical 7 evidence that they occur. AEOD has that in spades, 8 basically. I mean, you know, the M&O stuff. 9 CHAIRMAN APOSTOLAKIS: But I think, though, that 10 when it comes to severe accidents you really have to rely on 11 hypothetical scenarios, don't you think? I mean -- 12 DR. STUTZKE: Oh, not scenarios. Definitely those 13 have to be hypothetical. 14 CHAIRMAN APOSTOLAKIS: And human actions. I mean, O
\sl m 15 the guy has a medium LOCA in his hands and things are 16 happening, surely you don't expect the HRA research at that 17 level to be driven by observed behavior.
18 DR. STUTZKE: No , I'm not saying it's ever 2.9 possible to quantify it. What I'm back is rather than 20 postulating all the
- rays an operator could make a mistake, 21 let's see what scrts of mistakes they commonly make, and how 22 those relate to the core damage accidents we're interested 23 in.
24 DR. POWERS: I guess the problem you run into that 25 is one of analogy that maybe you're driving at, George, is ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washingt.on, D.C. 20005 (202) 842-0034 ll
290 1 that the common mistakes are things that are going to get f 2 addressed, and they're going to get addressed by multiple 3 arms of the Agency. They're going to get addressed by the 4 inspection teams. They're going to -- and addressed by 5 enforcement teams. They're going to get addressed by NRR. 6 They're going tc get addressed by the regulations. The 7 question that may be addressed is do they go far enough to 8 handle those low-probability events that happen fairly 9 rarely, like someone trying to bring a disconnected core up 10 from power after it's xenon-poisoned itself. 11 DR. STUTZKE: Well maybe what I'm interested in 12 here is the precursors, the sequence leading up to the 13 really bad accident, and we always have these sorts of near 14 misses, ASP looks at them routinely, and there ought to be 15 some sort of indication that then would let one extrapolate 16 or project to the really severe situation like that. 17 DR. POWERS: And in that context I think they do. 18 I mean, I think that's how we look at a lot of these ASP 19 analyses is that they do portend things -- I mean, the way 20 they do them is they say what if one additional thing had 21 gone wrong? And what are the consequences of that? I think 22 that's what they're doing. 23 DR. STUTZKE: At the rick of being an OCI problem 24 again, I think ASP does a very good job at looking at things 25 like that. It's very useful. In fact, I'd like to see more ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Stite 300 Washington, D.C. 20005 (202) 842-0034
291 1 ASP stuff done in HRA. It needs to have a bigger model, [J\ 2 CHAIRMAN APOSTOLAKIS: Is your basic thesis here 3 that a lot of the or most of the HRA research has not 4 utilized what we have observed to the extent that would be 5 appropriate? 6 DR. STUTZKE: What I was thinking of when I wrote 7 the point was, you know, I was reviewing the ATHEANA
.8 research that had been done and decided that the examples 9 they gave of error-forcing context were instrumentation 10 failures. And I remember being told by virtually HRA 11 analyst in my company don't model instrumentation failures; 12 they're not important. The problem is in the diagnosis, the 13 mental process in the operator's mind, not that he could 14 ever be misled. Now this was well before ATHEANA was ever O(_/ 15 started.
16 CHAIRMAN APOSTOLAKIS: But isn't instrumentation 17 instrumental in this diagnosis? And if you get the wrong -- 18 DR. STUTZKE: That's what I was asking. I've 19 realized at the time I was a very young, naive fault-tree 20 analyst, you know, and I would have modeled every resistor 21 if I'd been told to model it. 22 MR. MOSLEH: There is this category of events, if 23 I may add something here, that the plant is telling the 24 operator exactly what is happening and they have this 25 picture of a recent event, and they match that situation to i
\# ANN RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 I
292 i 1 the event-that they had, and they just make mistakes. It's () 2 3 a cognitive filter. CHAIRMAN APOSTOLAKIS: That's the availability 4- heuristic -- 5 MR. MOSLEH: Right. 6 CHAIRMAN APOSTOLAKIS: Have identified now. 7 DR. STUTZKE: Right. 8 CHAIRMAN APOSTOLAKIS: It's been what, 25 years,
- 9. the famous paper from Science with the four heuristics.
10 MR MOSLEH: Yes.
'll CHAIRMAN APOSTOLAKIS: That's what you're --
12 MR. MOP.EH: Yes.- Yes. 13 CHAIRMAN APOSTOLAKIS: You tend to correlate 14 something that you see now with the most available event in C/ 15 your mind. 16 MR. MOSLEH: Right. Right, ,
,17 CHAIRMAN APOSTOLAKIS: And usually that's the most 18 recent one.
19 MR. MOSLEH: That's Jim-Reason's frequency 20 gambling concept analysis. 21 CHAIRMAN APOSTOLAKIS: Okay. Let's go on. 22 DR. STUTZKE: So. Last slide. At the risk of I 23 ~ guess reiterating again, the necessity for peer review, 24 validation, things like that, it's just absolutely crucial 25 in this field. I'm a little concerned by tae wealth of O- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
293
- 1. papers that have~been presented on techniques such as
() 2 .ATHEANA when ATHEANA in fact has not been validated and
~3 ready for use, whatever. It implies something that doesn't.
4 -exist, and that concerns me, because I have to deal with
-5 this in my business overseas where everything that has an 6- NRC sticker on-it is the gospel truth, period.
7 DR.~ POWERS: Let me ask you a question about thin. 8 We have had frequent opportunities to hear from academics 9 and in particular panels from the Academy of Sciences level-10 harsh criticism at the NRC for failure to publish in the 11 open literature. That they say there is no substitute for 12 that. That these peer review panels that the NRC assembles 113 -just don't compare to the kind of peer review you get when 14 you publish in journals and you attend conferences. In
) 15 . fact, fairly harsh criticism for NRC's failure to 16 " participate-broadly in. conferences.. You seem to be 17 orthogonal to that.
181 DR. STUTZKE: .That's cor*ect, and I guess to rebut 19 . what you were saying that my conference experience is much 20 greater than my published journal experience, but what I can
.21 certainly say as the technical chair of the-PSA-96 221 conference -- '93,-whenever it was, down in Florida -- those 23- sorts of papers get very little review. In fact, a lot of 24 ' papers are revided. Now as far as journal articles I 25 personally, Don, I rarely get comments on them.
() ANN RILEY & ASSOCIATES, LTD. Court Raporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
294 1 DR. POWERS- I think that what the academics that () 2 have appeared before us and the NAS have suggested is it is 3 not the written comments that you get when you attend these 4 conferences, it is rather the exchange of comments -- what 5 goes on at the coffee breaks. I think that's what they were 6 driving at. 7 DR. STUTZKE: Oh, sure. Sure, d DR. POWERS: Let me be clear, so that you're not 9 too defensive. I agree with you 100 percent. I think the 10 NRC's peer review process is so much better than this 11 conference and journal article, but I am in the minority on 12 that. I'm in the minority on this committee I think on 13 that. 14 CHAIRMAN APOSTOLAKIS: Let me give you my opinion I% \, ,I 15 on this. First of all, I don't think it's an either/or 16 situation. I don't think that conference papers and journal 17 articles can replace serious peer review, right? You need 18 both in my opinion. 19 DR. STUTZKE: Sure. Sure. 20 CHAIRMAN APOSTOLAKIS: Peer review is definitely 21 very important. I have been on peer review panels and I 22 mean you complain, Marty, that conference papers don't get 23 enough review. Well, excuse me, but I have been on peer 24 reviews that where the review was really very superficial, 25 for various reasons. t
\-
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 , 1 l
295 1 DR. STUTZKE: Oh , sure. Sure.
/^T l 2 CHAIRMAN APOSTOLAKIS: And the higher the level of j
3 the peers, the mere superficial the review. 4 DR. STUTZKE: Um-hum. 5 CHAIRMAN APOSTOLAKIS: Okay? 6 DR. STUTZKS: Well, what was motivating this? 7 CHAIRMAN APOSTOLAKIS: And I think, you know, I'm 8 editing a journal, and I know what -- I think there is -- 9 what you get from that is, you know, you get anonymous 10 reviews, it depends who the referees are, that may point 11 out, you know, some of the things that you hadn't thought. 12 I always learn from those things, by the way. I don't view 13 them as negative. I mean, I always learn something from ' 14 those. O) (_ - 15 DR. STUTZKE: Sure. Sure. 16 CHAIRMAN APOSTOLAKIS: And the question somebody 17 asks at the conference. But one of the problems I think 18 with the way our industry is right now, and I see that a lot 19 in the papers that I receive, is that people don't have the 20 time or the resources or whatever, they're not used to it, 21 to go out, _and before they do something themselves to review V 22 what's out there, and what-you see is that they tend to cite 23 people from their own organization or things that come 24 easily to mind, you know, and I think you broaden your 25 perspective by submitting it to a journal and getting, you
,[ ANN RILEY & ASSOCIATES, LTD.
's Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
- _ . _ _ _ - - - _ )
296
-1 know, reviews _and so on from other people.
() 2 But invariably the comment is, you know, that 3 there has been work here, here, and there, and you guys 4 -don't seem to be aware of it. So that contributes too. But 5 it is no' substitute for peer review; serious peer review.- 6 DR. STUTZKE: That's true. 7 CHAIRMAN APOSTOLAKIS: So I think you need both, 8 -- and I think the academy was complaining about that because-9 there is some'value to that process. And it's true that a 10 lot of the articles-are not contributing much. I mean, 11- we're not going to kid ourselves. 12- DR. POWERS: In support of your point of view on 13- their -- the literature work I'm absolutely convinced that 14 many of my-colleagues at a major national laboratory don't () 15 know where the library-is, let alone what it contains. 16 CHAIRMAN APOSTOLAKIS: And, you know,.that's a 17 -- fact of life, and I think by forcing people to-go to i 18- conferences and submitting-journal articles, maybe that's a 19 way of expanding-their views and their-horizons. I frankly,-- 20 I mean when I present a paper.at a conference, there is 21 always somebody who asks a question, say gee, you know, I 22 never thought of that. You really learn. You really learn. 23 And I see it as-a positive'-- because that guy, for him or 24 -her what I'm presenting is new, and they see'it in a 25' different light. And I think if you work on something for a ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 l
297 1- while you-tend to take a. tunnel vision, right? 'So I think ( -2 there is room for both, and I -- 3 DR. STUTZKE: -I don't discourage either one of 4 them. 5 CHAIRMAN APOSTOLAKIS: Okay. Now, ase-you done 4 6 with your rebuttal comments? 7- DR. STUTZKE: Well, let me just comment why the e 13 last-bullet you've gat written there.- I'll tell you a small 9 story.- When the second ATHEANA document came out', it 10 i references -- well, I was very curious. I was having
-11 : trouble understanding what they meant by a lot of the terms, 12 and they quoted some data bases and stuff that=they had-13 built,_and I thought I'll get a copy of this data base.and-14 lxa able to look up the actual event and then things will
() 15 'become apparent to me. So the data base-was actually cited
-16 as a Brookhaven National Laboratory document. I had my l 17- . librarian call Brookhaven.- They didn't'know whereuit was, 18 - butlthey found the program manager. He didn't have a1 copy.
19 He referred me back to SAIC. 20 [ Laughter.) 21 No further comment to make on that one.
'22 CHAIRMAN APOS't.0LAKIS : Okay.
23' DR. STUTZKE: But it does make it difficult to 24 -review. 25 CRAIRMAN APOSTOLAKIS: I'm really anxious to move ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W.,- Suite 300 Washington, D.C. 20005 (202) 842-0034
5 298 1 on to a discussion of what we heard today. So we can () 2 continue the discussion or take a five-minute break if you 3 want. Wnat does the Committee need? 4 I don't want -- not to be an afterthought, I want 5 us to really go around the table and see whether we can -- \ 6 well, you have some questions, in fact. I mean we -- Noel, 7 we are going to have the staff make a presentation to the 8 Full Committee. 9 MR. DUDLEY: That's right. 10 CHAIRMAN APOSTOLARIS: At the February meeting. 11 We have to decide what we would like them to present. . } 12 Because, obviously, you know, they can not presenu the whole 13 thing. But, also, I would like to get some feedback from 14 ou-. experts here on what we actually heard that goes beyond I- 15 the srepared comments, and also, of course, from the 16 members. 17 So, do you want to do it right away? Take a 18 break? 19 DR. POWERS: I'm warm, let's keep going. 20 CHAIRM N APOSTOLAKIS: Okay. So, how do you want 21 to proceed, do you want to start from the right or left? 22 DR. POWERS: I'm perfectly wiling to start. 23 CHAIRMAN APOSTOLAKIS: Then we are starting with 24 you .. 25 DR. POWERS: Okay. My views on the Human l ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
299 1 Performance Program Planning changed little, although -- () 2 DR. MILLER: Did you say changed a little or 3 . change -- 14 DR. POWERS: Changed little, Over the roughly one a 5 year that we have been confronting this. effort. And, in. 6 that context, I think there is no merit in having a 7 presentation before the Full Committee. I saw nothing new 8 here that the Full Committee needs. 9 It certainly appears that human actions are 10 -important in the safety in nuclear power. I have, however, 11 not seen the quantification of the evidence that there need 12 : to be a -- an NRC mission need to do more, or to do 13 differently about human performance in nuclear _ power plants. 14 To the contrary,-I have seen that those monitoring-15 inspecting power plant licensees have all the tools they. 16 need. The most pressing need they express is the need for 17 an opportunity to digest and learn from the tools that-they 18 tave. 19 -CHAIRMAN APOSTOLAKIS: I'm sorry. Who'is they? 20 DR. POWERS: This would be those doing the 21- inspections, NRR, AEOD. 22 I did appreciate the comments from experts, 23 invited experts that perhaps it is our uncertainty in the 24 quantification in human performance that moves us to 25 continue to research this field. That seems as viable as a i ( ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
300 g 1 need,-a' demonstrated.need that there is a problem-in human () 2 3-performance. And-I would like to see some quantification of that'to justify some'of the work. 4 I ami also impressed.by the apparent consensus in ; 5 our invited experts that PRAs currently may over-estimate-6 the impact of human performance. That was stunning to me, 7 especially in light of the drumbeat of criticism that is 8 ' leveled at PPA every time the word gets mentioned, that it 9, is unable to handle errors of commission. 10 I continue to see no evidence that mission needs 11 have been identified that mandate additional actions or 12_ research in human performance, I do not see a vision of 13 what state of technology with regard to human performance 14 the NRC wants for.the future. I think that is essentially
'15 any time'you are setting up a program plan, to be able to 16 .tell me where you are -- where-you are going to, especially-17 -in an era when the research that-is available are probably 18 not going to match the aspirations for any -- and so in any 19' igiven year, you can only take a step in-the direction you 20 want to go. You can't take -- get all the way-there.
- - 21 I am not going to comment about ATHEANA because of r 22- a conflict of interest about something that I don't really 23 understand right now.
24 But I do want to emphasize that I think anything, 25 any research that can be dene that-will make PRA immune to ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
301 1 what I:think are gratuitous criticisms about what it can't () -2 -do,- that are not balanced by what the alternatives are, is a 3 very good _ thing, and a good use of NRC resources, research 4 resources to render PRA more robust and to give those that a' 5- may be suspect of PRA greater confidence in its use as a 6 basis to guide changes that we make in regulations in the 7 ; future. 8 I continue to see, by what crosses _my desk, that 9 NMSS has numerous recurring human error events-that do have 10 demonstrable human consequences. I still see NMSS as a weak i 11 _and perhaps even silent partner in the development of a 12 Human Performance Program _ Plan. In this regard, I see no 13 effort underway to ascertain if regulations and their
;14 implementation are error-forcing functions for.NMSS.
15 And that -- those are my comments, George. 16 CHAIRMAN APOSTOLAKIS: Thank you very much.
=17 MR. CARROLL: Can I ask Ivan a' question -- Ivan --
18 Ivan,.Jr. a question? 19 DR.._SEALE: They have similar hairlines. 20 MR. CARROLL: Yes,-that is do. That is about -- ! 21 DR. POWERS: I have noted-before this Committee 22 that God created a few perfect heads and the rest of them he 23 covered with hair. 24 (Laughter. ] 25 MR. CARRCLL: I guess, Dana, the one thing -- I () ANN RILEY & ASSOCIATES, LTD. Court. Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 \ . ... .
302 1 -generally agree with everything you'said, except -- () 2 -DR. POWERS: The beginning and the end. j 3 [ Laughter.-] 4 DR. MILLER: And all that stuff in the middle. 5 DR. KRESS: And the stuff in between. 6 MR._ CARROLL: Except I did not get the same-f7 impression from the users of these tools and prcducts that a_ Lthey had all they wanted. I think -- I-got the-sense that-19E at least they -- they. felt they;needed more to help them 10 evaluate management and-organization issues.
-11 DR. POWERS: I think that you are probably right 12 there, Jay.- I probably have a deaf ear when the;words
-13 management-and organization come up, because I -- I tend to 14- 'be-suspicious that that is a field where it is difficult for
-15 us to make_ progress.
16- But let me put a caveat on my -- the other part of-
~
.17- my statement. -The_part that you agreed with. Maybe you 18, agreed with. I did hear exactly what you heard, was that, 19 yes, we have everything.weEneed, including -- we need to 20- write a. user's need request. I think~I am becoming very
.21 appreciative =of the difficulty of design:a-research program
!22 .then people who_are your customers going around saying I don't need research, except I need this -- this user need 241 that I will let you know about in the future. I mean --
25 MR. CARROLL: Right. ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 w
303 1 DR. POWERS: -- that must be very frustrating. l () 2
- I guess what we need to do is see, as the plan 3 develapa,-is some articulation _of what those needs really 4 are. _And I put a lot of that in what I call. vision. You 5 know, where do we want to be in the future?
6- MR. CARROLL: Yeah. 7 DR. POWERS: Als. I think where the agency wants to 8_ be in the future is a less manpower-intensive organization. 9- And a lot--of the things that we saw the using organizations 10 doing in Human Performance and Human Reliability are very 11 manpower-intensive, and I think we want to get away.from c 12 that. 13 So when Mark puts up a viewgraph that says the-14 future we would like to see this become ATHEANA, or whatever 15- tool they use,-maybe it is -- maybe it is University of
.16 Maryland's tool, or whatever,- it becomes a computer code, I 17 think that is moving'in that direction that the agency wants
-18 to go to take---.their big cost is manpower. Their-19 available resources are declining, so you have got to find 20 mechanisms to get your manpower cost of doing some of.these 21 things down, and_that means automating:them, and it is a +
22 capital-labor trade-off here. 23 CHAIRMAN APOSTOLAKIS: Bob. 24 'DR. SEALE: Well, I guess my comments are-somewhat 25 contrary to Dana's in some respact. () ANN RILEY & ASSOCIATES, LTD. Court Reportera 1250 I Street, N.W., Suite 300 Washil.gton, D.C. 20005 (202) 842-0034
y 304 1 DR. POWERS: That's why there is more than one of () 2 us-on this Committee. 3 DR. SEALE: I agree. I agree. 4 MR.. CARROLL: But.11, 5 (Laughter.) 6- DR. SEALE: -I was rather impressed.with-what I saw 7 in the attempts to use ATHEANA that were done at Brookhaven. 8_ I thought that.was the first time I had seen some -- 9 CHAIRMAN APOSTOLAKIS: Seabrook. 10 DR. SEALE: The -- with the-simulator, yeah. I'm
'11 sorry. It struck me that that was -- at Seabrook, yes.
12 That was really the first time someone had really 13; demonstrated that this kind of rational -- or rationale, I 14 should say, would -- how it,would fit in with the other 11 5 things that are being done, and really get you off of -- 1
-16 perhaps off of dead center on some of this, these things.
17 So I think'if you are going:to have something for 18 the Committee -- whether or not-the large Committee should 19- anything, the Full Committee, I think something on the 20 ATHEANA application or the trial with ATHEANA at Seabrook 21 would be worthwhile. 22 CHAIRMAN APOST ;AKIS: I think that is related 23' though and maybe Dana also should give us his opinion on 24 this, whether -- to whether we want to write a letter. 25 Becauce, otherwise, I mean to just to come to -- for z ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
. - _ ____ --- = _ .
p, 3v5 1 information purposes. () 2 DR. POWERS: And, also, it struck me is it was 3 premature to hear about-it.- They are-huddled off in some 4 desert town right now, devoid of any hutaan comforts, writing 5 up a document.- 6 CHAIRMAN APOSTOLAKIS: So letter, we are not going 7 to write a letter then on anything. Is that correct? 8 DR. SEALE: I don't know that.we have determined 9 that, but that is probably the case at this point. 10 MR. CARROLL: Yeah, I would say, Rob, it would be 11 better -- I think the Committee ought to hear about it, but 12 .it would be better to wait until they get past this next 13 hurdle. 14 DR. SEALE: Okay. Fine. I guess also I had the ( 15 impression, you know, I am always glad to have an-16 opportunity for an inspector to come out of the woodwork so 17 we can see what they look like and make sure they really are
.18 still speaking dnglish, you know, and we are still 19 commo,-icating with them-and so on. So I would also say that 20 when we do have a presentation, that I think something on 21 the inspection program would be -- would be worthwhile.
22 MR. CARROLL: That was one of the better 23 presenters that I have heard in a long time. Very, very 24 good. 25 DR. SEALE: Yes. Those guys from Inspection do ANN RILEY & ASSOCIATES, LTD. O- Court Reporters 1250 I Street, N.W., Suite 300
, Washington, D.C. 20005 (202) 842-0034 e .. .. . .. . ..
306 1- impress you sometimes, yes. () 2 CHAIRMAN APOSTOLAKIS: Which one are we talking 3 about now? _The tall guy?- 4 DR.- SEALE: Yes. 5 CHAIRMAN APOSTOLAKIS: Meyers. 6 DR. SEALE: Don Meyers. 7 DR. MILLER: Yes. 8 DR; SEALE: But in any case I guess I am not as 9 pessimistic as Dana is on che whole topic, and while I agree L 10 with him on the availability of the resource crunch that we 11 are going to have and so on, I don't think that this is a 12 subject that we can ignore as a full committee in coming to 13 some of our conclusions. 14 I really think that we ought to follow what is , k 15 going on close enough to be able to make some 16 recommendations with a little bit more substantial basis, so 17- I want to hear more. 18 CHAIRMAN APOSTOLAKIS: Want to say anything? 19 MR. DUDLEY: Let me voice my opinion on P letter.
- 20L I think it is important for the committee as a 21 whole to provide feedback to the Staff based on what was 23 heard today and whether this-is the direction that you would 23: like to see the Staff go or provide more additional detailed 24 anggestions or recommendations how to proceed vith their 25- planning process.
l ANN RILEY & ASSOCIATES, LTD. ' O- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
307 l 1 The last letters we have written have been fairly
- () 2 high level commentaries on the plan and we'may be at a point 3 now where the committee could determine what specific 4 _ guidance they would like to-provide-to the Staff and provide r
5
~
that in the form of a letter. 6- MR. CARROLL: I think it could be summarized very 7 simply, get to work and develop a plan we are going to look ) 1 8 at. 9 DR. AILLER: What bothers me about that is that . 10 you met with them last November, and I thought kind of laid 11 out a plan or a planning model. I didn't see much of that 12 reflected today, unless I missed something. 13 CHAIRMAN APOSTOLAKIS: Tom, what happened to that? , 14 MR. KING: It's a resource issue. We didn't spend 15 .a lot of time since last November up until a few weeku ago1 l 16 where we reactivated-the work to look at the plan. 1 17 I mean there's just a lot of things going on that la the Staff'-- thatLis chewing up the Staff's time, and this 19 one suffered, and Tna are behind schedule. ,. 20- We had hoped to be further along now than we are. 21 DR. POWERS: Tom, is it a case -- the problem-10
- 22 there are a lot of pressing-things -- oo you find yrurself i 23 doing things that you personally feel that you would rather 24 devote those resources to the human performance program plan 25 or in fact you have in your own personal priorities -- if ;
I
. ANN-RILEY & ASSOCIATES, LTD.
Court Reporters 1250 I Street, N.W., Suite 300 l-Washington, D.C. 20005 (202) 842-0034
308
-- 1 ' this comer out sixth on the list of the five you can do?
l 2 It keeps getting dumped off the back of the sled. 3 -- MR. KING: Now this plan is not an issue of
- 4 contractor resources. This has to be done by'the Staff.
5 DR. POWERS: Suru. } 6 MR. KING: And the Staff has been working on other + 7 things like core capabilities, these DSIs that have come : 8 down, all that hawe due dates that are tracked by the D Commission in a very -- the Staff is told let's try and make ! 10- those dates. You know, they are given high priority and j 11 .other things slip. 12 Are there other thinge I would rather work on than 13 DSIs or . ore capabilities? Yes. I would, but, you know, I , - 14 don't set all my priorities. 15 DR. POWERS: The reason I asked the question is 16 lthe committee could certainly write -- I could imagine the 1 17 committee writing a letter to the Commission saying get off 18 their back and let them get on with the plan. 19 CHAIRMAN APOSTOLAKIS: Yes. I mean we did write 20 - something to.the effect in our last letter that you should 21 drop other activities, you know, and focus on this one. 22 On the other hand, I realize.what you are saying. 23 I appreciate what you are saying. When the Commission asks 24 you to do something, you have to do it. 25 We are meeting with the Chairman I believe -- ANN RILEY & ASSOCIATES, LTD. a Court Reporters
-1250 I Street, N.W., Suite 300 .
Washington, D.C. 20005-(202) 842-0034 I'
%y , .,>-,--.t-- -. g eg . r e . n,,-ww,w~,,, - - . . , - , . ,,,o .,.y ,-- ,,w, . ..,,,...,-,rn., . . . . - ,4%,w ,...--a,--m..,a..-, u , -- , , - - - . - . . ..--w
l 309 1 DR. SEALE: Yes. 2 CHAIR!RN APOSTOLAKIS: -- in February, is it, or 3 March? Maybe we can raise the issue then without writing a 4 letter. 5 MR. CARROLL: She certainly pays lip service to 6 organizational factors and that sort of thing in her 7 speeches but -- 8 CHAIRMAN APOSTOLAKIS: Well, it is not just 9 organizational factors. I mean we shoult' sensitize her to 10 the fact that we need thic plan as en agency. 11 DR. KRESS: I certainly would be reluctant to do 12 this antil I see these other priority items and until I hear 13 something from her that says which one of these would you 14 prefer that they not do first? () 1S I can't make that kind of judgment without seeing 16 what all Tcm is tied up with, and it sounds like he's got 17 some thing that need to be done. 18 CHAIRMAN APOSTOLAKIS: How about if we put it this 19 way. We are not going to compata tha human performance plan in terms of its-importance with the other things, because we 20 21 'are not aware -- 22 DR, KRESS: Certainly. That was my point. 23 CHAIRMAN APOSTCLAKIS: We can certainly say that 24 this la a very important activity and we are told by the 25 Staff that progress has been inhibited by the fact that they
-I \
ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
310 1 have to do other things and leave it at that. () 2 DR. KRESS: SJ,-you know, that's planning. You 3 _- are.asking the Commission to get off their back because 4_ they've got too much -- too many requirements handed down-5 and they can't do this other thing-because of it. 6 CHAIRMAN APOSTOLAKIS: She may not even be aware 7 of it. She may not know how we feal about it. 8 I mean if she hears us and then she says fine, and 9 then Tom comes back and says in June we still have other 10- things to do, then the message is very clear. 11 DR. KRESS: Well, you know what I would do. I 12 would go to Tom King and say what the hell you tellin' those 13 guys? I'd beat on him and -- 14 CHAIRMAN APOSTOLAKIS: I have no problem s 15 mentioning to the Chairman'-- - 16 DR.'KRESS: I don't want to get Tom in trouble 17 with something that is not going do much good anyway. 18- CHAIRMAN APOSTOLAKIS: I don't know that it 19_ wouldn'c do much good. R2 0 ' Wouldn't you like if-you were a. Commissioner to 21 know that.these things are happening? Do you assume that 22 she knows?- 23 .DR. KRESS: I think just tell them we'think that 24 human performance is a very important thing to do. i
~
i - 25 DR. MILLER: I-agree with you. ANN RILEY & ASSOCIATES, LTD.
\w- Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
311 1 DR. KRESS: Stop there. () 2 3 priority. DR, MILLER: Just tell her we believe it is a high 4 DR. KRESS: Yes. Stop there. 5 It's up to her to set priorities. 6 CHAIkMAN APOSTOLAKIS: And then what? Are you 7 happy with the progress? No, I'm not. Then I am blaming 8 him. But if I say that, you know, we are told that progress 9 has been very slow because there have been other demands on 10 the Staff's time and we believe that this issue is very
- 11 importa;.t, it should take top priority, and leave it at 12 that.
13 DR. XRESS: Once again, I'm not -- 14 CHAIRMAN APOSTOLAKIS: It's high priority 15 DR. POWERS: He doesn't know that either. 16 DR. KRESS: I think it's important and needs to be 17 worked on. I don't know what priority to give it till I do 18 a lot more -- 19 MR. CARROLL: Leave priorit?' out of it. 20 DR. SEALE: Just indicate that we haven't seen 21 anything yet -- 22 CHAIRMAN APOSTOLAKIS: But I don't want to blame 23 Tom. You know, the guy's not given the opportunity to work
-24 on this.
25 See, it's do I blame the operator or the context [/l \- ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 i
312 1 within which the operator functions? 1 () 2
-3 (Laughter.)
CHAIRMAN APOSTOLAKIS: We have been tqlking about 4 the context all day here -- i 5 DR. SEALE: Well, it's a management factor. We 6 .can't come to grips with this. - l- , 7 CHAIRMAN APOSTOLAKIS: Well, after all, we don't 1 8 have to speak as a committee to her. Right?- 9 DR FRESS: We don't. That's right. 10 MR._ CARROLL: Why is John smiling at all this? 11 Does he know something we should know? 12 CHAIRMAN APOSTOLAKIS: Speak into the microphone. 13 You must be_new to_this ploce.
?
14 DR. LARKINS: I was just thinking of the priority 15- issues that I have received from_menbers in the past that 16 didn't come out as being very high on the list. 17 bR. KRESS:' Didn't show up very high, did it?- 18 CHAIRMAN'APOSTOLAKIS: It's hard for me to believe 19 that studying human factors with everything'that goes with 20 it -- organization, management,-and so on, is not a high ; 21 priority item if I look at the incidents of the last five, 22 ten years. 123 DR. KRESS: I had it high on my list I turned in 24- to John.
-25 I lumped it in with the risk-based regulation.
4 i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W.,-Suite 300 Washington, D.C. 20005 (202)-842-0034-1
313 1 CHAIRMAN APOSTOLAKIS: Well, then it will be one () 2- man's opinion. I don't know. I mean or we keep _ silent -- we 3 don't say anything, but the moment you talk about it, I mean
-4 you have to say something about what is happening and --
5 DR. KRESS: Well, let'sLleave it off the agenda 6 then. 7 CHAIRMAN APOSTOLAKIS: Given the importance of the 8 issue, do you want to leave it off the agenda? 9 MR. CARROLL: Either we believe it is important 10 enough to say something about it, or -- 11 CHAIRMAN APOSTOLAKIS: We don't have to make a 12 comparative evaluation because you're right. We don't know 13 how important other things are, but the least we can do is 14 tell her, look, this is an important issue. We have been 15 working on it for a year now -- 16 DR. KRESS: Ten years. 17 CHAIRMAN APOSTOLAKISi Well, ten years, no, I 18 don't think so. 19 MR. KING:- Let me mention someching. 20 This activity is not tracked on-the Commission 21 tracking syt'.em. 22- They track a lot of things. Those are the things 23 they, at'least in our view, hold up as the higher priority. 24 You will not find this item _on their tracking 25 system. ANN RILEY-& ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
314 CHAIRMAN APOSTOLAKIS: Why not? 2 DR. KRESS: Good question. 3 CHAIRMAN APOSTOLAKIS: Have they made a 4' determination this is not important or it never occurred to 5 them, nobody ever told them that thiu._is really very 6 important and we have to do something about it? 7 MR. KING: We have told them we are working on 8 this plan. We have told them that a long time ago. 9 They were interested it seemed at the time, but it 10 never in their eyes got elevated to a point where they 11 wanted to track it as_something, like they track the 12 risk-informed Reg. Guides. I mean that is right at the top 13 of the Chairman's list -- and there's a bunch of other 14 things on the list too, l 15 You won't find this on the list. 16~ MR. CARROLL: It would be desirable to get it on 17 the list-in the sense that they would at least periodically 18 look.at it. 19 CHAIRMAN APOSTOLAKIS: But also, you can bring it 20 in the context that Dana mentioned. 21 I mean if risk-informed regulation is very high on 22 their list -- 23 DR. KRESS:
~
This is a big part of it. 24 _ CHAIRMAN APOSTOLAKIS: -- it'is creating a problem 25 there because every single time you talk about PRA, ah, ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 L
315 l' there are problems wihh-PRA. What? Errors of commission. 2 (f So,_you know, this-is addressing that. 3 MR. KING:- I am mentioning that from.the 4 standpoint if you wanted-to probe-the Commission a little 5 b't and ask them why isn't it on there tracking list -- 6 DR. SEALE: Is that an error of commission or 7 omission? 8 DR. KRESS: That's probably a real good-9 suggestion. I would go along with that. 10 CHAIRMAN APOSTOLAKIFs Is it a crime of commission 11 or an error of omission? 12 I certainly think we have to mention something. 13 Now maybe we can discuss it among ourselves how we are going 14- to present it, but to keep silent I think is not_a good () 15 . idea. 16 Are you done? Don? 17' DR. MILLER: Well, I think, first of all, Dana 18 kind of captured my view of where we are on the plan. 19 I think the data that was presented by both the 20' inspection and the AEOD_was in many cases rather 21_ enlightening and surprising to me, and I think that data 22 would'be valuable to all the other committee members and-I 23 would say the same, even more so, maybe of the co'nclusions i 24 -made by our' consultant here. 25 ~'m not certain I was totally surprised, but in_a I- ANN RILEY & ASSOCIATES, LTD.
\s Court Reporters-1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
316 1 sense it reinforced some of my views on the importance of () 2 human factors and where it lies and where it doesn't lie. 3 I think particularly with the nebulous management 4 and organizational issues-that were difficult to quantify or 5 even model, that doesn't surprise me that that came out to 6 be a high item in the entire -- that came out high in both 7 AEOD and or course it's somewhat hidden in the inspection 8 reports. L The other thing I would like to recommend in the 10 reports is, and one of the consultants mentioned it -- if 11 the two reports tried to use the same terms so we can 12 compare apples to apples instead of apples to oranges, but 13 otherwise I think the reports were quite good. 14 CHAIrJ1AN APOSTOLAKIS: Tom. C\ ( ,/ 15 DR. KRESS: Well, I think it's premature to 16 present anything to the full committee at this time, 17 including this data. I don't think it's been evaluated, 18 analyzed, looked at. There's no -- we may be drawing policy 19 implications from it, and it's interesting, but I think it's 20 something we want to look at later on. And I don't see 21 anything in the plan that's different enough to present to 22 the full committee, and I can't see the need for a letter at 23 this time, so I think a summary of things, status, from you 24 would serve the purposes of the Committee as-good as _ 25 anything. O' ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 2 84 -b3 l l l
_ - _ . . - -..._. ...__ _ _. _ _.- - - ~.._. _ ._ _ _.._. _ .. _ __.._._.. _ _ l I 317 1 I share a bit of Dana's view about the status of ; () '2- the plan, except I-may 1me a little ore understanding in the 3 sense that the vision statement might just be, we'd like to
-4 be able to do a better-job with human factors and PRAs.
! 5 That might be the-vision. And this is a -- the bottling of 6 human performance in PRAs is in such an embryonic state that 4 7 any first step to me is a good first step, and I think this 8 is a first step. 9 CHAIRMAN APOSTOLAKIS: Well, this is a second
- 10 generation.
11 DR. KRESS: Second' step. And so I certainly think 12 it's important and it needs to be pursued, 2nd should be 13 given more emphasis and more resources. l 14 At this point I still' don't know have I heard () 15- anything at all about-the technical content of ATHEANA, so 1 16: I'm looking forward to this peer review mceting. I think 17l it's after that meeting that we'll be in more of-a position 18 to make better input, i 19 CHAIRMAN APOSTOLAKIS: Good. 20 DR. KRESS: So that's where I'm looking forward to ,
.21 learning a little more.
22 CHAIRMAN APOSTOLAKIS: So I take it you plan to go 23 to that meeting. 24 DR. KRESS: I plan to; yes. 125 DR. MILLER: My remarks on ATHEANA are consistent i ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Guite 300 Washington, D.C. 20005 (202) 842-0034
. _ _ . ~ . . _ . _2; .- _ . . . . _ - - . _ . .- _._ _ _ . . . . . . _ - _ _ _ _ _
318 1 with Tom's. I think it'd be very valuable to have a report () 2 after thuc. I think the paper and the report on what 3 they've done at Seabrook will be quite positive. 4 DR. KRESS: Yes. 5 CHAIRMAN APOSTOLAKIS: Jay? 6 MR. CARROLL: I have nothing to say, other than 7 I'm getting thirsty. 8 CHAIRMAN APOSTOLAKIS: Ali. 9 MR. MOSLEH: I just want to maybe repeat again 10 that the data that we have seen need to be analyzed using g 11 some model, or the models that exist need to be made 12 explicit. It's hard to have correct data without having a 13 model, so I'm sure there is some picture of what the 14 relationships among the elements are that exists someplace, O) (_ 15 but need to be made explicit. 16 And all these need to be consolidated through some 17 coordination among various NRC offices, because there are a 18 lot of, you know, very valuable data, information, and 19 activities that I think would benefit from sharing -- being 20 shared among the different offices. 21 CHAIR'4AN APOSTOLAKIS : That's it? 22 Marty. 23 DR. STUTZKE: I'm impressed by the effort AEOD's 24 putting into things. They're doing a good job. Do more. 25 Share your data with the world and make certain that the ANN RILEY & ASSOCIATES, LTD. C;.tro Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
319 1 Office of Research actually gets ahold of it and makes use () 2 of it. It s always refreshing to hear the inspectors come 3 and speak, and it makes me remember 20 years ago when I used 4 to do those sorts of things. I think we beat up the plan or 5 lack thereof enough, and I'll reserve comment on ATHEAHA 6 until I actually see it. 7 CHAIRMAN APOSTOLAKIS: So let me te.1 you then 8 what I get from you, that we don't need to write a letter, 9 it's premature. We don't need a presentation. And I will 10 brief the Committee on what happened. I don't need more 11 than 15 minutes. 12 MR. CARROLL: Well, you ought to in future 13 activities get some of the things in the pipeline that you 14 want to hear after that takes place. 15 CHAIRMAN APOSTOLAKIS: And we have to discuss what 16 exactly we're going to tell the Chairman. Okay? And with 17 that, we've reached -- does anybody want to say anything 18 from the staff? 19 MR. KING: No. I think we're available to meet 20 with you again on ATHEANA, on the plan, once we get some 21 more meat on the bones, and appreciate your taking time to 22 give us your feedback, 23 CHAIRMAN APOSTOLAKIS: So what we said this 24 morning then you agree with it, we will have a presentation 25 to the subcommittee before you guys finalize it. () ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington,.D.C. 20005 (202) 842-0034
320 1 DR. SEALE: Tom, as I understand it, this peer () -2 review will be primarily addressing things that are 3 contractor-generated. Is that correct? 4 MR. KING: I can't answer that. I have to talk to 5 Mark. 6 CHAIRMAN APOSTOLAKIS: He's here. 7 MR. CUNNINGHAM: The peer review for ATHEANA, the 8 majority of the hours put into it have been contractor 9 hours; yes, 10 DR. SEALE: So the distractions you guys have 11 right now with other things shouldn't seriously impede the 12 availability of that report for peer review sometime in May, 13 as'we've discussed, or at least the distribution in May. 14 MR. CUNNINGHAM: It shouldn't. () 15 DR. SEALE: Okay. 16 MR. CUNNINGHAM: We consider getting that peer e 17 review in the May time frame. Very, very important. 18 CHAIRMAN APOSTOLAKIS: Are you going to pay the 19 peers? 20 MR. CUNNINGHAM: We haven't worked that out yet. 21- We've talked about -- 22 CHAIRMAN APOSTOLAKIS: That's a critical decision. 23 MR. CUNNINGHAM: Yes, it is a critical decision. 24 We kicked that around, and haven't-decided. 25 CHAIRMAN APOSTOLAKIS: If you hit them with 0 ANN RILEY & ASSOCIATES, LTD. Court. Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
321 2 report that has 300 pages and you say please do it for the 2 _ service of the nation -- 3 MR.- CUNNINGHAM: In your spare time, yes. Yes, 4 that's a problem.- 5 DR. SEALE: You just have to find a patriot or 6 something. 7 CHAIRMAN APOSTOLAKIS: Okay. I would'like to , 8 thank again our invited experts who came down here to give 9 us their insights. We appreciate that very much, and maybe 10 we can invite you again if the price is right. And thank 11- the staff for the presentations they-made today and their
- 12 willingness to stay until six-o' clock, and with that, I 13 declare this meeting --
14 DR. KREGS: And we.need to thaLk our ACRS 15 consultant also for contributing. 16 CRAIRMAN APOSTOLAKIS: Oh. 17 DR. KRESS: I mean,-this is a differentiction 18 between a consultant and an expert. , ; 19 CHAIRMAN APOS70LAKIS: -Yes,-yes. A consultant is-20 not an expert. 21 MR. CARROLL: My feelings were hurt when I wasn't 22 an expert. 23 CHAIRMAN APOSTOLAKIS: We'certainly want to help 24 our former colleague and senior member for coming all the way-from California, and might as well thank you guys for
= 2 5 --
() ANN RILEY & ASSOCIATES, LTD. Court Reporters 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034
322 ,, 1 coming'here. Thank you very much. 2 (Whereupon, at 5:50 p.m., the meeting was 3 -concluded.] 4-5 6 7 8 9 10
- 11. --
-12 13 14 l15
-16 17
-18
=19 20 21 22 23-24
--25-- -
ANN RILEY &' ASSOCIATES, LTD. , Court Reportera 1250 I Street, N.W., Suite 300 Washington, D.C. 20005 (202) 842-0034 k . . . . . .. . . - .
REPORTER'S CERTIFICATE l This is to certify that the attached proceedings before the United States Nuclear Regulatory Commission in the matter of: NAME OF PROCEEDING: HUMAN FACTORS SUBCOMMITTEE MEETING DOCKET NUMBER: PLACE OF PROCEEDING: Rockville, Maryland were held as herein appears, and that this is the original transcript thereof for the file of the United States Nuclear , Regulatory Commission taken by me and thereafter reduced to . typewriting by me or under the direction of the court reporting company, and that the transcript is a true and b)( accurate record of the foregoing proceedings. O , i An - wvL on Hundley Official Reporter Ann Riley & Associates, Ltd, O P
.4, -
~ O O O Integrating Human Factors Considerations into The Inspection Process . Stuart Rubin, NRR Glenn Meyer, Region 1 January 21,1997
O O O 1 NRC Strategic Plan i l Substrategy Applicable to inspection Program:
"We will inspect licensees' operations and activities to help ensure that licensees identify and resolve safety issues before J
they affect safe plant operations."
O , Simplified Proda Dirgram fEr O Integrating Human Factors Considerations into the inspection Process a Huma.i Performance g Dt Information f r PFR and -+- @ Event , Compile Reports Routine inspection -> g ,, p' " ,' *_ Monitoring Report l of(Human ) + Human Factors Performance Information Human Factors inspector -> at Plants
- Observations inHRS Information for Plant Issue 0'
D2 - Assessments 40] v
& Inspection ilumart Factors Generic Planning Guidance in Communication
+ Coreinspection - C -* of Significant A I Pragram H.F. Inspection I Procedures Human Factors Findings Perfo nce --> D3 B2 u ted Re ew PR) -> in Inspection Human Factors Human Factors Reports inspection Guidance in -* Procedure
_, RegionalInitiative - Guidarece
& Reactive inspection improvements Program Procedures I
o @
, @^ @ @@
E Fi } } F2 F3 F4 Hu " Human p,c s Human Human Human Factors Training for Factors Factors Factors Knowledge
-> KSAs of Assessment Contractors & Experience inspectors Staff (e.g. DETis) of NRC Mgmt fCla Specialist & HQ Staff
& OJT)
. . , . _ it
. ..._.i.--- . '
O O O Facility inspection Plans and Activities to Assess Licensee Human Performance Are Addressed in the Plant Performance Review (A) Objectives: - Integrated Assessment of Licensee (Human) Performance Determine Needed Changes to the Current inspectiori Plan, Process: (Human Factors) Inputs to Assessment and Decision Process inspection Reports, Open item List, Status of Allegations, Enforcement History, Licensee Event Reports, Performance indicators, Human Factors information System Identify (Human Factors) Issues in Plant. Is. sues Matrix Assess (Human) Performance in Functional Areas Recommend Changes in (Human Factors) Inspection Activities
. s. .
y 4.A..----4.-+--- --_--.= 4-,u-,- ~_ -A4.m-h*4*. - w-en_- J -- -_-+-,,i-__,.J_hA.4--A,5.----J J J. 4 4.AA Jac.I.p J -w 41 _A__J=A.A.-.iAAu--4. en-~J-_a#wMw- _ha3A s:4A4h h umi. M A &4-d l I W U. C m l E m 4 ' O C O W 'P 1 - c c 8 m
~NW A
Z t e c . O %== Og 8 m i mm o e R
@ E C o C a t6
'C 5 5 We ,o o
4 t p E + a e - G O O
- 3 o t 'm o N C O e C V O o - Q. c
'O @ S E C
w
$S 8 5 m O e * -o em E O .
m & 5 O E o sI 3 ee O J
" ' ~
~'~r - , - - - ,_ , . . ,
o o o ) Human Performance Category How Individuals Do Their Work Within a Given Environment, Using l the Program and Processes That Support the Work i i ; Work Performance - Actions of an Individual During Conduct of 1 Work Activities - Attention, Awareness, Communication, etc. l Knowledge, Skills, & Abilities - Capabilities of Individuals ! including Training
- i Work Environment (Human Factors) - Context, Programs, and Processes That influence and Support The Execution of Work - i Physical Environment, Workload, Supervisory Oversight, i
Procedure Quality and Completeness, etc. ! t
i O O O Human Factors issues Are Documented in The Plant issues Matrix (A) Date Source item Description Anoarent Cause/ Comment 11/95 IR 95-21 Multiple Examples of UFSAR Errors Personnel Error During UFSAR Regarding SW System Review and Update 11/95 IR 95-20 Failure to Meet TS Action forINOP Personnei Errorand No Second Containment Air Lock Door Verification; Pressure Test Connection Left Open 6 days 1 11/95 IR 95-17 50.59 Evaluations forTemp Mod Contained Temp Mod Program Guidance Was Limited Technical Details; Overall Weak; Management Oversight Was Documentation Packages Were Poor Lacking 09/95 IR 95-19 Improper Procedure Usage by Nonlicensed Procedure Usage for Non-Safety Operators During Non-Safety Systems Systems Has Not Been ; Manipulations Emphasized i i
O O O , Human Factors issues Are Documented in The Plant issues Matrix (A) (Cont'd) I Date Scurce item Description Arwuarent Omnh 08/95 IR 95-15 Weak Document Management System For Poor Userinterface & Training MOVATS Testing Contributed to MOV 2SI- Resulted in Outdated Drawing Used 4 1188 Failure Event. for Field Work i ! 08/95 IR 95-15 Two Charging Pump Discharge MOVs Maintenance Tech Made Overthrusted Causing Valve Damage Unreviewed Change to MOV Wwing 07/95 IR 95-15 Maintenance on One Train SAVS Opwstions Support Error During i Inadvertently Affected Both Trains Work Planning . 06/95 IR 95-11 Spent Fuel Handled When Vent System Opeietur Unaware of Status; Poor Had inoperable Heater Supervision 1 t i !
O O O Human. Factors issues Are Recommended for inspection Follo.wup (A). Example: Operations Assessment of Licensee (Human) Performance: Recent operator errors represent a continuing trend. Corrective actions not fully successful Operators continue to perform well during plant events & transients. Operators not performing thorough plant tours. Questioning attitude and observant behavior not demonstrated. ! Recommended Changes to (Human Factors) Inspection Plan: Continue routine core inspections for the operations area. Resident inspectors should continue to monitor problem areas. Perform one-week inspection focusing on operator errors, licensee corrective actions scope and effectiveness of operator plant tours. _9_
E i rO E
. ~
g O O g 6 2 e o a e n. v a Q - C O L C .s $ 'E
- a~ E 8 5c E o a O
ac bE , Z 1 t e g_r
-ye g 3 3
a e a % O m - ) ~ Du CL =o K cp E o 03 gV . CU C -5D ' C L > t
. (U u) (D .C O 3m e pm cT a5 o m o $
Do, = co U Z y (U O .c gc o) 0.
. o 0)
- C
,9 e c en ,
Q O B o. E
- r O no@ o S te'g.Em > o m.
.cm E T c .c .t CCVx e o N zu c EE
= = v) 120 .e b .oe ww
- E c RR m a awRa <
e E en e w U
% r.
e o N O O u 2 $ $ Whr o o LO roo ooo O w - @ E K 5 LO o LO LO Z
' 3 "Nb e- h o ~3
- O 2 b co y; n co e z - g O 1
+ 1 - - , ~ . _ _ _ _ _ . _ _ _ . _ . _ . _
~ -
O O O 1 v initiative and Reactive Inspection Procedures (B2) Number Procedure 71715 Sustained Control Room Observation 41500 Training and Qualification Effectiveness 42001 Emergency Operating Procedures 42700 Plant Procedrues 93800 AIT (Augmented Inspection Team) O O O Inspection Procedures Without a Human Factors Focus Often include Human Factors Assessment Content (B) IP 37550 - Engineering Examples oi Human Factors Assessment Guidance Operating procedure and emergency operating procedure changes were identified, and implemented, as part of the modification package. Operator t,aining program revisions were identified and implemented as part of the modification package. Necessary documentation revisions for preventive maintenance, inservice inspection (ISI), and. inservice test (?ST) requirements were appropriately identified as part of the modification package. Design change calculations, analysis and design output documents ensure correct usage of design information between technical disciplines., e.g , process values developed by mechanical systems personnel are correcr/y used by instrumentation-and control personnel in set point calculations.
.=. .= _
O O O Human Factors Findings are Documented in Inspection Reports (C) Examples of Findings From IR 50-280/97-02 There are no procedures which specify material acceptability. I Procedures provided no limitations relating to reverse rotation of 58A and 58B fans. The weld leaks in the Unit 2 letdown line piping were another i exampie of ineffective corrective action. The cause of the missed sample was miscommunication between the control room personnel and the HP shift personnel. Labe/ing on breaker 1E1 on MCC-1H1-1 A did not match the description contained on the system, alignment sheets. Workers were unable to hear other alarms because of high noise level.
~
- o O O Plant Operational (Human Performance) Data is Monitored Daily and Discussed in the Headquarters
. Morning Call (G) l
Purpose:
Screening for Potential Generic issues
Participants:
NRR (including HHFB)/AEOD inputs: l Event Notifications (10 CFR 50.72, 50.55e, 21, 26.73, l 50.9) Regional Morning Reports Plant Status Reports Preliminary Notifications Projects' Daily Status Calls Outputs: Recommended Staff Followup Activitius (including inspections for Human Performance issues) z.- .,
l Human Factors Assement Branch Logic for Suggesting Followup Review Activities (G) Monitor Human Performance
#1 :
JL 1 r . Is an event no is performance no m involved? jL docilning? -
#2 yes yes 1r 1r is on-site Do human no is a s cific Does past performancs N no most recent no year HFIS An a no pro tem i sHFIS unalyses indicate .
evaluation (dontifiable? show spect - needed? pecific probi problem area (s)? area (s)? yes yes yes yes , U O Conduct on site 1r 1r $i evaluation of event using HPIP 1f If specific aree is Human Engineering 7 go to chart 1 Indicate for no if specific area is Procedures - A go to Chart 2 review ares (s)?in soec[ific end/or y,, if spectric ares is Training - go to Chart 3 If specific aree le Cy .a'.- - (C, . -h^'s .,Overeight,Orgentastion) go to Chart 4 Legend
#1 Monitor performance using the following:
Event reports; inspection reports; reeldent
& PM insights; LER's; 107Es; Regional &
Program Office insights; Senior Menagement meeting insights.
#2 Decoining performance is beoed on number ON of human performance probleme identified in HF18 the inspection report & LER rnodules within the last 3 months ad monitortr J insights.
l
Training Program inspection Protocol
.Jonitor all program (Page 1) 9 areas for declining performance,
#1 le .
declinin9 no t she d n 1/ performance no , A decl4ning 'yes 1 r ir Conduct IP 41000 on the specific program
,, area within 3 months, performance DO problem 1 '
Operatio,ns related y.s lI 1 r Do review Do results indicate no , review results no Indicate progra,m A immediate conc.mssa,fety wea$y..s Y'8 ,, *yes Request
/ JCO from '
facility is g, a "For Cause" am no - a "For Ciuse" no IP 41500 inspection 1 l' n' c needed?
"O '$ Y'8 l
JC0o.e a s th,an uaie , , , , Condact IP 41500
*yes on Ops Training 1' within 3 months, Conduct Op Eval immediately 1 r i f Do IP 41500 t rmns no .
Does Op Eval Support JCO7 00 '"' Appr>priate B msulta indic* a content DO
[a MT Actions problem f M hm
*yes *Y'S 'yes 1 P 1 r Conduct IP 41500 within 1 year ]r" S @
Issue Date: 06/13/95 A-5 41500, Attachment
Training Program inspection Protocol (Page 2) O m 1P De roylew retutta Indicate immediate O safety concems?
*y9S 1 r 1 P 1 r 0
i i y nort year. 1 r
\
AN no e nam tosulto ; unacceptable' L ,
'yeS 1 r d
O 1 r o i Do Do IP 41500 results no IP 41500 results no Indicate a ;
@- indicate en ;
content proble [ SAT process problem? Legend ; St Monitoring may include enam reports, inspection reports, LERs, resident inspector insights, and requalification examination I results from IP 71001 L F2 Determination should incPde: wo ers Take
-degree of performance related concern no meet m'nimum ; appropriate
- Residontinspector insights requirements? ectlens.
- Operating history trends (LER's, etc.)
(Accreditation status should only be ,, considered when determining a data for a *y45 - IP 41500 inspection.) : 1 P 83 Program weaknesses of most concem are those listed in ES 601 N entemd thru Go To if Cntered thru , Go To Input to "yes" responses should include 11 entered otherwise, Go To @ - Region and Program Office discussic:. and review. O Review may include meetings with licensee up to and including en enforcement conference 41500, Attachment A-6 Issue Date: 06/13/95
- . . . 1 l
Human Factors Training is Provided for NRC Inspectors (E) Required Training with Human Factors Aspects (TTD): Fundamentals of Inspection (G-101) Inspecting for Performance (G-303) Root Cause/ incident Investigation Workshop (G-205) Effective Communications for NRC Inspectors (ECN) PRA Basics for Regulatory Applications (P-105) Optional Training with Human Factors Aspects (TTD): Human Performance investigation Workshop (G-206) Human Reliability Assessment (P-203) Examination Techniques (G-107) Root Cause/ Incident investigation Refresher (G-207) j incident investigation Team (IIT) Training (G-600) '
m m l~Jm mp -
- e e o Human Factors Training is Provided for Reactor inspectors (E)
(Continued) Professional Development Courses with Human Factors insights : Gathering inspection Information Through Interviews (GINFO) Interviewing Skills Workshop (IS)
-Analyzing Human Performance Problems (AHPP)
Problem Solving and Decision Making (PSDM) NRC Inspection Team Leader Workshop (NRCIT) Outside Courses with Human Factors Content: Human Factors (U of Michigan) Nuclear Power Reactor Safety (MIT) Design of Human-Computer interface (U of Maryland) TAP ROOT (System improvements,- Root Cause Analysis (Performance Improvements international) THERP Seminar Project Management and Management Courses (Various) l
l Q Q O Human Factors Assessment Specialists are Integrated into
- the-Inspection Process (F2)
Human Factors Assessment Branch Staff Support inspections Involving Significant Human Factors Emphasis Nine Professionals: . Senior Operations Engineer Senior Human Factors Engineer Senior Human Factors Specialist Human Factors Analyst Hurnan Factors Engineers (2) Training Assessment Specialists (2) Engineering Psychologist 3.0 FTE Budgeted to Support inspections 17 Inspections Supported in FY 1997
195WWWPN y g 9 9 l l Diagnostic Evaluation Team (DET) Inspection Guidelines Evaluate Management and Organization (M&O) Factors (F3) Contractors Are Used to Support Management & Organization Evaluations M & O Evaluations Are Broadly Scoped To include the Following: Management Factors Ogsnization Factors Mission and Policies Organization Structure Goals and Objectives Attitudes; Safety, Quality lavolvement and Oversight Organizationai Teamwork Priorities and Decision-Making Accountability / Ownership Planning and Scheduling Communication / interfaces Work Processes Training and Qualification Resource Allocation Standards and Expectations Roles and Responsibilities Management Info Systems Performance Monitoring Performance Feedback Corrective Actions Programs improvement Programs _m i
Inspection Tleport Human Performance issues are Coded and l Compiled in the Human Factors Information System (D1) Human Performance issues Are Coded by Category-
. Training !
~
l
. Procedures and Reference Documents l l
. Organizational issues '
- Management and Supervision
> Communication
. Human-System Interface and Environment
. Work Factors 1
I
01 No Training T1 Initial T Training 02 Training LTA T2 Continuing /requal 03 Training process problem T3 OJT 04 Task qualification T4 Simulat:r 05 Individual knowledge LTA 06 Sis.ulator fidelity LTA / 11 No procedure / reference documents P1 General operating P Procedures and 12 Procedure technical content LTA P2 Abnormal /off normal / Alarm condition Reference 13 Procedure contains human factors deficiencies P3 Emergency (EOPs & Emergency response Documents 14 Procedure / reference document development and PA Reactivity control wintenance program LTA P5 Maintenance / modificat%' P6 Surveillance / cal 1 brat 1ur/ test P7 Chemical / radiochemical P8 Refuel 1ng P9 Administrative P10 Special 21 Inadequate staffing 01 Staffing 0 Organizational 22 Poor task allocation issues 26 Inadequate controls / poor scnedule design 02 Overt wa 27 Excessive number of hours worked / fattgue 28 Frequent use of overtime 31 No supervisio9 M1 Management and Oversight M Management and 32 Inadequate supervision / comand and control Supervision 33 Management expectations or directions LTA
?* Percet ad pressure to complete task 36 Scheduling and planning L N M2 Work Control / Organization 37 Worker Sele tion (unoualified worker) 40 Individual corrective artion LTA M3 Corrective Actions 41 Action .ot yet started or unt w ely 42 No action planned 43 Operating experience review LTA 44 Correctt_ve action program tiA y 47 Root cause LTA 48 Audits or problem icentificatto.i i TA M4 S?lf Evaluation y
hi 51 Misunderstood or misinterprsted information C1 Cral C Comunic ton , 52 Comunicatien not timely 53 Comunication content LTA lC2 Written 54 No comunication/information not cocrunicated 55 Communication eculomeet LTA 61 Labels LTA H1 Alarms / annunicate's H Human - System 62 Information organization or format LTA l H2 Controls / *rput dovices Interface and 63 Size. shape or coding LTA H3 Displays Envircnment 64 Placement or location LTA H4 Panel or Workstat'on layout 65 Information content LTA H5 Equipment 66 HSI does not exist / not available H6 Tools and materials 67 Function LTA (accuracy /prccision) 68 Too much information to effectively monitor , 71 Temperature / Humidity LTA H10 Environnent 72 Lighting LTA 73 Notse 74 Radiation 75 Work area layout or accessibility LTA 76 Postings LTA 81 Work package quality LTA W1 Work Practices W Work Factors 82 Brieftng preparation or turnover LTA 83 Work practice or craft skill LTA 84 Non-conservative cecisionmaking/ Questioning attitude LTA 85 Team interactions LTA 86 Work untimely 87 Wi.'ful non conservative action 88 Tag outs LTA 89 Housekeeping LTA 91 Independent verification / plant tours LTA W2 Awareness / Attention 92 Self-check LTA 93 Awareness or attention LTA 94 Log keeping or review LTA July 8. 1997
- o O O Human Factors Findings are Documented in inspection Reports (D1)
Examples of Findings From IR 50-280/97-02 (With HFIS Codes): There are no procedures .which specify material acceptability. (11) Procedures provided no limitations relating to reverse rotation of 58A and 58B fans. (12) The weld leaks in the Unit 2 letdown line piping were another example of ineffective corrective action. (40) The cause of the missed sample was miscommunication between the control room personnel and the HP shift personnel. (51) Labe/ing on breaker 1E1 on MCC-1H1-1 A didnot match the description contained on the system alignment sheets. (61) Workers were unable to hear other alarms because of high noise ! l
/ eve /. (73) l tii' 7 Pr Mi- 'i
a g. 18 - Oi
' " M - - "
0 O U1 O c1 o M ci Cu o (O m A o - -- m m a , g - m Tra. .ining - 2
$m FC a
3 e
' a m ca Procedures m,, -
9 3 3 c u e 3 U
- oL ]
0
$y
- y co OT & Staffing A 0
- h 2 o
O e+ a
. m c M 3 m Mgmnt & Supv m p [ ,
@- ]
. 33 l-
$ Communication
' 3O. o o
g 7 ,___ m }_ 2 _. __
$ HSI a Environ . ,
. . .- Sh v> g
; 3 L__
Work Factors * - M n
;l l
I e _ _ . _ . . _ . _ _ _ _ _ . _ _ _ _ _ _ _ _ - -__.__.__________-_-________...___._____._____m__ _ _ . _ _ _ - - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ . _ _ _ _ . -
e o en
' f0 CJ A (n A o o o o o a y a 'D y Operations ' >
m
. : E 1
$m o>
to Inst. & Control C O A ' O Electrical y in . E, r
- a C
o O g L f Mechanical . f
=a .
"T1 io Chemistry _
p g % 4 o m CD Q e==t= - io Rad Protection : Eis o A n , 3 2 V
- a. Y Y a
, Engineering o- ,
x O - o Training e J \
- 3. in . . .
g% 0 2 3 3 0 2 3 QA/QC S a
- 5 co I ns to Plant Management W r-+
g. iu Contractors . o a Planning : : : 9
s BBB a m o> m mm bBB A V 0) zzz C 11 11 11 s y e m SSS m o a m m o o o o o o Q
$i8 Plant A
~
X z ] c S8% 3 T (
~
Plant B X t (D 888 Plant C
^
X $ E "M* Plant C
= = m ,
. X -8 8 m . 0 --j 880 Plant D N X h.
s E 8
$Q O
85 e Plant E X 888 o a @Jc) Plant F _- X $ g 888 Plant G X 8 T !" GG0 ant H .* X S T o O k Plant I - X z bb A88 Plant J '
-X 4 EQ s 5' t-t I 8"8 Plant K X g 8
*O z n3 su 88s Plant L X 2 3%
R88 Plant M X aO
$$$ Plant N .
i X . 8G8 Plant O X E g 8s@ Plant P m 88$ Plant Q -
MW
- e e O l
l Additional Headquarters-Regional Coordination Occurs l To lategrate Human Factors into The inspection Process l 1 Regional Operating Plan and HQ Operating Plan Development Task laterface Agreements , Headquarters Participation in Regional Counterparts Meetings Headquarters-Sponsored Workshops with Regional / Industry Groups Procedures Training Telephone Phone Consultations on Human Factors Matters inspection Planning , inspection Report Preparation and Characterizing Results Information Notice Development to Feedback Significant Findings l
9 g Simplified Progs Dicgrcm for integrating Humcn Fcctsro Canaideratisns O into the. Inspection Process , Human Performance G '- Event
- Compile - for PPR and Reports ,
Routine inspection , SMM Probriefs
~
MonitorinC Report l of(Human ) + Human Factors Performance Pec r -> at Plants information
- Human Factors in HFIS Information B' I N h MB D2 - A***** men:s v a inspection Human Factors Generic Planning Guidance in . Communication
> Coreinwtion -
C + of Significant A I Program H.F. Inspection Procedures Fi W Plant Human Factors Rev (PPR) J* B2 ed D3 Process in inspecdon , Reports Human Factors Human Factors inspection - Guidance in RegionalInitiative
- Procedure Guidance
& Reactive inspection improvaments Program Procedures A =@
(0 @" @ @@ E n Fi i f fF2 F3 F4 Huma Human F M ors Human Human Human Factors ~ Training for Factors Factors Factors Knowledge
'"" -> KSAc of Assessment Contractors t< Experience (Cl Inspectors Staff (e.g. DETis) of NRC Mgmt a g,q Specialist a Ho staff g-
O O O
., s,,
United States Nuclear Regulatory Commission l (g/%,,, ) ACRS Human Factors Subcommittee Human Performance Human Reliability Plan Status Briefing January 21,1998 J.J. Persensky Office Of Nuclear Regulatory Research
O O O p ~m,,, [ ) United States Nuclear Regulatory Commission
Background
Human Performance Program Plan - 1995-1996 Human Perf. & Human Rel. Plan - 1997-1998 Last ACRS Briefing - Oct.1997 - ACRS Letter - Oct. 8,1997 High-level model needed Base model on insights from operational experience XRC Strategic Plan - September 1997 NRC Performance Plan - September 1997 Office Operating Plans
/
f ~s,, [ ) United States Nuclear Regulatory Commission Statu.s 1 Draft Plan to ACRS - Aug.1997
- Revisiting Plan Structure Based on Agency Strategic Plan
. Revisiting Model Data / Event Driven Framework Revisiting Prioritization Absolute Ratings Analytical Hierarchy Process (AHP) l M . . . .
. k.
O O O
- ,s " %,,
[ } United States Nuclear Regulatory Commission 4 Plan Structure - Based on NRC Strategic Plan
- Mission Vision
- Strategic Arenas
- Strategies Substrategies
- Activities 5
3
l 0 0 O l' f'p arouq%.
,)
{ United States Nuclear Regulatory Commission
\ ,..v... /
Linkage Among l Planning Activities 1 l Human; Office- USNRC- USNRC [*"**' Opemting; Pedonnance Stmtegic Human :
! Plans Plan - Plan -
Reliability ; j "lan
; (q m , .
~ ,. .,
HP&HRIP. - Planned - - 2 Activities: Accomplishment-I
~-
t v
4 O O O United States Nuclear Regulatory Commission ( .....
)
STRATEGIC ARENAS
+ Nuclear Reactor Safety Nuclear Materials Safety Nuclear Waste Safety Common Defense and Security and InternationalInvolvement Protecting the Environment Public Confidence
. Excellence f
o o o United States Nuclear Regulatory Commission I Strategic Arena: Nuclear Reactor Safety Strategy: " We will assure that licensees discharge their primary 1 responsibility of conducting safe operations." Substrategy: We wil! regularly assess, objectively measure, and report on licensees' performance ..." i HP&HRP Activities: Identify measures of organizational effectiveness and assess their potential usefulness as predictive indicators
; of plant safety and risk.
Conduct systematic evaluations of event data to identify safety critical human performance trends and insights
- Conduct programmatic inspections related to human performance c!.
O O O i
,s ~c%,
- [ i United States Nuclear Regulatory Commission
%,...../
Strategic Arena: Nuclear Reactor Safety Strategy: "We will maintain a research capability to provide { timely and independent technical bases for NRC regulatory decisions." Substrategy: "We will pursue a reactor safety research program that includes elements of both confirmatory and anticipatory research to meet current and projected regulatory needs ..." HP&HRP Activities:
- develop guidance for reviewing HSis associated with advanced alarm systems, hybrid control rooms, and control-display interface management a Develop improved guidance for root cause investigation of human-performance related events
]
! O O O , i ( .....
)
United States Nuclear Regulatory Commission MODEL
- Data / Event Driven
- Supports Agency Strategic Plan
- Function Orientation
- Serves as Overall Framework for Assessing Input and Identifying Activities Specific Example I
, t
Driverslinputs Nuclear industry Experience Risk insights Other Industry Exp. Allegations Stah Expertise Events Industry Initiatives, e.g., Events Assessment DigitalI&C Performance Assessment
.. Standards - Regional Requests Plant Perf. Improvements User Requests Deregulation Operational Reviews Decommissioning
^ s , , n Risk \
,/ [ ./ 'x (yignificance 2 Monitort '(Prioritization N
'x s
, Inspect /
/
'v' Review s\/
/ N Collect!
- Professional ..:
Analyzel Evaluate (_/ HP&HRIP x O. 4 ACTIVITIES a p- ~%
. Administrative "v,"a,g y
(A J
- r Operating Plans Performance Plan Strategic Plan - k o.-- .. -. . . . - - . . . . _ . . ..- _ - . . - . ..
L Inspection q ,.
,_ _ P_._ ' r- m Findings /[pection ,
Report ' w- f' i r Regulatory . - 'f Weaknesses .IJcensee Weaknesses
' F p 1r ' r i r oo f Process ( Requirements
( - HSI {Procedu- { Org. Factors Training -) Communications Y Y Y
' r
' f-m ,
Data Base Regulatory
' r- m Action ;
Ceneric f> Communication l Analysis p-4_d 4 User Need r
.h Anticapatory Research Research
- Activity
' r -,
gg y, -- 7 A Italics indicate _.h_ Sfonitorf Link to HP&HRIP Activities Rev W , 5 _ - - _ , __ _. . __ _
w User Need O [551[\
'x Expertise,/ n
-D'efib.
5 Problem
- s. -
/-
~-
i r I ! Data feeF '
'iterature inlug Then@
_ Base Interaction .s Experience , Other Arenas ,
, w - - .
%s r.s State of the Art
& Practice
%s
.?"
Scope &- Approac21-
-m u ,
r A 9
/ Adopt Adapt u
' Existing _
Tool Existing Tool g " Develop
~
D~onduct Conduct from Field / Case Lab /Sim fmm SME Study Study Existing Interaction Dab i r Data l 1 1 I
+
Analysis / n Synthesis r Data Base l l m n.- Tool u f~thalics Link to Indicaterm _ Te$nical HP&HRIP .h Reports
%ac,ivi,ies uonitori Inspect /
a- g Review
i O O O 4 l ,s* "*%,,, ( ) United States Nuclear Regulatory Commission ) PRIORITIZATION l
- Absolute rating
- Seven Criteria Numerical Ranking
- Analytical Hierarchy Process (AHP)
Paired-Comparison Technique l 1 Three Criteria
/2j
) O O O s, . [ } United States Nuclear Regulatory Commission PRIORITIZATION CRITERIA Criteria for Absolute Ratings l Safety / Risk significance Commission Milestones Reg. Effectiveness / Efficiency Reduce Uncertainty Operating / Licensing Efficiency
- National / International efforts i
Criteria for AHP Safety Regulatory Policy 4 Likelihood of Success 13
O O O
' p aro
/j uq\
( ,) United States Nuclear Regulatory Commission i
%,**4 *** /
i SCHEDULE
- ACRS subcommittee - Jan. 21,1998 Reformat, reconsider activities, prioritize activities - Feb.- Mar.1998
- Draft Plan to ACRS - Mid-Mar.1998
- Full ACRS - April 1998
- EDO/ Commission - May 1998
, i
O O O i
,p' ""%,,
, [ ,) United States Nuclear Regulatory Commission
\ ,..u... / l i
., 'k , Back-up Slides ! L l t t l
/.q
[ } United States Nuclear Regulatory Commission
\, /
MISSION
" ... to ensure that human performance and hwnan reliability are incorporated in the agency's activities to regulate the Nation's civilian use of byproduct, source, and special nuclear materials to ensure adequate protection of the public health and safety to promote the common defense and security, and to protect the environment."
14
O O O :
,s ~~ _ i i j United States Nuclear Regulatory Commission VISION
" In implementation ofits mission, Nuclear Regulatory Commission human performance and human reliability actions enable the nation to safely and efficiently use nuclear materials. NRC's human .
performance and human reliability actions should be such that the public, those it regulates, and other . stakeholders in the national and international nuclear community have the utmost respect for and confidence . in the NRC." 17 :
O O O
,p* ""%,
[ .) United States Nuclear Regulatory Commission j
%,**-*** /r t
GOALS
- Prevent radiation related deaths or illnesses due to civilian nuclear reactors
- Prevent radiation related deaths or illnesses due j i
to civilian use of source, byproduct, and special nuclear materials
- Ensure treatment, storage, and disposal of nuclear wastes in ways that do not affect this or future generations
- Prevent loss or theft of special nuclear materials
- Protect the anvironment through implementation of .
applicable laws
- Provide all stakeholders with clear and accurate information I
- Carry out agency programs efficiently and effectively i;
t
O O o 1
,s -~~.
United States Nuclear Regulatory Commission ( ) , STRATEGIES - Nuclear Reactors j Assure licensees conduct safe operations Incrementally implement risk-informed, performance-based regulation j Position NRC for licensing future reactors and renewing licenses Position NRC for reactor decommissioning Maintain research capability ; Improve measurement of plant performance i i
* ~
! 19
O OBSERVATIONS ON HUMAN RbLIABILITY AND NUCLEAR REGULATION Presented to: Human Factore Subcommittee Advloory Committee on Reactor Safety Nuclear Regulately Commiselon Proeented by: Marty Stutzko Senior Risk and Reliability Engmeer . Science Applicatione internabonal Corporation 20;P01 Century soulevard, Third Fioor Germantown, MD 20874 omee:(301) M34179 FAX:(301) 4284144 Emett: ma%stutdagccmeil.gmt.saic.com January 21,1998 O HOW DOES NRC REGULATION RELATE TO HUMAN RELIABILITY?
+
NRC regulations exist to protect the public health i and r.afety NRC has adopted a risk-informed regulatory philosophy - Important questions:
- is there a problem with the ex; sting level of human reliability in nuclear power r 9 4?
- Can we quantitatively axpiata now influencing factore
, aNect human reliability?
- How can human reliability be predicted in a riek context?
O 1
O RECENT HRA RESEARCH
. Most recent HRA research efforts have focused on the development of " fundamental" models and frameworks of human performance, not observation, measurement and analysis of i events involving poor human reliability l NFtc should perform some fundamental data l
collection anri analysis: t - Datatmoe of Ali. events involving poor human ! , performance
. Actual events occurring at nuclear power plants
+ Simulator training
- Summary n,tatistics; data-driven (not expert opinion)~
estimates of human error probabilities / frequencies G HRA ANALYSTS ARE OVERLY FOND OF ANALYZING THE ACCIDENT AT THREE Mll.E ISLAND
= The times have changed; post-TMI modifications affecting human performance have been implemented: .
l - Symptom-oriented emerge.cp operating procedures
- Improved operator training
- Addition of the Shift Technical Advisor (STA)
- Creation of the Technical Cupport Center (TSC)
- Additional and improved instrumentation
- NRC should measure the quantitative risk impact that post-TMI modifications have had on human l performance Ol 2 1 l
HRA RESEARCH REQUIRES A CLEAR FOCUS
- NRC's mosaic of HRA research:
- shutdo vn and low power risk
- errors of commission (emphasis on full-power events)
- error forcing context
- premature termination of safety systems
- safety system " bypass" events
- organizationalfactors NRC should develop an integrated plan to direct HRA research in the future HRA research should be driven by observed (not
- hypothetical) risk eignificant events -
O NRC SHOULD USE PROPER RESEARCH ' PROTOCOLS AND METHODS Proper research demands adequate debate and peer review
- Conference papers and journal articles are useful in communicating NRC research, but are no substitute for debate and peer review
- NRC should aggressively solicit comments from ALL interested parties on HRA research ALL supporting data and evidence used should be available for public examination r_
,5 O
3
O O O . Human Performance and Reliability Research and
- Application '
1 i i Ali Mosleh Presented at the i NRC ACRS Human Factors Subcommittee Meeting January 21,1998 i
O O O Questions Common to All Human Performance Issues f
= What factors affect human performance? = In which way will the performance be affected (e.g. error modes)? : = Are they risk / safety significant'? = How can we prevent them or mitigate their impact?
O O o , ! Observations . o Human performance assessment and prediction are very difficult problems o We must have a realistic assessment ofthe what ' it takes to solve them o The strategy should reflect the level of resources we are willing to commit. Our expectations of the outcome should be defined in this context.
o o o HRA Needs l
= A systematic method ofidentifying and integrating correct or incorrect operator response into risk models (for both pre and post imitator actions) = A credible method for quantification of human events
O O O Obstacles and Challenges -
= Current PRA modeling framework (static, hardware oriented, passive)- Not very suitable for
" context" modeling
= Lack of a scientific (theoretical or empirical) basis for identifying and relating " context parameters" to performance
= Actual observations are limited and controlled experiments to collect data on the combined effects of various " context parameters" are prohibitively resource intensive.
- O O O ~~~! .
i A Formula That Has Worked ' I i
= Observe DDefine Probelm DModel DObserve DModel D D Solution
= Do not reinvent the wheel
! = Do not over extend the model/ tool beyond its design domain
= Bring the broadest possible input and j participation by domain experts j i
O Appendix B IIPED Data Fields Table O O B-1 l i l I 1
HPED Data 2 Ids Tcble Field F5 eld Title Definition and Codes, Definitions, and Seggestions for Things to Consider %%en Coding Examples ' Locating Data if Field Size Limit 1 NUDOCS NUDOCS nurnber e.g.,91 1212 0028 Find on report Be sure to include the report number, not the cover letter number. 10 characters 2 RPT_TYP Type of repost AIT-Augmented Inspection Find on report A three character code. If source types additional to those Team report cover currently identified are useti, create a unique code. 3 characters llT-InitialInspection Team report ! IIPS-Iluman Perfom ance Study report LER-Licensee Event Repo t STI-Special Team Inspectio report 3 LER Licensee Event Report e.g., 3339104 On report cover f ence or 8 characters LL R inde.v. 4 IS_IT Inspectioc *:am formed Y-yes, N -no In report Were NRC personnel sent to site to analyze the event? If the data I logical charac er source is an NRC report, then the mswer will almost always be "yes." If an LER mentions an NRC inspection team, then the answer is "yes." 5 B_IDATE Inspection visit (month / day / year) " In report inspection visit dates should bc inclusive. If two separate sets of
. . e.g.,03/12/90 dates, use the set where the substantive investigation took place 8mnmg date (usually the first extended visit). Inspection is used in a broad 8 character date way, to mclude the visit dates of study teams, such as the IIIS teams.
B-2
Field Facid Title Definition and Codes, Definitsons, and Soggestions for Things to Consider When Coding
. Examples Locating Data O Field Size La.mit 6 E_ DATE Inspection sisrt ending (month'daj/3ear) In report Irispection visit dates should be inclusive. If to sepame sets of e.g.,03/20 90 dat.-( use the set w here the substantise insestigation took place (usua!iy the first extended sisit). Inspection is used in a bromi l 8 character date way, to include the visit dates or study teams, such as the HPS teams 7 , ' 19 Facility identification (see " unit info" database for Listed in
- unit Some Inspections include aII units at a facihty. Include the FID cdes) info ~ database for the primary tmit ofinterest. Or prst include the first umt FID 1 characters e.g., ANOI and be sure to explain in " Event that is w as a multiple unit
! inspection.
5 FAC TYP i vality type P-PWR in report or " unit B B%R info dbf" I character pg 9 NSSS NSSS vendor G-Genera! Ecctric in report or " unit W-Westinghouse info.db!' [ l character C-Combustitm Engineering B-Babcock & Wilcox A-AC 10 REGION NRC Region I.2.3,4,5 In report or " trait Currently, the datab se includes Regions 1-5, includmg ca , nts that info dbf' occt:rred after the mercer of Regions 4 and 5. This was done so I character that a plant w ould not be listed in two regions. 1I EV.DATE Event occurrence date (month' day / year) Date when The event data is usually given in the report text. Use the gisen critica' sequence esent date, esen if operability was chaIIenged cariser. If no event 8 character date (direct cwnt date given, ther use the date when the tech specs / operating prwursor) began conditionCoperabil:ty were challenged. even ifidentified a* a later date. If it occurred at some unknown time, the last date of operability will be used. B-3 ra______.. . . _
"L.aon
Field Field Title Definition and Codes. Definitions, and t agestions for Things to Consider When Coding C Field Size Limit *"E 'I"3 *** 12 EV TIME Time evcnt occurred (24 hour clock) Time critical For many n ents, the f. vent time is straightforward - w hen the e g,0126 sequence began single initiating event occurred. Ilowe er, sometimes it is harder cters to identify a single point in tune for a hen the cent started (e.g, an inadequate condition has existed for a w hile but hasjust been noted). Use the event time that indicates when the plant was placed in the conddion identified in EV_DATE. Use the focal time. 24 hr clock 13 IPWR Initial reactor power (% numbers only) In report 3 numerics
#'E' le F_PWR Final reactor power (% numbers only) In report Use the minimum (or maximum) power leveljust after the
, e.g,50% initiating event (within one hour). fn indicate if the transient was a large change or slow degradation. Jould be steady state where the initial and final power les els are the same_
15 CLASS Event emergency pla ALT-Alert Usually
. GEN--General Emergencv mentioned at the Use the highest levi of emergency, if there are more than one classification SAE-Site Area Emergency end of the report classification used.
3 characters UNU-Unusual Event NCL-not classified UNK-unknown 16 IS_CLASSOK Event classified Y-yes N-no Usually only Does the report indicate that there was a pntlem with event no*ed if NOT classification? Did the study team determine that the classification C""'CIII correctly was cerrect or incorrect? (This wit! default to'no'if the report I logical character classified does not mention an incorrect clanification, therefore input "yes' if the report does not directly indicate a classification problem.) i B-
Field F5fd Title Definition and Codes, Definitions, and Soggestions for Things to Consider When Coding E mP esl Locating Cat,
# Field Size Limit 17 RX_ STATUS Reactor status at time of B-extended outage in report C-routine start-up D-routine shut-down I character E-steady-state operation I-load change during routine y power operation H-refueling I-cold shutdown K-hot shutdown L-hot standby X-other Z-unknown'not applicable 18 EVENT Event summary In executise Summarire the o ent. Was there a major equipment failure as the summary initiating event? Was there a e wat or automatic scram? % hat 3 254 characters was the cent outcome? To tne extent possible. this will be pulled frem the title, summary, initial section of the report. For LERs, mciude the Resision number (e.g Rev. 0)as the first item in this field.
19 IS_PRECED Preceding subevents Y-yes. N-no Report mentions Were there subevents that preceded the mitiating event that were series of relesant to the event occurrence? One w as to think about this is to I logical character incidents think if you u ere w riting a timeline of all relesant factors, would occurring before there be anything included before the cent? Examples might be event some maintenance or test activity that left valves in a particular configuration; another example is an operating o ent that led operators to behase in a certain w ay the next time sornething happened. 20 IS_ FOLLOW Following subevents Y-yes. N-no Report mentions Were there subevents that followed the initiating event that were series of relevant to the cent continuing'recosery? One way to think about I logical character incidents this is to think if you were writing a timeline of all blevant occurring after factors, u ould there be anything inch.ded after the cent? event B-5
Field F5td Title Definition and Codes, Definitions, and Suggestions for Things to Consuler When Coding Examples Locating Data , O Field Size Limit 21 IlUM_ACT lluman actions Executive Summarize the human actions. Equipment should only be summary mentioned ifit was acted upon by someone. (Equipment info is
""*"U summarized in EVENT.) What actions did operators perform.
254 characters including recosery actions? This is a high level, oseral'surrmary of the human actions such as alt rod causes. Further descriptions can be included in the following sections. 22 IS_IIF lluman factors a Y-yes. N-no in esecutise Sometimes determining if a factor i< a contnbuting factor is summary or straightforward. If the factor is cited in the report as bemg a contributing factor ; ; g; 7 g ; g, . RmMhh I logica! character body coder should not really be interpreting the analysis of the report. Other times, one needs to think if this performance shaping factor was a critical factx in determining the human performance, was it not at all importsat. if this factor had been ditTerent. would the outcome of the event have been more or less the same. 23 IIF lluman factors in executive iluman-machine interaction issues list any items of interest, such summary or as poor labels, arrangement' placement, inadequate wmmary section in report instrument' displays, inadequate controls, unit differences. 254 characters body 24 IS_WORKENV Work environment a Y-yes, N-no In executive summary or [see input for IS IIF] contributing factor ; ; I logical character body 25 WORKENV Work environment in executive Any work area'en ironmental issues.such as houscLeeping poor, summary or hot, cold, poor lights, noisy cramped high contamination.
""***7 section in report vibration. respirator ensironment. l 254 characters body 26 IS_SYSDES System design and Y-yes. N-no in executne !
. summarv or [see input for IS IIF]
configuration a ; ; contributing factor body I logical character i B-6 i
y - J - i Field IIld Title DeEnition and Codes, Definitions, and Suggestions for Things to Consider When Coding 0
.
- Examples locating Data Fie8d Su.e I s.mit 27 SYSDES System design and in exect 've Describe instrument configuration and other system factors, such summarv or as analog or digital interface, too much informatkm to be c nhguradon wmmary '
section Ei report processed concurrently. change in trend too slow to observe 254 characters body mcrease or decrease. Conditions for response (conditions undetectable, conflictmg conditions, incorrect indication) may be listed here or under human factors engineerine. 28 IS_ PROC Procedures a Y-yes. N-no in executive contributing factor
"* mary w
[w input for IS_IIF) section m report I logical character body 29 PROC Procedures summary In executive include any information about procedures. Were they used' Was summary or a wrong procedure used' Was the procedure incorrect or 254 characters section in report incorrectly followed Were the procedures poorly presented - body text confusing. poor graphics, etc.? Be sure to include the type of procedure. Would also include here if the pnefure pnp is , poor. 30 IS_ TRAIN Training a contributing Y-yes. N-no Iri executive summary r [see input for IS IIF] factor secteon m repart I logical character body 31 TRAIN Training summary In executive Any training issues, either individual (refer to personnel number
,54 characters summary or assigned in database) or as a group. Discuss simulator ismes.
section in report identify if simulator does not match actual conditkms. body 32 IS_COMM Communication a Y-yes. N-no in executive contribu.ing factor " * **'l I'** I"P"*I"'IS IIF] section m report I logical character body
= . - ~
B-7 : h
A
~
\
Field fed Ttle Definition and Codes, Definitions, and Suggestions for Things to Consider When Coding {
**P * "A *"
# Field Size Limit l 1
1 33 COMM Communication In executisc Any communication issues. including any teamwork questions and summarv' te coordination issues (unless better described under
** **'I section in report supenisor' command & control or Management & Organization).
254 characters body Use phrases such as shift-turnover, wrmen. oral. policy, w rong communication. confusing, conflict cross-checking. intramwp or intergroup coordmation as .4n,q= wie to identify the type and probkm with communication. 34 Supenision' Y-3 es, N-no In executive IS_ SUPER command a control a
* * "*'I
' I'##I"I"'I"'ISIIF] -
section m repat contributing factor body I logical charster 35 SUPER Supervision / In executise includes the issues of Immediate Supenision and also includes
""""*'I" '""**" ' ' *#* '# "P'" E' "I command & control "."
section in report a hat's happeninc. leadership of subordinates, mon:tonng. etc. summary body 254 characters 36 Management and Y-y es. N-no In executive IS_MO summarv or- [see input for IS IIF) orgamzation a section in report contributing factor body I logical character 37 MO Management and in executise Address the crewr shift organization here (idemify any dual roles). summary or Also addresses things such as policies. administrative control. organization summary section in report delegation resources. 254 characters body 38 Fatigue' stress.' work- Y-y es. N-no in executive IS_WL load a contributing **"."*'} "' I'#*i"P"'Ik*ISIIF]- sectron m report factor body I logical character B-8
. p -
Field Field Title Defintion and Codes, Definitions, and Suggestions for Things to Consider When Coding
. . Esamples Locating Data # Field Size I;.mit 39 WL Fatigue / stress! work- In executise Address anything related to fatigue, stress, or wenLicad_
load summary . section m report 254 characters body 40 IS INDIV Individual personnel Y-yes, N-no in executise
. summarv or [see input for IS HF]
issues a contributmg . .- sectxm m report factor body I logical character 41 INDIV Individual personnel In executive Any other issue or item ofinterest that doesn't seem to lit
. summarv or
- elsew here, relating to an individuars actions (e.g.. situatkmal ismes summary
,,c,;o, ;n report awareness, decision making. cognitise tunnel sision, mcorrect 254 characters body hypotheses).
42 NUM PERS Number of personnel 1.2,3.4.5.6.7,8.9 Number of The number of personnel coded represented the number of pew!c personnel insolved in the esent about -shom you has e suflicient infivmatico covered in the to add to the database The question is what personnel are i numeric report in considered to be ins olsed' Personnel w ho have significant xthms sufficient detail in the esem should be included. Especially if by lindmg out and entering informatkm about them will add to the understanding of the human perfonnance related to the esent. Personnel have to be mentioned by title in the report. 43 P_ TITLE Person's title in report body 254 characters 44 P GROUP Personnel group MGR-Manager in report or E* "" ' #I "*"' *** *
- 50 characters OP--Operator title ENG-Engineer MTG-Maintenance Group ADM-Administrator B-9
Field Fb Title Definition and Codes, Definitions, and Suggestions for Things to Consider % ben Coding
. Examples Locating Data
# Field S.ue 12mit 45 P_ TYPE Personnel type SS-Shift Supervisor in repmt or
# # " ***"' # I"*
60 characters RO-Reactor Operator title STA-Shift Technical Advisor NLO-Nonticensed Orcrator CTE-Contract engineer CTA-Contract administrator CTM-Contract technician' mechanic CTR-Contract radiological CTS-Contract security UDE-Utility design engineer UPE-Utihty plant engineer USE-Utility support engineer UPM-Utility project management OPC-Utility outage planning and control UTA-Utility administrator UTM-Utility mechanic UEL-Utilny electrician UTC-Utility technician URD-Utility radiological USC-Utility security UMG-Utility manacement 46 DUAL _ ROLE Dual staff role Ifmentioned in Are any of the personnel participating in a dual role capacity? 25 characters P""' #"E" assume NO dual role 47 IIRS_WORKE Ilours uorked since e.g All Personnel =9 In report The number of hours since reporting on shift o' ten has to be calculated by finding the difference between the shift start time D reporting for shift - 40 characters B-10
- ~ -
Field fed Title Definition and Codes. Definitions, and Suggestions for Things to Consider % hen Coding Etamples Locating Data 0 Field Size Limit 48 SIIIFT_ ROT Shift rotation e.g,4/4 230(M)700 in report identify shift rotation by number ofdays into the shift rotation when the event occurred, and the shift hours if known. For 0 characters example. 6/7 2300-0700 indicated that the individuals were on the 6th night of a 7 nicht back shift. 49 SillFT,DUR Standard shift duration e.g., All Personnel =10 in report Ilow long is the standard shift? Different personnel can be on 40 characters 50 IS_LIC Licensed e.g., RO SRO=Y In rcF Are individuals licensed' If the personnel are assigned as SRO er
, RO uhere they are required to hold licenses, then the licenses can be assumed. For other individuals, licenses should not be assumed unless specifically stated.
5I LIC_TYP Type oflicense e.g RO= RO. SRO=SRO In report List out type of license (SRO or RO) by individual 40 characters 52 LIC_ YRS Ilow long licensed In report in years 40 characters 53 YRS _ TRAIN Years of training / years e.g., SRO= 15 Any mention of This indicates w hether or not an individual 4 new to thejob or is training or trained and experienced. These data were usually collected for I' N #"C# experience is IIPS but not for AITs. 40 characters coded here B-l 1
m _ . . Field fed Title Definition and Codes, Definitions, and Suggestions for Things to Consider When Coding
. . Exampics Locating Data O Field Se.ze Limit 54 I_ACT Activity type at NONE-None ongoing In report This is the actis ities that personnel coded wcre imolved in at the g; . ;
IIOUS-Ilousekeeping time of the initiating esent. Therefore, each person coded has an RADI -Radiological activity-45 characters INRS-Incident response OPER-Operating SURV-Surveillance MODF-Modification TEST-Testing ENGR-Engineering MAIN-Maintenance Call--Calibration TRBL-Troubleshooting OTIIR-Other UNKN-Unknow n B-12
I i f# ~*%<& b,. I6 ~fb O. E I e .
- o, ' A k g l
+n '$
*+*** .
HUMAN PERFORMANCE EVENT DATABASE i NRC/AEOD Staff Presentation at the ACRS Human Factors Subcommittee Meeting January 21,1998 TWFN, North Bethesda, Maryland i Eugene A. Trager i ! (301) 415-6350 l Reactor Analysis Branch Safety Programs Division Office for Analysis and Evaluation of Operational Data i I
.. l O O O l DEVELOPMENT OF THE HUMAN PERFORMANCE EVENT DATABASE (HPED) *- 1980's: AEOD began studies of human performance during events e 1990: Began multidisciplinary team studies of human performance INEL provided technical assistance in studying the human factors On-site studies performed in accordance with a protocol Twenty event studies completed from 1990 to 1993 e 1992: Human performance database initiated Working group representatives from AEOD, NRR, and RES e 1993: Specified database content and authorized INEL- to construct database
- 1995: Delayed work because of budget cuts e 1996: Resumed work-and expanded scope 2
o n- o
~
~
Table 1. Current HPED Data Sources as of 01/09/98 AIT llT HPS STI LER ASPLER* Records Events i 1990 9 1 7 0 14 10 41 25 1991 15 1 6 0 16 5 43 26 1992 11 0 3 0 11 2 27 14 1993 17 0 4 2 20 3 46 27 1994 3 0 0 1 3 3 10 7 1995 2 0 0 4 1 5 12 11 1996 7 0 0 0 6 13 7 ; 1997 2 0 0 0 2 4 2 Total 66 2 20 7 73 28 196 119 i ASPLERs are ASP Database records with Conditional Core i Damage Probabilities greater than or equal to 1.0E-05 ' l ** ASPLERs for 1996-1997 are still being identified. ! -1 t
O mA,x<e O Table 2. Factors Contributing to Events in HPED Number of Record Type 73 28 66 2 20 7 LER ASPLER AIT llT HPS STI
?e"""**e*s" N System Design 32 15 42 1 9 1 Individual issues 30 9 28 0 13 3 Communications 22 4 25 1 14 7 !
Supervision, Comd & Control 13 3 27 2 16 7 ; Training 11 2 29 2 16 3 Human Factors Engineering 10 4 26 2 14 1 ! Workload 4 2 8 0 9 3 Work Environment 1 0 8 1 2 0 t i r
1Am
~
0 O t Table 28. Factors Contributing to Events in HPED Contributing Factors AIT %AIT llT %IIT HPS %HPS STI %STI LER %LER ASPLER ASPLER i Management & Org. 51 77.3 2 100 13 65 5 71.43- 27 37 % 13 46 % i Procedures 41 62.1 2 100 19 95 7 100 27 37 % 6 21 % System Design 42 63.6 1 50 9 45 1 14.29. 32 44 % 15 54 % i Individual issues 28 42.4 0 0 13 65 3 42.86 30 41 % 0 32% Communications 25 37.9 1 50 14 70 7 100 22 30 % 4 14 % ! Supervn, Comm & Ctrl 27 40.9 2 100 16 80 7 100 13 18% 3 11 % Training 29 43.9 2 100 16 80 3 42.86 11 15 % 2 7%
- Human Factors Eng. 26 39.4 2 100 14 70 1 14.29 10 14 % 4 14 %
Workload 8 12.1 0 0 9 45 3 42.86 4 5% 2 7% ! Work Environment 8 12.1 1 50 2 10 0 0 1 1% 0 0% i Total of Each Type 66 2 20 7 73 28 : i l i t
'5 I !
r
O Am O Table 2B. Factors Contributing to Events in HPED Contributing Factor %AIT %IIT %HPS %STI %LER %ASPLER Management & Org 77.3 100 65 71.4 37 46 Procedures 62.1 100 95 100 37 21 System Design- 63.6 50 45 14.3 44 54 Individual issues 42.4 0 65 42.9 41 32 Communications 37.9 50 70 100 30 14 Supervn, Comm & Ctrl 40.9 100 80 100 18 11 Training 43.9 100 80 42.9 15 7 Human Factors Eng 39.4 100 70 14.3 14 14 Workload 12.1 0 45 42.9 5 7 Work Environment 12.1 50 10 0 1 0
__._- . . _ .m.___.- m. _ _ _ . _._... _ . - - _ _ _. _ ___.- .- _ . _ - - . . _ . _ _ . . _ _ - _ _ _ _ - - - . . - _ - - - _ _ _ . _ . . . _ _ _ -
.t O- TABQ ^ .XLS
- O '
1 i i M&O AS AN EVENT CONTRIBUTOR SYSTEM DESIGN AS AN EVENT CONTRIBUTOR 1 I J I iOO M i AIT IIT HPS STI LER ASPLER AIT IIT HPS STI LER ASPLER ! PROCEDURES AS AN EVENT CONTRIBUTOR INDIVIDUALISSUES AS AN EVENTCONTRIBUTOR i 4. m { AIT stT HPS STI LER ASPLER AIT IIT HPS STI LER ASPLER 4 i I _ _ _
. _ _ _ _ _ _ _ _ _ _ _ _ - - _ _ _ _ l
~
i O O TABLE 2C.XLS O ! i ! i f 1 j i l COMMUNICATIONS AS AN EVENT CONTRIBUTOR r 100 [' j 90 < eo . 70 . . _
* , x:
}
, = 1 i n. e
'l i M i y l 10 - 0 - - - ' ' I AtT . hr HPS STI LER ASPLER i SUPERVISION, COMMAND AND CTRL AS A CONTRIBUTOR I I i ' i 100 l )
= t s
! 60 l I 70 u= l
- e so I
E e. l N i l 20 i i 10 ! O- - ' '- ' - i ' AIT IIT HPS STI LER ASPLER 4 i 8 i
,,t !!!!i!ii!!! !lI ll i i i !i O
s _ l a u _ d i _ ev ) s N ci nd s O ain e I n T A md r n e r _ o a a M f r _ e pws w R a _ O n F N n er o ac i I T my t a . N h ub i t u _ E e s V nc n n , . E on a i o i o g d t a t n nm a E o e O C ar o g r u i n i t a ( . N s . t f n g dr e _ A nr i i z e o vpe r f i e n o nd u _ M e o C a glos as R ed n C i _ O F ee ht nc i g d d n rkl Or e R nt n E n a on E nu e ea dWo n n . P ol f smn r ong i n o a/s t s s r _ N nn tcros i t ne e A oi a n erP Faies i t ioc t M ta a vDre nimSl n uguie/a s U mt h nE md n vg euid u _ H r s o e ak mr t eeimrn cimpnii e a gv n us u o syr oa f _ r ou at d a i si s HWSPTCSMFI n i n/ s ar t o nt O oc Cf a- eeeeeeeeee
;! !!l :! <!!lj jl { - i ;j: i! !! 1 ~
L LO.- O O~
- HUMAN PERFORMANCE EVENT INFORMATION (CONT.)
Database Structure has 54 Fields to store human performance and other general , j event information, including, i
- e Event Summary and Classification
! e Time and Date i e Operating Mode and Power Level i l e Preceding or Subsequent Subevents i e Numbers, Titles, and Types of Personnel i !
- Whether Licensed and Number of Years i
- Personnel Trasnmg
! e Shift Rotation and Duration l e Activity Type and Time on Shift ! i l l te i i
! O O O~
[: l i PLANS TO DEVELOP THE HPED i l
. Distribute a report and request comments on the current version 4 ,
. Update the HPED database with recent event information from l
AIT and other reports-l
. Include data from other sources, including other Accident l Sequence Precursor (ASP) events with high CCDPs, and reports i l of Regionalinspections performed using Human Performance j investigation Process (HPIP)
! . Perform studies of human performance using the HPED data !
- Examine risk significance of events and develop information for
- use in studies of human reliability / risk assessment i
! . Assess database adequacy for modeling human reliability
- assessment techniques-
- i I fl
Q, o o
) UnitedStates
~
..... Nuclear Regulatory Commission Results of a Nuclear Power Plant Application of a New Technique for Human Event Analysis (ATHEANA) presented by Mark Cunningham Probabilistic Risk Analysis Branch Office of Nuclear Regulatory Research 301-415-6189 to the Advisory Committee on Reactor Safety Human Factors Subcommittee January 21,1998
, - - t O O O
~
l Adapted from l- ! Nuclear Power Plant Application of a New ! Technique for Human Event Analysis (ATHEANA) 1 l J. Forester', D. Whitehead', D. Bley2, A. Kolaczkowski , S. Cooper , 5 ! J. Wreathall', K. Kiper', J. Dalton 5, S. Kessinger 5, E. Spader , l C. Thompson5 and A. Ramey-Smith' i. i i
'Sandia National Laboratones l Buttonwood Consulting, Inc.
- ' Science Applications International Corporation i ' John Wreathall & Co.
5 ) Seabrook Nuclear Station
- ' Office of Nuclear Regulatory Research, USNRC 1
, l 4 I _r_ , , , .. _ m _
!- O O O ATHEANA Development i [ e. Development of ATHEANA-l
- Sandia National Laboratories (SNL), Brookhaven National '
i Laboratory (BNL), and their subcontractors. l l e Several documents issued: NUREGICR-6093, BNL&SNL, NUREGICR-6265, BNL, NUREGICR-6350, BNL l Two documents currently in draft form: e I - Frame-of-Reference (FOR) manual - technical basis document (BNL) i l
- Implementation Guideline - step-by-s+ep guidance for i applying the method (SNL)
I 3 4
I O O O
- FOR Manual and implementation Guideline ;
l e Provide information needed to identify, characterize, quantify, l and integrate into probabilistic risk assessment (PRA) models: i - Human Failure Events (HFEs) - Basic event modeled in l PRA - represents failure of a function, system, or ~ l component that is the result of one or more unsafe , actions.
- Unsafe Actions (UAs) - Actions inappropriately taken, or i
) j not taken when needed, by plant personnel that result in a ! degraded plant safety condition.
- Error-Forcing Context (EFC) - Situation that arises when particular combinations of performance-shaping factors
! (PSFs) and plant conditions create an environment in which unsafe actions are more likely to occur. 1 i
- 4
, - . . . , , . , , . - . . , ,..---.._.e
O O O ATHEANA Demonstration Goals. e Test ATHEANA process as described in FOR manual and implementation Guideline i e Test a training package developed for the method l e Test hypothesis that plant operators and trainers have significant insight into the EFCs that can make UAs more likely e Identify ways to improve the method and its documentation. I; 5 i t
O O o Success Criteria far ATHEANA Msthod and Demonstration e FOR manual and IG " work"
- Documentation is understandable -
- Process is usable
- Training is effective
- Motivates team
- Facilitates use of the FOR manual and IG
- Enables leader to direct team o Results in plant team applying useful retrospective analysis e Process identifies demanding scenarios involving errors of commission
- Plant operators judge the scenarios to be " demanding"
- Plant identifies and implements fixes for some scenarios
- Plant believes that ATHEANA can or will identify important problems e Users identify improvements in ATHEANA tools and processes 6
. Seabrook Nuclear Statinn Demonatration Praiect e Human Reliability Analysis (HRA) team leader - PRA expert, experience in applied HRA (not ATHEANA) e Plant analysis support (HRA team)
- Two individuals from PRA staff ~
- Two from training staff e Consulting and documentation support on application of method provided by the ATHEANA development team.
- Scheduled for 20-week period, beginning with 3-day training session in July 1997 1
7 ~- .
~ - .. - -- -
O O O l j An. nroach Used in Demonstration e Step 1: Set priorities among initiating events. Medium loss-of-coolant accident (MLOCA) , i
- Loss-of-offsite power (LOSP) - Station Blackout-(SBO)
- Transient (followed by anticipated transient without scram (ATWS) under special conditions) e Step 2: Using FOR manual and IG, identify possible HFEs and associated UAs for functions identified in scenarios. - Cornpleted for MLOCA and LOSP-SBO MLOCA (3 critical functions) LOSP - SBO (3 critical functions)
- Makeup - 6 HFEs identified - Heat Removal- 5 HFEs identified
- Heat removal- 6 HFEs identified - Support (Diesel generators (DGs)
- Long-term heat removal- 5 HFEs and/or cooling to DGs)- 5 HFEs
- Depressurization (manual)- 6 HFEs 8
l
= = - . __.
O O. O Approach Used in Demonstration (Continued) e Step 3: Identify / derive potential EFCs that could lead to identified UAs. e Step 4: C onduct simulator exercises to evaluate impact of reasonable EFCs on UAs (includes information from debriefing of operators) e Step 5: Quantification of EFCs and HFEs l i e Step 6: Document demonstration and submit method and results of demonstration for peer review - Planned for April 1998 i 9
O O O \ MLOCA Scenario ; e MLOCA initiating event in and of itself may pose a problem to operators
- Do not generally receive training on this size LOCA
- less familiar with postulated plant conditions and impact on stepping through the emergency response procedures.
e MLOCA scenario consists of the IE, two HFEs (with one specific UA analyzed for each HFE), and various other
" failures" that provide EFC.
e HFE0 -Inappropriate termination of makeup
- UA analyzed for HFE1 is " Operators stop pump."
- EFC - Failure of wide-range reactor coolant system (RCS) pressure -+ Operators believe that RCS pressure is higher that it actually is.
10
. MLOCA Scenario (Continusd) e HFE2 -inappropriate depletion of resources
- UA analyzed for HFE2 is " Operators operate pump outside design parameters." l
- EFC - failure to receive an " Empty" reactor water storage tank alarm -+ May delay operator action long
.enough for the pumps to be damaged due to inadequate suction head requirements e Other " failures" modeled that will not necessarily lead to core damage but which demand operators' cognitive resources:
- A small (3 gpm) steam-generator tuba leak
- A restorable turbine-driven emergency feedwater pump trip on overspeed
- A diesel generator failure that is recoverable.
1 11
O j - O O. Evaluation of Demonstration Against Success Critaria e Do FOR manual & Implementation Guideline (IG) work? Self-evaluation: ;
- - Improvements needed in some of the guidance and support ;
information contained in the FOR manual and IG
- Improvements needed in process documentation tables
- Basic search process for HFEs, UAs, and EFCs working well.
l EFCs being identified are reasonable and in principle quantifiable. l e Was the training effective? Utility evaluation:
- Positive about overall training package
- Needed improvements: More extensive initial overview directed at plant management, brief review of PRA, detailed training for later steps just prior to beginning of step, e.g.,
quantification. I 12 i
Evaluation of Demonstration Against Success Criteria (Continued) Did the process identify significant scena Ios? e Operators judge them as cognitively demanding?
- Operators on the team thought so e Did plant changes result?
- Procedure changes are being considered and it is clear that the method will allow plant personnel to consider their procedures from a useful new perspective.
e Plant believes ATHEANA can identify important scenarios?
- Yes i
13
. Evaluation of Demonstration Against Success Criteria '
(ContinuedD , Did the users suggest improvements in ATHEANA process i and tools? e Tables in FOR manual used to guide identification of EFCs could be tied together in a clearer and more systematic way. l e Some aspects of search process for EFCs in the IG could be ; improved by more directly tieing the description of the process to the use of the tables in the FOR manual. i 14
. Plans for Future i
= Document demonstration as part of NUREG-1624
- Issue NUREG-1624 for comment in Spring 1998.
- Sponsor peer review of NUREG-1624 in Spring 1998
- Sponsor additional applications of ATHEANA
= Link ATHEANA with M&O effort at INEEL
- Continue participation in international efforts of "second generation" HRA techniques
- Continue analysis of operating experience to support model refinement and quantification
- Identify approaches for translating ATHEANA into a computer-based analysis tool 15
.__-______}}