Text

Intervenor Exhibit I-SC-52,consisting of Undated Article Entitled, Critical Incident Technique, from Psychological Bulletin,Vol 51,Number 4
	ML20238A428
Person / Time
Site:	Shoreham File:Long Island Lighting Company icon.png
Issue date:	04/30/1987
From:	Flanagan J; AMERICAN INSTITUTE FOR RESEARCH IN THE BEHAVIORAL, PITTSBURGH, UNIV. OF, PITTSBURGH, PA
To:	;
References
	OL-5-I-SC-052, OL-5-I-SC-52, NUDOCS 8708310074
	Download: ML20238A428 (30)
	v • d • e

{{#Wiki_filter:M\\bi* 3.JA 3 22 b-5 "MM C.;Wsh A E fNTi .33yJ0jld"P987 TiOx "#128;%7hk f2 ikMk g6% M Ri$' 3 j-sc-5ZJ dEM f% .c htd TNE CRITICAtj INCIDENTkEC $fh 27 'NT ., ( (' UQ{@;.. JOHN C. FLANAGAhUs A.?g $b $merican Imtitutefor Ruearch and UniniRQP Y g&..' s a w .m. gg w. a.~. 7 Q 4Du'ing the 'past. ten years the leave :little!@' ppy ij:ggitQ r writer and variotai collaborators have effects. ..psg .;;.;g, been engaged in developing,an.d uti-CertainlyfiQ.i MM. s lizing a method that has been named basic appra ~ eli $e.ki. the.!.' critical incident technique.N It - techniqueM7".m ish.? .m -.c

e..

isp.ie purpose of this article. to de. new about it.9 gm edr.i6e the development of this me'thi ing observeiidEst pTE r odoldy, its-fundamental. principles, centuries 0 ThWf -- -Miof.i

r.

great writerM,ii and its presen< status. In addition, indica F. the findings of a considerable number that they weie Ibo

" 5tdennefs.

of studies making use of the critical fellow men. Sdih$*oWt$$iifih*it8dSQ.I must have relied An7dehiindnii6tes$[f incident technique will be briefly re-viewed and certain possible further made from their observatns20thers uses of the technique will be indi-have had un.usiial*abilit.ie.s tod may cated. reconstruct memory -images irnvrvidy .m... m The criticalincident technique con-detail. Some may have even made a sists of a set of procedures for col-series of relatively systematic obser-lecting direct observations of human vations on many instances of a par-behavior in such a way as to facili-ticular type of behavior. Perhaps_ tate their potential usefulness in solv ' what is most conspicuously needed to ing practical problems and develop-supp ment these activities is a set ing broad' psychological principles. of pr cedures for analyzing and syn-- The critical liSEident technique out-thesi ing such observations into a lines procedtik for collecting ob-number of relationships that can be served incidenti having special signifi-tested by making additional observa-cance and meeting systematically tions under more carefully controlled Se defined criteria. conditions. i T. 1 '8"f fly an incident is meant any ob. I Eb servable human activity that is suf-BAcxcaotrND AND"E'Antv% ". So ficiently complete in itself to permit DEVELOPMEKTI.g '{ g inferences and predictions to be made-can be traced bacg;reschiprbNduNs.7 The roots of the p 88 a' bout the person perforrning the act. difestlyjj$the% i:s* To be critical, an incident must occur studies of Sir Frane@Galton'nearly - $x in a situation where the purpose or 70 years ago,)anditslater de~ve~ lop. .n, EEo intent of the act seems fairly clear to ments such aattimemmpling studies. the observer and where its conse. of recreationakact6tiiegcontrolled"; quen,ces are sufficiently definite to observation taiki('Maiins2dotilVec. - - - ~ ~ ~. ~ u

L~;a.J u. __ __ ~$ G 047t 05.o'r m dy Gk#~.hd.iti j m DeeW k is e..mv w 3kullom % dear

mma _ /

swt mum __ a*w JitJtetto __ leienaam _._/ -.JIAft l _N_'O~ caert oer -. teatracter_ =_ -o, _ m tm0J A - Reporter _ - 4W I 1' l I - ~ _ _ _

l -n~ .cw.~ ...C Qhy a ords. The crit' incidant techni,qua Althod ~m the prepaia 5 as such, howefuMi%n]best be re-reports ush greater emphasiaiasi, ~ garded as an oi 'of studies in given t determining tiie precise facts the 'ation 19 7 Program of in the c9se, it was apparent that in l nited States Army Air Forces many instances the~ official reports ; in World War 11. The Aviation Psy. did notfrovide'a complete record of I l chology Program was established in all the important events. Even with I the summer of 1941'to develop pro-these liinitiations, i the information j ~ redures for the delection and classifi-given was found to be of considerable ration of aircrews. value, and the systematic tabulations that were pre $ red hoyided the basis g One o( the first studiss740)' carried 3 out in this program was the analysis for a series of recom.mendations that of the specific reasons for failure in resulted in impoityt changes,inJir ( learning (to fly that were reported for Force selecti.onnatidF 'i.n ~ M4M.. m 1,000 p4 tot candidatap ch.mmated duresA,,.. jam >,-

<o

.~. . &g 1.. e from flight trammg schools in the In the:su~mmer' 'oF&.. hasic source used in thisanalysis wan'.lem ~ ofhoiin ~. pro 'I nummer a,nd early fall of 1941. The, studiesf74 fwd ' ,.n [ht ~ t f 4 htes ip Fore } the prodedings of the elimination United S boards. } In these proceedings the These represen.# ve-rat large-se M. tet w.e% plot instruc torn and cherk pilots avstematic efiorta to.rgr;specificm".. ~ reported their reamma for eliminating idi ide n t s[ fo f e ffec tikpsglyff"Mt'ive behavior wit h respect to a designated t the particular pilot. It wam found that many of the reasons given were activit y. The instructions asksd the clich6s ami sterent ypes nurb as " lack _..rombat veterans to port incidents i of inherent flying abilit y" and "in-observed by t hem thatTnvolved be-adequate menne of nuntentation," or havior w hic h was essweially helpful generaheations sm h as "unsmtable or maelequate in acrooiplishing the temperament," "ixxir judbnent /' or: assmned mission. The sta temen.t ~ "innutlicient prog rens.", f lowever, timshed with the request. " Describe along with these a numlwr of qweitic the ottii eri artion. What did he do?" observations of;.particular lwhaviors Several thousand incidents were col-were reimrtedsyflhil: studs provided lerted in this way and analyzed to relatively

objective and the basis for timresearoh program on provide a

selec ting pilots. ' Although it was factual detinition of effective combat found very useful, it also mdicated leadership. The resulting set of de-very clearly the need for Iwtrer pro-scriptive categories was called the redures for obtaining a representative " critical requirements" of combpt 3 ample of fact ual incidents regarding leadership. ^ Another studv (74Fconductedlin pilot performance. A second study'(13), which 'empha7 ~thEAviation Pskhologyy'if6paiDne, sized the import 3nce 'of factual re-; volved s# surveys &disBiiisiiftstiW porta on performance made by com. while Aynig.' LDis5deYtitidii%his" 1This'stixfy was planndhy'IG@M. Fites,' ,f% ji jf3 petent observers, was carried out in the winter of 1943-1944 in the 8th,. 9th,12th, and 15th Air Forces. This Jr! wWah contributed to tlw p@lously; mentioned USAAF'ntudies and ptsanad'and-study collected the reasons for the carried'6ut.the interview 3tisf9'w,dh'iiikts'de. failures of bombing missions.as re- ,,cribed.below on the designe ported in the Group Minsion Reports. controls.,and arra.ngements F.R.instrunwatea 6 .a. a n. i:

N gas defin to iNIude any ex-i cip I N i

DrT erience denoting unWinty as to ceduyes should be the determinist 6dgchical.

p .w.... reqNrements. These requirements include oneaspatsp i d.ition m relation to thod which have been demonstrated to have ( the verticab i'this study pdots mad 4 the digerence between success and fait. returning frfniPcombat were asked ure ih carrying out "to think of'some occasion during job a'ssigned in a s,an important part of the ignificant number of in. ' lombat flying in which you personal-stanm. Too often, statement 3 regarding job ly' experienced feelings of acute dis-requirements are merely hats of all the destra-ble traits of human beings. These'are prac. orientation or strong vertigo." They ticatty no help in selecting, classifying, or were then asked to describe what they training individuals for specific joba! To ob - "saw, heard, or felt that brought on tain' valid information regarding the truly the experience.,,,I,his study led to a critical requirements for success m a specific number of recommendations regard-as.ignment, procedures wire developed in the Aviation Psychology Program for making sye. ing changes in cockpit and instru-tematic analyses of caune of good and poor - ment panel des perfwmance. order to overco[i n and in training in ce%Qyjp y Essentially,.the1 pro, dure 7was. tonobtam. me 'and prevent vertt-- first hand' reports l or' reports (p7if tifei-tiVB! records, of satisfactory.and uniiik'ac(tory 1 go.while flymg. 4 in a project carried out in the Avia-cution of the task a'sigried gg' "" " Idil" *

N"fuj%The cooper a

tion Psychology Prokram in 1946, individual describs' ?iit tiEliYwhich J. d Fitta and Jones (12) collected descrip- !ngby.specifi gr ported caunes. .; gy s. tions of specific experiences from Thin procedure was found.:-gdg.o;;. Wry ~elfeEt ve ini pilots in taking oti, ti>ing on instru-obtaining information-fronti fidiWd' alTcidi-u ments, landing, using controls, and cerning their own errors, from,iors, from su-sulmrdinates using instruments. These interviews conmning errors of their super with pilots were electrically recorded, pervisorn with respect to their subordmates,. ~l hey provided many factual mci-and aim from participants with respect to em participants (13, pp. 273-274). dents that were used as a basis for planning research on the design of DF .onom . STUDIES AT instruments and (ontrols and the THE AMERICAN INSTITUTE arrangement of t hese wit hin the cock-von RESEAHCH ,l pit. At the close of World War 11 some in addition to the collection of of t,he psychologists who had spar-specilic incidents and the formulation ticipated in the USAAF Aviation ' of critical reighements, as outlined Psychology Program established the above, the susmary volume (13) for American Institute for Research, a i tlie A via tion"l sycholog y l'rogram nongyfofit scientific and educational Research Reports contained a dis-organization. The aim of this organ-cussion of the theoretical basis of ization is the systematic study of ~ procedures for bbtaining the critical human behavior through a coordi-requirements of a particular activity. nated program of scientific research Perhaps the best method of describ-that follows the same general princi-mg the status of these procedures at pies developed in the Aviatiort. Psy-the close of the war is to quote from, chology Program.. It was in connec-the discussion in this summary vol-tion with the. first two studies'under-ume,.which was writ ten in the late - taken by the Institute in the spring spring of.1946. In the section on of 1947 that, the critical incident techniques for defining job require-technique ~was more formally de-ments, the present author wrote as-veloped and'givertits present name. follows: .TheseTatudifisiwne huural exten. ~. ..;.w:, . n.4ws: a n 2. -W :~ * 'ML k

7 tyM E q y torieo we$e [n_'terviewed. Mese sci ' sions%f the previgus research in the entists r%rted more titan 2,500 i Aviatiqn Psycho!qy, Program. The critical iricidents. The criticat' be- ". --etudf" reported bg&eston.(52) dealt with the deter'minalT6d 00$e critical ' haviors'wdie used to formulate in r tively a set of 36 categories, which requirements fortWwork of an officer constitutes the critical requirements in the United States, Air Force. In for the effective performance df the this study, mang of the procedural duties of research personnel in the subjected to problems; were first ;d 'evalnatibn. physical sciences. This initial sthdy systematih.'try6iit an Six hundted and forty officers were provided the.. basis for the develop-ment of selection tests, proficiency e interview)d,'and a total of 3,029 ' rit-measures, and procedures for evaluat-c ical incidpnts were 'obtained. This ing both jdb' performance and the ledjo thejdevelopment of a set of 58 research report. (< w, _ critical requirements classified into six major ar.cas.. The scand. study",..,. Anog projec,t.f.undertakengK,the m. reported by Gordon~ (27,' 28), was American. Inditift for'ResEarTFin the spring' of 1948 provided faldable' 4 ' caNied oqt to determine the critical requirements of a commercial airp experience with.the,criticalQncident.p ~ techniquet.This study,[ reported by C. pilot. In this study, severald erent sources were used to establish :the Nagay-(48),Lwa's done for thiqCivil f-critical requirements of therairline Aeronautics Administration 5under i the spcmsorship of the Commiffes on? E pilot. These included training rec-Aviation Psychology of the National i ords, flight check records including Research Council. It was concerned the specific comments of checktifats'; critical pilot behaviors reported in with the air route traffic controller's accident records, and critical inci-job. One of the innovations in this dents reported anonymously. in: in-study waithe use of personnel of the terviews by the pilot s themselves. Civil Aeronautics Administration From this study, 733 critical pilot who had no previous psychological behaviors were classined into 24 criti-training in collecting critical inci.: cal requirements of the airline pilot's dents by means of personal inter-views. In previous studies all such job. These were used to develop selec- . tion tests to me.asure the aptitudes interviewing had been conducted by .and other personality characteristics psychologists with extensive training found critical frm..succese in the job. in such procedures. In this study, aeronautical specialists from each of They also provided the basic data the seven regions conducted the inter-for the formulation of an objective views in their regions after.a brief flight check to deterrnine the eligi-training period. An interesting find-bility of applicants for the airline' ing from this study was the clear re-transport rating. The third application of thecritical tiection of seasonal variations in. fly-incidgnt technique by the staff.of the ing conditions in the types of.inci-American Institute for Research.was dents reported.. The study Falso demonstrated the selective recall'of.. in obtaining the critical reqdenients. dramatic or other spicial types.of for research personnel on a project This bias was especially incidents. sponsored by the Psychological Sci-noticeable in the incidents reported. ences Division of the Office of Naval Research. In this study (20), about several. months af ter their occur-500 scientists in 20 research Labora-resce. The incidents obtained in' this ) . v.,,. a a y.g, y,.p - __ J ^ ^

study were used to develop ~ proce-only at the end of the two-week peri-dures for pyaluating the proficiencyof od appeared to have forgotten 80 air routeig:raffic controllers and also per cent of the' incidents observed. for develo~ ping a battery of selection Although it is poss~ible that the find-tests for this type of personnel. ings may be partially attributed to in the spring of 1949 the American daily, fact that the foremen making the Institute for Research undertook a records actually observed more study to determine lhe critical job critical incidents because of the daily ~ requirements for the hourly wage reminder at the time of recording, it employees in the Delco-Remy Divi-is clear that much better results can sion of the General Motors Corpora-be expected when daily recording is tion. This study, reported by Miller used. and Flanagan (46), was the first Another analysis based on data application of these techniqueii in an collected at the~ Delco-Rerr.y Division industrial situation. Forenien who compared the number of hitical in - were members of a committee ap-cidents of various types obtained. pointed to develop employee evalua-from interviews with those re4rded' tion procedures collected 2,500 criti-daily by the foremen on the pJ fform-cal incidents in interviews with the ance record. 4 'Although ther@were other foremen in the plants. On the some differences in 'the relatin fre-basis of these data a form was pre-quencies for 4pecific~categorics, the pared for collecting incidents on.a general pattems appeared to be quite day-to-day basis as a continuous similar. These results suggest that record of job performam e. critical incidents obtained from in-l ~ sing this 6,rm. t he Performance terviews.can be relied on to provide Record for Iburiv Wage Employees a relatively accurate argount of job (21). three groups of foremen kept performance if suitable precautions records on the performance of their are taken to prevent sysjematic bias. employees for a two-week period. A in addition to the developni,ent of group of 24 foremen recorded inci-the performance record described dents daily, another group of 24 above. the critical incidents collected foremen reported incidents at the in this study were used as the basis end of each week; and a third group for co)structing selection tests cover. containing the same number of fore-ing h6th aptitude (18) and attitude men reported incidents only at the (2) factors. end of the two week period. The three groups of foremen represented S m utu (.ri:RIEp OUT AT TIIE comparable conditions of work and UNIVERSITY OF PITTSBURGH su pervision. The foremen reporting A substantial number of studies daily reported 315 critical incidents: have been carried out in the depart-the foremen reporting ' weekly,155 ment of psychology at the _ University incidents; and the foremen reporting of Pittsburgh by studentsiworking only once at the end of two weeks for advanced degrees under the au-reported 63 incidents. Thus, foremen thor's direction. Mostof thesestudies who reported only at the end of the had as their objective the determina-week had forgotten approximately tion of the critical requirements for one half of the incident's they would a specific occupational group or ac-have reported under a daily reporting tivity. Many of them also included plan. The foremen who reported contributions to technique. ~In 1949

7-- 3 f) 33g. 72" .w 'T' j-R

f n -e.g.g JOHN C. FLA s

~ 4 Wagner (66) com' leted a dissertation tai ~ ' ' mfquestioins.inti ".c on.the critical" ments for den-elicitf e definitely effektive "^ y this lii$ incidents' eRectidbshavi6rsNSompxiff$6iF E $: l p ~ of thess. questions are: i patients, dentist ',". sthree sources:

1. Thinkof a time when a forenhn has done i

were obtained and dental school instructors. The' incidents were clas-something that you felt shouWbe(encouraged i beeme it seemed to be in.your o n an ex. i l sified intorfour main aspects of the ample of. good foremanship. Rective-- I dentist,s,c job: j(al d. demonstrating slight deviation from norm.) technical proficiency; (h) handi, g

2. Think of a tinte when a ft an did F j

m patient relationships; (c) accepting ,omething that -you thought was isot up M. professinhal resp <mnibility; and (d) to par. jinedective-slight deviation, imm acceptind ;wrsonal responnihilit'y, As ""',":k;~nk of a tidM a fom i$ 1 might bd expected, the patienta,did vour opinion, shown definitely good foremanh incidents for demon'st' 7r.iting technical e *~np@erior foreminNE5ec rt an.large a proportion of .hi not. repo proficiens or acce'pting, profemional datiaa ImH} the nwmHg ,a

4. Think of a'tigf iLwhenta

?iss - responsibility an divp the other two ,o,t of diinn.which%pj!iii6. Enh -.lt; r opinisiDhT>w groups, aind the instructors reported wgijisted- 't that the mast i@ndtisWKd 'T F' l only a relatively. sm.dl proportion of- ',M*EiiIik'< M Lf W+4 ~ t heir incidents in the area of handhn$',lh i m q, c yr -4Wwemme patient relat.iotuihips. v. On the banin of the findings from

fhe frequencies,3f inEidiint' slo nd.

t hin et udy, a battery of sleMiWi'5W 'i'liiiFd'Th~earh of the 40 categor'iN.- ; was developed for une by the I m-mto which the effective behaviors ilassified were compared' for? I vermit y of PitIaburgh N hool of I)en-were t int rv. A ntimiwr of prhbrigticy 1catn 'the questeonn requesting slight'und 4[ lor measuring abi4it v. with respect substantikljdeviations from the norm, j to certam of the i ritwal requirements and the sigiiiheam e of the difierem en j w a~ tes'ed by means of the rhi' j were aino developed using thew re. sulin an a banin. square test. Two of the differences i ~ were nigmfwan t at the 1 per cent l Another di+crtation rompleted in 1949 wan Finkie's.(11) study of the level and one at ihe 5 iwr cent level. critical requi 7ts of induntrial ( 'om parison3 of the frequencies.in foremen. Thii ,'y wan ronducted carh of the 40 categories for ineffer-io 'the !?, ant Pi r'gh plant of the tive meidents failed to reveal any West inghouse l',ler t tii ( 'oi po ra l so n. chi iwpiarca signihrant at either the. 5 wr cent or the 1 gwr cent level t ('ritiral incidents were obtained f rom i foremen, general foremen, and stal.I The questions involving only. a person nel. A number of points gwr. slight deviation f rom the norm're-taining to technique were studied. suited in - more effective incidents - One finding was in.r.efe,renrt: to the-co, ncer.n,ed with gaining the..res.pe.ct. i eff#i' on,the;t, degree'of importhnre. and loya.lty7of thejo ypes h( ncidents ob. ~ .m i o tained of' the in more incidentst that #mvolved" or exceptionainete net up as a criterion making, encouraging, anh acceptlk*i for regnrting or ignoring incidents. suggestions. They produced signiti. The incidents obtained from.the use cantly fewer. incidents regardingJtt of questionn that asked for incidents ting men to' jobs.a.The small nuinbech, on.ly slightly removed from the norm of significant, differences. m',three.t only __ w c s #.e m ~ ~..... e m .m

"i .m , y ny., pg _ g;.[. 4 types of incidents obtai,ned are nothqWdialokel(72) datErmlin w.g. gpygvarigtipsiaWidapregafi)mentsyformlif Esc ' $im w6rd uestioiis 'c"dnipfi6ncy heads. ' A principal feiluie

f..>.rable't'd Wn'dthve/It seems

'of his study related toothe comt ari-i Plikelyfth. IINileast partiallF.gson of 'two types of dency heads--- 3 adue to the fact that the persons-l.it@4belleVedMhat.;the critica minagers and gener agents. Ittwas

$- terviewed ' report bnly incitlents tha't

represent a' fairly'lshbst$iiial'devidf j{ane 6f/ agency l head.,might El _p$tiinid%r tion from the' norm regardless rovide'a. iff ent pattern than that the precise. w(rding of the questio@6 Tsis'wAs ns corifirmed by 6

'e other. This bypothe. asked. .E e nalysis Another comp (trigon made,in thif%the o sinell incidentsr AThe pat-stud related to the fas.% ' i,tica,l... mg or anu effe:n'tife..o, influence k-eats " m.che%w.e,re ...mw

x. x m -

.a ww ~ c raan me d. c ve - ~ ~ quit sim t y

. incident first. About: 10 per t ent m, ore 3i.f, v. of'a.limin'iitriitorii

~ 2.. .e u%p %,.i m 2.

meidents were ob...

lets request,ng effe...amed fr.om..b,o. t o. Epam u p'. x ctive irleidents: y .e i 1 a; utre ts 7 ? . t hean' from bo,oklets. reque.stm; 3gri 1pr.a Rto.

p,sy. -

~' g r .u c - effect.ive incidents b.rs t'. This diffprW.,. rmco. v. rpaps3 ..e e ng-iently small so. hat. generkh, g...a..w.w 13 stu ,,f ..v. ..w ence was suffic. t taycg it could reasonably be attribu t.o+ the* exi.stencel?o(Msh_b Wtid ffe's ,; a..~ences beI wove >*he.m., spatjerns o cr,i g. w m .m .m - chance sampb.ng fluetuatmna. ,' cal i The incidents collectep' in ihis iniildnis regidfied by"fttiddritii%nd study were used, along,with other faculty. The'facultyreported asignif- .datat.in the prepamtimpoig.Performe. i% larger percent @e ofr ffe'ctive 4 s e ance Record for 1'oremen and Super-h havmes, in the following areas: 'O giving' demon,,strations visors (23). or experi-s A study was conducted by Nevinls ments, using discussion group tech-(50) on the critical requirements of niques, encouraging and ascertaining Imkkeepers in sales compnnies.J She _ students' ideas and opinions. rollected incident's relating ~ to appli-The stmlents, on the other hand, can t s for bookkeeping, positions as cont 'buted a larger percentage of well as forjmployees working in'this behi iors in the following areas: re-capacity. [jp vie ng examinations, distributing Fur thegec. tion of the informa-grades, and explaining grades; using ti,on.abopt the practicing tmkkeep-lecture. aids such as. d. rawings, charts. w ers, a modific~atmrt.in the crit.ical..,emei-movies, models, and apparatus; usi.ng . dent technique was made. This was project techniques; giving test ques- ' done because, in 'the Imkkeeping tions on assigned 'm..aterial; helping t .professiun, success and failure..... studen.... ~fter class a d ;d..uring clags ts a n ar3 .usually defipMnhermy of persispnt. rec %t e. manner of thgins'trjagtor. h -behavioral patternsJ Occasirmal muk'EgAThy.facyityl report $lglarg, er: peri takes in addirig aird balancing ac.a-repeatetFj.cen..tageoe.f.in.~effecti9e.,,b.d.eh M -....e....n.. counts are expected, but rmng. mamtaining or er;.The...;in-errors are considered serious. Instea'd 74Kastive behniors,lE by stud thst were reported of the single. incident, therefore( irI%gerkrcerda many of the items included reprep vol i - sente i either a pattern of behaviorgosiggh% gt of;'thypurse,lusing effec-urHun{degnpresis@n, de' ling a series of simila'r behaviors. ' pamam tiVe okes a w

u TNYD ' M t. h i

W * ".j. : i

~ ~ ' ,,.j.,.e.%.g A..,- <. j g_ Because of the indsfinite'natur'E$h3 M with' students' qyectis(s, pointing out fallaciesi#glind summa-

the co'nceptfithics felt"thut"l'tihEckN b

idiples, using should be mado on the contributor;'s " l rizing basic (ActY kverbal diag-understanding & his task. Twenty of l project techniq nostic teaching tWiiques, achieve-the participating ^ persons were asked

ment testing students on assigned to summarize briefly their interpreta-material,.otyjective type achievement tion of what they had.-beeh. asked to do. This appeared to tie very useful ' q(

~ = testing, vai'ng humor. in developing the phrasing of the This is k good illustration of the problem ofLthe competence of various questions so that they were uniformly types of aidailable observers to evalu-interpreted by the various bbservers. The author ei the,iet@dpyclassified .~ aec the contribution tf the general mat yof.ag g.., alljhe.,in(t>on sy's aim of thqyr,tivyy gf,.pyc,ific ac, tion. classifica Mn d Exammarxin of the reports from stu-denta indicated a nomewhat limited preliminary cat za ?- ~ spherp of lompetence. Apparently by six of.the' " N

i. -

one of the { principal reasons for this sifi4ation' wafs'd 1" h" M 3 was the latk of perspective on the atrists for rg sk. f part of the urudents and their inabil-tolndicate-ategor y> ty J %@Neen they wereYillingMa ity. to keep the general airn of the of immaturity 'hithnC Y j instructor dearly in mind because of defined.in, Jan~offibid "nYeM'. $P its divergence from their own im-mediate aims. In many cases, thin More than half thelcaIki6 ties'were C' latter aim seemed to be direc t ed accepted by at least' )3Nif; thie 14 ~ ~ I toward achieviilg a natisfactofy grade j udges. seedlImpse vias' rejected by more" than 50 per'cest?of the judges. It was in t he i ocrac. F.ilbert (7) developed a functional felt then that the system kas accept. l de.scription of emotional immaturity. able. The c contributors of critical incidents This study illustrates the applica-included psychiat rists, psychologists, tion of the criticalincident technique, ps ychia t ric social workern. occupa-to the study of personality. It is tional therapists, nurses, and corps-believed that this study provides an men from a mjlitary hospital, plus excellent example of the possibilities 13 psychologikin nonmihiary oc. for developing more specific behavi-ganizat ions. TWsubjects of the in-oral descriptions. cidents were primardy patienta under ifolley (24) reported on the critical, requirements of sales clerks in depart-pavchiatric care. The behaviors were The contributors were given a form ment kores. abstracted from narrative records of.. g that oriented them to the concept individual shopping incidents writtent. imma t urit y" by sug. "emotiona) it was revealed generally.by. shoppers who wEti' relatively in gesting that exp'erienced in chleatidFsales per "> by childlike modes of behavior. The questions used to elicit incidents sonnel. For various reasons, including Have you recently thought of the competence-ofdhe' observers, were : their training, and theElimited point ~ someone as being emotionally im-(regardless~ of' diagnosis)?., of $ view,. the resultingddescripticria mature must be regaqed ai4nly.' partial.

Wh.it specifically happened that gave vou his impression? ^.What would a ip the past fepaSipmaity oth$f individuals a"ndD fou$ hive made have been a more mature reaction to g'fh.

.gcribed

g --. e. a studies. Some of these g mg the relevance of' va us con. ' ' 7 'suid' g$ ports fre;Lbsing ditions and actions on the obserped '[ ' reviewed.brief f.in ' 1 pdIi8ations.2' f(Q[ success in attaining the defined pur ' + 'p Q:g.,M i pose for this activity. i i m;.. . 3 g, gyg.dt is believed that a fair degree of f IibgENT Fox,1N, ksubcess has been achieved in develop.

EDURE,

.] ,,y,.C i$.prodedd.resthat will be of assist-j W mes stom i forego.ing discussion,'t,t:tsr rance $miga$h=e:ing facts.in a rather a. m. ....s-r 3.@m3m.,lda.rlth.a.t.4. h. 6... critical incide. nt tifdlMbiedtivF...f,ashidn" E. th only a m ? e. mque ts, ntially. a procedure.for m.um.of m. ferences an mterpreta-V w e ..r ons of .n' m. a.r.,u,c i.~ ore stibjec 11atu,re.. gather.

certa.m important facts 6coshrs "htvio'r,in defined siEifa*W..tr..dWftifr'is

't'tEfdEo. tit T that . M~ #. E,31ods.Q.ip" Edtd b(L t emphasiied tha't3Iirisessehtial ifjhese itiqidents'are to M M i $ g.f .b tie.a{conipa[ ' pf. -h'(khhgrnihifsuch dafa'gtdgsetgl.EfinE@en . e.,not co qNea smgle.rs yi .h' ett;(p,b ~ M ~ rl[Mie!1iEBU coll'ectient @$@(RahlfM8uld be'tboUghkof n Jeef$tMidilifp[Trindiplesihien Yo. gls5ch] j)h 7 .1;beinodified and adapted to meet the ;Vd6' h,f P.E 7 .X,d CMThf$ Etf ' teps 3R$pecific?situatiopat hand. "ThEleMe'dge.'.of.. 'the technique; is consmts 'of the" AsifiEat ffth"e ^ 3 Ethat only simple types of judgments 'critid$1 incid8n'td ljr$EU bserEc' Eof a an i a'dequatei. th'cory.. of"hhman be-are r.equir'd of,the. observer, r.epg'arr. hioV, this step is usually. an in-u e '" #. iro'rit brriy,[6isalificil ' oMrvers / ~ ductive one a' d is relativily subjec / included? and all observations-are n evaluated by the observer in' terms of tive. Once a dassification system has an agreed upon statement of the pur-been developed for any given type of ' ~ pose of the activity. Of course, sim-critical incidents, a fairly satisfactory plic.ity.of judgments is aTrelative degreg of objectivity can be acbieved matter. 'The extent to which a re-in piging the incidents in the d'efined ported observation can be accepted categories.

3. as a7 nds primarilp on the The second step refers' to inferences objecti Ithis observation.

Ely regarding practical priicedures' for . objectih fant the tendency for improving performance l. based on the $r a mNrh% na independent ' observers observed incidents.. Again[in' our to make, the same report. J udgments present stage of psychological knowl. m - that.tw'ogrifts have the same effect edge, we are rarely able to deduce or or thatrofie-has more or less effect predict with a. high degree:. of con-than [thi' ther with respect to sonfe fidence the' enc ~ts of specifi6islec' tion, d se or training, or.opsrating procedures on 'pi.i Li ju. goal represent.the wdefined %,Mi ,.,...dgments (NEilan. fut_u.MFeh"'rs of".t.hs tMd obMNed. e.y r,n61l refore.4 The;;.m use b.me stu i y

t..

,,sa. c, cura. cy and the, m 2 n.gn be: m vant establishn,,e,.~ied prmciples aw.- ect:ivity of the j. dgmetits de.+lig.ht'o v ..u e .the obj,fih.Ehredision with whicli thp.of humi4[ " vion'ani 6,ff K. e k,a6own u peed o t - 1 n chare.cteristic:has. heen defined u.d..e. facts:w~ .c aw- ~. ..e en$fr.aitin. f..,if,m.m

ackgroun. dMactors t

M*.th.,.e,gorn" YnIE.. deftmtion with relaa. .#. fi nf.[ h b i fi .an' ? v . th#spe. i.e. d ..this,.

v. t e. o, scr e. r v

.w ci rom;.this ' total pic-q mterpret ~ '%. tioritol, ;.a,c. idEntf

ryed.'IEthis A fure,s '

Mformulaidd. In ri ~ w - latter process, certadt more diffic.olt onlya few t u activities are there 4f.1,7 %g@h.pt ny W M.'

g.m ., e x , = t i, **f 3. Y'. ** ( y llA .A,,* li * ,1 'p. situation..iactivity.isi og,iudged successful' S T$wg ~ t I$,.n6deNi..r.'cIidd I t ' rs/cliarly" im possible'tda / 6e tap m y -.u ,., report thaPa..,. ive or:ineffect..has been e.ther:- person i i.n.., _.reso.. ..r.. effect.. .W $nAv;,ty by ; ;.fw v..i,ve...e{ a part.icula,

r...

m

w. -u proced.

s . L...- a ttvi pp ormmg a.spectfic. actvM. +. > c es of:,t S' tvem Sm~'.Fe$.~is eilpedtNih,,hd. I4IEt s.iodsh,itic..u(I.s.l$sE,.4% ' tess N E ks 4TMbie.u.AL. f i sect e steps included mgh~etpree-n.w. l. axagnplc, a.. super-:. n ~ n five:ma..m des,cribed bre. dy Y.f.in.. der =, to pron work.:.'Vinetoi t. teleasing a"..kety/- 3 i F vin d. s eitM..o.i.d... u.m..ahe e,t m . n a n.. er forta ,,~,:a.. .to participat a e or fM'ZCb* vide the worker withfnaxinfun/Se4%iiv a;.ryst '~ - "N 8 ppreWN' FyM ,.m iQj ~ $[M._~.mnera',l,,,. a. prze ,s 'to ex.,am. s.. y the... 'Walongh m tiik.. M%pndstlying[th} M8h1 ) dh@ h,9iipp pgat

hon, I.-

. li i d t /. 3. s s a ens wi,:.k N ,.8%)4 a I 2 su.lt' i.n. f atu, te . in: mMt m .~ r a m mm.. m...m ~ a. w:e 'ceerveceeA ~ 1: General A s ' es%gjgggf,,.fl {,f ln W f met ed. v - l ~

g... p y

j . g g.. g ;g,,,.,_ y. s it' M..,.,, ,g, nchN$e f ms c. i s n. .n a.u..- A basic condition necem.4ry for any piected tgup e work on the formulittinn 4 a funr. certain other t $5@ Ic N C ~ inn 31'deEriptMmMjkimtt,vitciadd Fi t .,..%.~~^,,....... f un.damenta! m. ien. tat. ib,,n in: terms on. tivities, tly-e mt W .. gin,ey n 4:s..n9. ;s y.7,p 1::b'. + ys .a .y .y the generat.u,ms of the acttvitg..Ag o;n y v: w r.,I'he,.. bjec,,tives o partici nw o phinning and no evalitaitEdf sprMc in the.n tivitFtTiunt 'thevi& ditN,.N ' p anO M M thdiM Cat mined f rom the.partielpanEthfint '1. O behaviors are gendal statement of$pfwld tpwardti.ves.,hN selven. In some in trend in the scientiti not le verbaliud to a nufficient ex!.,.,,, - operation.d stdtutuentsw, bas.. led -a tent tu (Hake it peihlc to ohtain..u IW . %. o . b R eli desEribe them directlyl' ~ numtwr of.^ .m,,,;,. u. .c.... t., unfortunately,,n moet: activities or na tn terms'of t6e there in no one general a..,situatmmrr i im wh.rh w arly,._there.,gai..e; performed, the ic . acts or oper s a w. = n tjy g nat uatiga.m.c. the correct,,o.ny.= N..md. . materials a.ctedy.- gy volved, the results or. prmlbrte, and rarely one. person or, group of persons. the relative importance of various who constitute:an; absolute l auth6ri20 on tlE g u. source,im,..x _ ...e. neral aim @n a typ.:ic b.. pful in empha a.n. a.... h.. ave un.tive tivityh, m.wc act.s and. results: T.. h.ese-lywa h .~ m 1 Iwen.hel siang t e nee.dmthe ac. ... c A-he. (v..manu ,g,9 ~v !"9f.%'l*fif.... - xv. a .senptions'ofiWe'SMY/8. ddif8, T d"r %WJAmt eng. re4.x.gelinents1 z. u C. t manN. t e e.: .:ny _ a.4 vn t.ivities.c .l'y,p. sgy. e.,ver. ; '.. r S.,...a i-3; e g ,~ sto emphas..ize ~. aim..a g' discuumons have. .s 7 i.f.f.g 4N st hgg_& IE..%,., the dmninant rotei.f%...,eigen. eral'a.imin formulating afscriptio t..a., . occ.&s s.~.e. igh.g.; g ns paRfhulid'aitbdtkisTWT}ustmerit m a4%FWsR&ps ktNTP 9 - d8iN grou w.6, cts - asms 4 cessful behavmr orad "E it..im plEstWmitse~fiiioviisntr$EWMk]7y ~ grjim%ga,t.;,. a nd .iquninf

n,p r . si,., b y -iN .i M Ment? m.m. ., mpg x g

iNU N

kMf4:w$h..m 3 n... w n, hat wes -these a~uth>_ bw.v%...rities or-toThers.... R m, ...u e e v:the+ag "geam;3 to ot ti ' telh!o. be tain. a finaiM. c'er.iO.r.iC. &Yttietgeneriild.m - .. A.. - @... n. ~ ~ t ~2 .a. . s genera. l aim for.- am. e tha:..t.is.. a.cceptab,.li.to the m.'. '* ject t . a.,! la.- 4~1e. ncipal. Necessary~ revisions.should be made .n n..a 1 spect-. u .9 Iter.w n.;. g. - 1.:ndw..e.u.res e as. indicated.y.by9: hese. discussions. .w no .a.. s e7 o.a. est.a.,blis. h., h.e am s t.~* y u r ' p.v io e s. t v.

.ulN. 4,...f th e.;., Usuall y.

4 use' ' to avod,.on. sider.. ble effort is req r mo c a , i 17 9 $3V +- tivit,,.y' shor, l 4 .h

4#

c ~ u p...e' pro t .p.y y*s a e 4 .the%' fiinct' ~ gri... of the;e. w. 4 &ys.w,v. M{r11ess tlWge'neiivi,tyJpi.ichlib@ptj.. n.fogg, ~ m n'answAC j .t* g g;,.y e j i.. ,g g ~ w.g:4j,p t infuW isaccepte ~ - '7 $ ~.AM k k W $ 'h&NkNhf f iled sta. c.e .ta.ho..ffor. men. ements > the%mi eta -2 t + ~ <w le,:e - tin. m y m.m$g.,.. y,_dy . j uc, .m

. :...g.t ts.a.w,dw w-te+.;

w w ment.y.ffth. wave' ,9 c w,w ,5 y , i. mostius.eI.u g.:..a..t e i a m ,y ll. ..s. e ..x A.,. T h e.c. tti statementoof'aimsk %~ m. .. center oun.epomEsim r, a. E e n .,,.w;. s. +. ....... v.. w.n,+.d. g. m,..a.e.- %.' ;;fw,h a.,t.k.m +:,w.cp thee or catekwgAtr.m ' ha,ch..aw. it slog.:.. 4, t:m C.- a p- .2, v 1 wo ant m. . t i hie.'in' char.a,cteF.Os Nordshrnvideny. $. i ~ -- ~ .d a maximum; of coru....munica4 r n. wit,4.. 4 g. t -- u ' * 'only'a/mininiihd hpmoiTe,miliidter.' ' Q L, . _ ~ ~ nEli$ik pretations Such~words% "apprecia ' Kf...ihe $ N - ^ v. tion,9 "efficiendy."< de' velo $$h Mi # d '. algof urtio8ldMnI. W,rb..pment,"! "'r $ LI# C.M.am w&rortf ne-a m- -f enz. p. statements, yk.i. n.kFds likely to be prom, ent m, of general alms.J. a =dr...hleghc-i'n. Gur.ukAiwJSIf,gsonum-m odxam EM a.,m m :. a.x. general aim of a teach m eleme.n-z.c - tary school art clas@(pciistica ofiyitl,e.the-general aim by cl development of.aEsp statemen t with specifie detail,s and various visual artjornigun the part quali lying conditions. of t he students. Thk dueral aim of IrAummary, the general aim of an t he good citizen,might che taken an. activity should be a brief statement iciphIon in th/Nevelop-obtained from the authorities in the I effective. ~~ ment. n iFati6n of the rulea and' lield which expresses in simple terms proce{ }h,ikh individuals.,andi thme objectives to which most people s.. ~ groups are aumted-in achieving their, would agree. Unless a brief, simple variou's goals. ~J ntatement has been obtained, it.will With the ai[6f a form ofthe type: be difncult to geCagreement among showiin Fig.1, the ideas of,a r.umi the; authorities., Also it.will be much her of,well-qmdified..authoritin can. harder. t'o convey.a uniform idea to be cob.. r ted,.It W '. ... -w m theb responeQr ths.,,,,,,. expected : thaj ine.e'.questma;on the pri v will g ~. P*nrticipants.iThis lat.ter group l ~ a,. rr6ve:,r.ahwimpression and w .n....... -a . mary pb'poserEf;thi.acti@ily_irenhthieshdik%ssiblerg l UseM1 iesired ~ r perwia + vill gid a fairlyriengthy.and'r. detaikEstatement; JThe s'equest' tal'ientiral aiswaarpei ~ summ:p:2e ni expected to ge.;them toy.b..lans# 6'W Tk g@a ^b - hos%. u s e 2: P - at Spee 1 ,.... - %.4 ' ont tNgere .gg .e inde,nse1 tMr intsM) " 5(EshbteWTo foeWa osers state. ment ofe cru.cial'4,.,.$n.fo.r,e 're bel.G.. ed.,'pects er

s. f bqhn w

to be statem.en t... Th. ese.%. ~ t.he..d. b,e po~oled ; ui and a_ttrial fo. rm ufe...m m r . my m.. na func. tional genery. i developed.i.This state. d.escript..-mn of;the a.s.jm. ;ty, precise j c a c J

p. r.x,m gg M.~ON C.FLA

.ww - y.y w ,2. - 'r' H %.w.. g A.k .: + inM;ec;.ytion3. m be givdG"to th2-cisc beba

shoukt be obeedeit"-

WIB. r 'ltha corded.? % *

J After the de;.' * @*Q^y

' ?.,, ~ o g g It' jfic as. 't these w v

b. Relewaue toIhe generalam,

..u.,, c possible cision.has been made that a particular situa-instructions l 0 witMrespectI standards to be tion la an appropriate one for making observa Q. j undin evaluYi Id[ classification, tions, the next step is to decide whether or not L. 'TM[h.oup to udie&&lebEnceds a spwi6c behavioryhich is observed is rele- ". %.-<o .m - vant to the general aim of the activity as de-j

g cm..

to b4 Ape yg .W (med in the section above. For example,if the i One practicW@3g,w! for obta..mmg general aim,pf the activity was de6ned as sue.V specific d(ta isi.t&obtain records *of tained high Tuality and quantity of produc- 'l "ctitical iEcidentM' observed by the tion, it might t4discuit to decide whether oc p.g " *

I"clud"
i"'"Ch'"*ac!J'*8i"8D an un usually idfe@ctive subordmate to get trainWW re'porti'ng herso'asel. Su6h fricident:

o -enther outstandm(. extreme, behavior, Ing that wayWassist him in developiiiE~E3d are defined as ~ gly efrects.ve or m-abuity in an ~ ~ tional or reerestiona effective with" respect to attaining the ity not reis't" ~ % w&ks.Wt 6 ^ general aiins oOth' activity.k The might be' _ II'._thayaniscti$ 'A e d"*IY ' "* i'*I7f'"Id 4 procedure has chEsiderable efBciency u $--~"~L" .befkURlf beKavve$ihmf(fyt- $oiQthfth . on the trernes o Mis:w could noHie$dli%iGi j, enown whether te MWald. + g~ thEt cittreme incidents ca more should #mbaWat bsgnad ~ ' accurately 'identi6ed thari ' more.nea.rly ave havior - The exteht:of:detait required >to. m which.ts y rage.in jectivity witti' respect toln tyg b-character. depends to a 90nsiderable degree' k ~ One of the primary aims g scien-ground and empenences of the .5 ~ ~ " ' tific techniques is to insu~re trejectiv. respect to @@ vity;' Foc exasti g *' visors with ' experience m a part,igt,.h, ity for the observations being made lar company can be-expected toragro and reported.,Sbch aueement by whether or not a particular beha'vior is re ^g independen t observern can~ only be vant to the attainment of the general ala e attained il they are all follo. wing the on the other hand, if outside observers weEC same set of rules.. It is essentisj that to W used, it would probably be necessary to.. spetify m conmlerable detad the activities a thene roles be clear and speci4c. gn that can be expected to hase an egeet on the most situations the followmg spec,- general aim. t heations will n. sto be established

c. Aasent of efers on the teneralaim. The re Ac.'-

and made expl ]r to collecting maining decision ehat the observer must msg

how important an eEcct the observed itics-e u

5 dent has on the general aim. It is necessary to /- Ihe d.ita: u 's. The situtirmi The 6rst neces. specify two points on the scale of importance: mary speci6catirm is a delimitation of the situs-(a) a level of positive contributions to the gen-e tions to be ub.erved. This spwibcation must cra t aim in speci6c terms, preferably including._. include information about the place, the per-a concrete example. and (6) the corresponding cons, the conditions, and the activities. Such level of negative erect on the general aim ex. epect6 cation de rather easily dc6ned in ma,ny pressed in similar terme. NMW For..czample, such brief specisca. A defmition which has been found useful' s Iw n. tic .s observationarePthe bkhavior in clase that an incident is critical if iC rooms of reguladfiinspioyed taschere in a ni6 cant? contribution,' eithey ~ m. sped 6ed high schoot whale instructing students negatively, t,o the general aiai ofvE 7 during class periods." constitute a fairly ade...The defmitio,nlof "sigriificanti a ~ s quate de6niticm of a situation of this type. the nature of the actkity'. If'the te In complex sitnations it is prubably esaen. of tbe activitsis in iersas of 6- # Qs'eng/$y% . 4 ni6 cant contribution night Toids],$1chQ ti21 n6e only that the speci6 cations with re-caused, or; nushtJoeva)can. tin d epect to the situatape.be relatn:ely complete and speci6c, but also that practical examples change in the dailgproductioiif _p be provided to amist'the observer in deciding ment either lin!tisthwes elam or a 35s%. ' ' TW$, M-in an objective fashion whether or.not a spaa-crease. -n

.., w e p~ z.?-f.f THL' CRITICAL INCL 2.. 'R a~

p Q, n'certain$' yh.. Ops %?p:

W%f P G.h. (, g,4 '^ W 'Sc situations it[pight be de.$ 34Co#ectingW Didsw ?*"?% g tgeepupga, quantitative m.We<& e, ii.::p. =09, N sarable angi -{ dan a.g gng orpasting 15 minutes '. _lfxproper. plans and specifications ~ r criterion ser*r production. In some areddnTeir ped 1the data 'ccliection' . ppaituatior@. ofsigni6cance might be. jhkis grt atly"iimplified. A neces-i u set up m terms liais'ssNedorlovt;iboth dl.,., i ic,n" for th. hase is that i rectly and indirectly. "?J 1 sagon(.t. is p Actions which fiifluence the at' tit'udes of oth. the behavior: or results observed be ers'are more difficult to evaluate objectively. evaluated,fc.lassified,' and recorded _ Perhape the best we might be able to do is to whiledhe faitfare still fresh in the state it in terms of a probability estimate. For des. m$ "th"eibArver" mind It would be ' example, one such criteriod might be that the 4 4 ac irable for thpe ope ons to be minimum criticallevel would be an Action that T would hansd inf6enEe sitchihat'at least orie @N MfMlN9%V * " - of importance to the companyl."& 'i; person in ten might change his vote on an inue ; w [up....., - NTMg[Q e ~.. g >.- '.? 4 @:q;.;y

d. Fjrsons to make the observations. One aQ gay.g ditional set of speci6 cations refers to the selec.-n T 1. m'

.oceervations.y g,.g, tion and training of the observeg who are toQ W , activ. 4 ~ - ic. make and report the judgmeraAMn & Aretepi above' ' I' ' 7!'Dd#Mi~';. K

' Wherever $ossible, the obMMhidtid W

N selected on the basis of their faInitiarity withi?* Ai cM'Y the activity. Special considJ MuhWftBF" 7 ~ - - - - 4 given to observers who have made numerous ).Jocag observations on persons engaged in the activ. er:A y - FM fM6PW servers are supervisors whoac responsibib.ty it ". 'd.S.Timend. W5%e ity. Thus, for most jobs, by far the best ob.. .. P 4w, is to see that the particular job being studied is 3. v be_.. d.. b..Q. e done. However, in some cases very useful ob. s'. Gen ' IM of iactivity. " T. +;, servations can be contributed bp consumers of E Specific behaviors.h the products and services of the activity. For

s. Criteria of ' relevance to general example, for a study of e6ective sales'activi.

aim. ties, the customers m ay have valuable data to

d. Criteria of importance to general contribu t e. For a study of effective pasental 1 aim (critical points)

'j activity. the chihiren may be able to make val-f unble contributions. In addition to careful selection of the per. Fic 2. Foau rom Duvutor:No Srsqarica. sons to make observations, attention should TIows RsGARDtNo OBSERVATIONS be given to tbfir.. training. Minimal training should include ~a review of the nature of the activity and a study of the Perform,ed at the time of observation ' general aim o@' an(d de6nitions for the judg. so that all requisite facts could be speci6 cations ~ ments they will be required to make. Where determined and checked. Memory is the situation is complex or the observers are improved if it is known in advance not thoroughly familiar with the activity, su-that the behavior to be observed is to pervised practice in applying the!se definitions be remembered, it is greatly im-shouki be provided. This can be done by pre-proved if the spec.fic aspects of. what t paring descriptions of oba.ervations and askin's the observiEs'to make judgments about these is.to 'be,, obeegved, are. defined and. if materials?Their judgments can be immedi. the opbratides t&be perforrnediith r ately con 5rmed 'or corrected'dnilog's'uch s& res)EcfTcTva[$Ytiohiid cfassifica-U pervised practice periods. tion are'cNNhpe$fied. ~ +M In Fig. 2 is shown a form for use in develop. The critical-inc~ftliitttechnique is ~ ing specifications regardmg observations., The. gg ggg.M. M ; data 'on useof this form m makmg plans for the collec- - n.. ,we tion of critical incidents or other'typea of ob E O.bservatgns pge,vgusQ,,made which servational data should aid 'ini objec'ifying.i are. DeportQo,tn; memory. This,is t dd ma66~satkio. usUallg satisfactory whe@ the.Em-i

47.;=3.g.3 ~ly recent and production requirements and accepts 491 3:y g y .pg pq ts:re _. .g. "' tivated to ing changes 'in jobs 'are ' higher in%% I6Inke ditta ~as and' eval. terms o[. the recorded than the re-1 l""udti.isisME ~ liscident. called'.iricicients. The fact that items I ' N' l o.4 y u..-. w h m -as wast.m g t.ime and assist.mg on i .. ~....-..d.occurre. yc -gS suc + '. TThe.importapce of obtaining re-problems arg lower for the " recalled cent incidents to insure that the in-incidents suggeists tha't part.of.this. cidints!are' representative of actual discrepancylies'in theinterpretations i happenings was demonstrated in the of 'the ' category definitions. Thej da air fo'ihe ftiaflib ' car. troll classifying"of recorded iricidents w o ' study ~yNagay:(48) reported above. done by'the foremen 3rhile it}s classi-lers b Howe as al.so discusm_i in.athat ficat.io.. n of.t.h.m.. e recalle.d. incidents.w.. 'stHdy.g, ~ -~ -s w

m..

.gm some situations adgquate done by the4researcD.c m' W cfe s.. I co gragmcannot beyhtained iT only' fairnessdi h'hi( C~ IE9!$fsh? ' ve ry re'ctnt inddents are included. tho' defii' i Evide nce+regardinlei$ac9uracynword 6i ofPreporting is usual.ly 1ontained in 'were?!nis. Ul the incicents themselves. E full and -@c ~ usually tie assumed that this informa-to'assa " ad e d i 'OfiVe? ' etails are" K ven, it can precise tion is adrurate. Vague reports sug-tiodiiiare takii[resal i 4.3 gest that the iriciden~t is n'ot well re-. be pliid[on to pR$idhYd$ii5atifditI ' membered and that some of the data. for a fairly satisfasf6r'yLfirst appr5x$f ~ may be incorrect. In 'several situa-mation to augiakmgagt the require; tions there has been an upportlinity ments of the activitt.5 : Direct obser~- to enmpare e b in of incidenia vations are to be prefer'ed,' but'the rep >rted under t wo. cogionn (a) efficiencn immediacyQnd minimum f rom memorv and wittinct i" list of 'demandel ais cooperating' personnel t he' typ-n of incidents antirtpated, which are achieved by using7ttiiie+ and (la thone rep >rted when' daily incicient data frequently make their.. observations w$re bring.made in a use the more practical procedure. -outine work 'tyation, sind the eval-Another practical prd,L..._.m irotF ' uations amii 'T@atiom, were made lecting' the data for describing an'.' ' , and recordedo$ prepare I check list activity refers tu the problem of how-within 24 hours of the time of obser-it should be obtained from the ob-vation. The results of one such com-servers. This applies especially to parison were dincussed briefly aheve the problem of milecting recalled in connection with the American data in the form of criticalincidents; Institute for ReemarcE' Eddy of fae-Four procediires havenri'uned drif ' wiu beM$dnamed brieflibek'w:~7"2PC i tory employees. ( C %F%; $'"""Q!iks@M*S a During the pbservati5nal period a -- M***7

d "a'

smcpy:g"* igg .n.egligible numbd,....of incidents were' 8-P ~ . ta reported by the foremen as not fitting. gg.g;,. m g g. into the general headings meluded on w.,, tha}t og 6m6.wy &

r. W. yip gyi...x the' list. 'Although the proportions of probably :tEWiisjiistist consciion incidents for the various items'on the W--d,4This type ot radervws,w e.what, diferent fiens *sthef sorts"in.seutyswa and W I

list are not identical, they are reastm. g gg g g ably close for most7of the-items. win _be Mih4MM_ m n ___m ~ .wm m zss

yy? .,p.;. a m p ~,4 g g &. the observers is collecting the data, it is ordi. ,vilopes, avoidance of id:ntifying info'rInstion 9CTnarily desirnbie to indicate on whst euthorit ' the mailing of data immediately tic' distant r y the intervigs being h:Id. This part should point for enalysis, and similar techniques em x3 be as brief arep6ssible to avdd any use of time helpful in establishing the good faith of the in-terviewer j.n taking all possible precautions to for'theh6r%WeYtfriiWes inMiaNWdiscuseion of a togic irrelevanti.Y feguard the tiii I' sa v "N stances all that needs to be said is that some. . (v) The guesties. The most crucial aspect one known and respected by the observer han' of.the data collection procedure is the ques-suggested the intervjew. tives asked the observers._ Many studies have (ii) Purpose of tu study. This should also shown that a slight change in wording may be brief and ordinarily would merely involve a. produce a substantial change in the incidents statement that a study was being made to de. reported. For example,in one study the last ' scribe the requirements of the activity: This part of one of the specinc questions asked was, wonid usually be cast in some such informal "Tell just hqw thi. employee behaved which form as, "We wish to 6nd out what makes a caused a noticeable decrease in production." good citizen," or, "We are trying to learn in LThis question resulted in almost all incident's detail just what successful work as a nurse in-reported having to do with personality'and at-cludes." In cases where there is some hesitae titude behaviors. This part of;the qu' istion tras a ' "NLti6Pseems desiffisli,Ysfat'eE6ent "csn beMilchIzuusia noticeable *d'ith,ik', '.si tion about cooperating ori littie'more expla-changed to "Telljust what ~ 3 ecrteurin produc 5 ~ added concerning t6e vahic and probable uses s. " J1..Thi'ge oflacidents.qTo th i tion of the resulta. This frequently takes the fortn 2 of improving i. election and training proce Ewriter. l be behaigid" and '? wha. did"' dures. In nome instances, it would involve im-..see'med like about the'same thing. To' fore. proving the results of the activity. For ex-men whoiere reportingincidents ff be- "~lii %d' atti-ample, the interviewer might say. "In order to ; hajedFoounded as p[EThe~subtWbiase get better uten clerks ~ we need to know jmet. ' tudels$ers beia[ stud' what they do that makes them especially efL 2 inYnihed in the ~wertfing]of questidiin"are not fective or inedertive," or, "If parents are to alwa no easily Iqut#Buestions should al- 'tried out with a small group'of typical be more eHectis e. we need4a. Whee-tn teF * ^ n them the things they do that are effectise and observern before being put into general une in ineffective." a study. '

i.,.

hii) The coup heing intereirwed, if there in The question should usually refer briefly to any likelihood of a pern6 i feeliits;" Hut. 49 ihegenssalaim of theactivjty. Thisaim might a-k me?" it i desirable to forestall thin by be discuaned more fully in a preliminary nen-piiniing ont that he in a nwmber of a group tence. It%hould u*ually state that an I'ncidenty w hirh i% in an unusually grned punition to ob. actual behavior, or what the perum did in de-crse and report on thin activitv; The special wired. It <hould briefly specify the type of be-quali6 cation of nwmbers of thin group an ob havior which in relevant and the level of im-servers can.be mentioned briefly, an, " Super-portance which it munt reach to be reported. It ah uld ale tie down the

election of the in-cidep)ts to be reported by the obaerver in n siwirs such$ifyourwlf are countantly obaerv-ing and evaluating the work of =witchlwaard girls." or, " Students are in an unusually good way, such an asking for the most recent obser-pmition to ob,erve the ceintivenes, of their vathm, in order to prevent the giving of only teacher
in a number of wavs."

the more dramatic or vivid incidentn, or nome Gv) The anonymdy of the dato. F.npecially. other aclected group, such as those which fit for the collection of information almut inef-the obarrver's stereotypes. fective behasior,. one of the principal prW " An effective procedure for innuring that the lems is ta convince the observer that his repurt interpretation of ihe perenn being interviewed cannot harm the perwm reported on. in any is' clone to that intended is to request a nample way. L'aually he also.needs to'be convinced of'perstme typical of thnee to be in'terviewed to that the person reported on will never know statein theirownwords what they underst nd that he has reported 1he incident. Assurances they havelbeen asked to do. These no are not nearly so effective in this situation an-shduld he'aclected no as to represent all t pen actual descriptions of techniques to be usedii. who will be interviessd. From a study of their handling the' data, which enable the observer. interpretations, necessary revisions can be o judge for himself how.well the anonymity ob made to insure that all interviewees irill be in u the data will be guarded. L'nder no circurn,x.. agreement as[to ths; nature of the incidents c N hey are to provide @s. stances should the con 6dences of the report J The interviewer be violated in any way. The une of acaled'.,n.,,%K73rGipiWiersegline?:s w. w x 3 sm,ngegy. u,gy

y%;bould cvoid sold:[s needmg questsone citer -qg.g f y g' }&, 1 1-. R \\ s a e7 j,, A 2 gf...yMg, the main questina has boon stated. His sv. 57"Th k'of the last tune youf ""subordma 3 w W:

L harks should be, neutral and periaissive and y

~ @tihe acceptss the observer as, 4 'i'"'. wry helph:1 to your M%y$ %p shouki ebow t.g Y the C lisdatini that he understands; 4 the expert? Bb what is beingini faMd permitting the observer . meetmg their, production schedule.", e lkm. ~the m. tarviewer can (Pause till he mdicates he has such an to do most of th g, Incident.in mmd.),.,Did his action re. usually get unbia ir.cidents. If the question '"I' I" I"Cr***e in Production 'of as much does not seem to bei understood it can be re-pented with some reference to,Narifying just

- ""* Per cant!w that day?-w some 8'*'g,

,, ~ e what is mdant by it. If ~t$iiiobserverh given '. what seese,s lines only 'part d tk story, p- - (H the answer u.-lr,,, say),,I won. F,> no should be imcouraged by restating the espance . der if you could th k of the last time \\ m r of kie remarks. This usually tends to encour. > that someone did something that did age him tp continue and sesy renn3t m hu. have thE nmcWan dact in m. +- cressmg' bringing ont.t inany reMt Mils that the m, ; pg . (When heTndicates hehas e such a .e g@'gE..uati.on m. mmd,.sayb.,k.What - Larviewer did not know-the'*gltuation well n a

rm

-w e enough to uk for. In eonne cases,it is desira-f

"m w W $

w b4 to habe the interviews recorded elec ; g %','1 to.mily and transonbedO This 'incrence the J gf ' ~ ~ R t work load [substhetially, and trained inter : l W'N viewers casq usually set entisfart ports at 3 [ f the time rr;by editing their notes lyaller.; KM " D, tbe interview. S7 ftNi Y m.gM 7' '~ 3 . Usually tbe interviewer ahuuki asiply mr-tain criteria to the incirlents while thev are be- . _ ' g~ ' r' ing collected. Some of the m,e ;.wasn t crie' m e .M z- .~ s._. teria are. fas is the actual behavior repuited; (6) was it observed by Ihe reporter. (d were all, y gggggp Weh 3.y w gm y ..v relevant factors in the situation gi$en, fd) has your group's job donisW *w- "S he obeers er made a dc6 nite indgspent regard. 7 M-ww3ie - ing the criticalndas of the behasied:(d has the e gg r w/ mm%.I ISC i Aw ob.crver madm it ricar nuet eniy he be!ieves the -4 behavior was criticaJ.

- When dJd'thisi incident happeni'Nk;.

in Fig. 3 in shown a nample of the type of - 5 form uned by internewers to cu!!ctt ctsticat in-cidents. Of tourse the form must be adapted to the needs of the spranc situadon. ..What was this person's job?"

h. Oroup & Elf rF6 dos. OeCauas Of the cou4 in time and personnelot theirutividualinterview.

"How long has he been on this job?*i a group interview technique has been desel. 2~ oped This reteln the advantages of the indi. vidualintervie(ki regard to the p ramal con- '"How old is he?* tact. explanadoOI,and availability y of Ihe inter. Wh newer to answrr questions. l'u some entent it I~~~~~~

w.;

also prmides for a check on the data supplied ItG. 3..%MPt.F, oF A Foam som USE' 54 A,N by the intervieweca. Its other.tdvantages are INTF aVIEwF.m IN Cut.t.ECnwo ' Errscriva. that the language of the actual ob.erver is pre. Ca:TscAt. lxcmKNTs ' ' t y.2. i. cisely reproduced and the time for editing the 1 ..A; . M'4r~t@ interviewe is virtually eliminated. ble for the interviewer.ta rend the responses es The method cunnets of having the inter. each member of the group'to the Arat' question viewer give his introductory remarks to a wul 'mske. su riv ilist.16e 4M(Msii firhich group very much as he would doin an individ.' wastad.6%iemagtFbs a ual interview. There is an ' opportunity for of Eiisal'lacilissidand the'risultaih most questions and clari6 cation. Then each pernon airmations' hitGi biiea[encelleistFin~tWeport is asked to write incidents in answer to speci6c of the. Erat.usato(jhis procedure by;;j(4uired agner. questione contained on a spec.ially prepared (65), the amoistt of interviews tiene fe form. The size of the group which can be han-per usabielachit3ke).FHauses for thei died effectively will vary with the situation. group intervisAsV 'cinnpared with if the group is fairly senH,(is usually possi t.K7.gnisugun[ isserviewiiT$The' s m&,s 2 n:How m

^ ".l Air Force, aobtained'from ii a

7. "tien. cystern withg ~

~M f~airilie two'sittia'pped deista.;dEGusti co M i For moet (, tioiisc tWi ' "et ^" hined$ jff he group bemm4s ~ whe'n the addition of I ciit i in~xcidents tD' n ' ? C:?, g t iew procedure is morein ~ questionnaire procetture. 2 [ the. sample adds only r three critipal bik,h th"h y ~ hahiorst For jobs of a e isory nature, it, ,3 T er t rail types of combinakione a asesselGFbetween 2, and 4,000 critical 'Torjiin used. The onegEcents are required to e bliah a compre-W 'mc st d , irom tisoee discussed is the ~ hensive? statement of regtiirements that in-is i t . mailed q' uestionnairs. In situations where the

9beervers are motivated to read the instruc-cludes hearly all of the di6erent types of criti-Metions" carefully and answer conscientiously, cal behsviors. For semiskilled and skilled jos

' $f#nia fechnique ineems to give results which are adequate to cover the critical behaviorsbe t l .dnot 'erimentia% diferent from those obtained Coverage of all'or nearip%11 of the'various T$liisthe int &yaiw meQiod. Except for tpe ad. ~ critica[for'hetTe

4 tion ofintroduc behaviors is not the on criterion as to l

M.

n W e 5 M iiig V remarks, th's forres,used!, whethe ber of critic. I
= b' 7nesanstof"' ?iiscideh i

s beenii:ollecTed.. relatively ^ .s snailed questionnaires are abo , m.i..those used in group iitterviews. .Eiresmaki@b pf{cije[ litionfoPff.Qidbeh&ViReat-f . TjM '~ ..we H 4-Rm,? '~or:iQne other procedQM

ireddt, y3 ocessar)(t'djet ap fidir e j-dN coleecting'da2is"t>y ;,Eidim -,

-}' itcordsN bah Igif eac bei L",, Tlieri Erg two varieti'engf recordin one is teJ7 toMisdha a ' " ',"1T ai r 9:%Wr$gord de 'TThissitya ilsTa$,, @ ; hap. JhappeitT Otr' "NTidcon ~t is v,ery samtfar to that d=O b ' ,l hd _ .ito ' " more.inci .'be with obtaining incidents by ind gy 9eHiews al$ove,,ezcept that the observation'C asi'gividg 'of in 'u' a b for. s the'diy nie"n'N taajana f[;Nthe introductory remark and the presentation t 3 mmary', alt M*"EbbeM. iitical3a no sim 'f ' reiiuta fordetermisiingQMrsd this is i M "of the questions until an incident is obeerved n c incidents that will bi'ri

to happen.,

I i t)stIprt>cedure is/to' reds 5 mpettant considerailonj"ashe plan-of the " T'l ' ;A ' iriatiods.,.- edNEmedy: checka s' hould be maile both on the first v such incidents on forms which describe most of the possible types of incidents by placing a hesidred or no incidents and^again after ap-check or tally in the appmpriate place. proximately half of the nu'mber of incidents As additional information becomes availa-believed to be required have,been ob'talised in ble on the nature of the components which order to make it posible to revise the prelimi-nary estimatn, if neceuary, with a minin um make up activities, observers may thus collect lo.m in effort and time.- data more efhuently by using forms for record ing and claneilying observations. In thy mean- \\ time, becau.e of the inadequacy of the infor. 4. nalyzing the Data mation currently. available regarding these componentar t seems desirable to ask.obeerv-The d>llection of a large sample of era to re " . '" observations in greater de. Incidents that fulfill the various con-tad and ha' i .classi6 cation done by spe. ditions outlined above provides a cially trai ~ ~ I. - Ssu of sanipk. "A general problem which' "functional description of the activity overlaps the phases of collecting the meidents. in terms of specific behaviors. If. the and analysing the data relates to the number sample,.is representative, the judges of incidents required. There does not appean well qualified, the types of judgments to be a simple answer to this question. If the activity or job being de5ne I is reladvely sim* appropriate and.well defined, and the i . - pie,it may be satisfactory to collect only 50 or procedures for observin* and report-e JM 109 incider[tEOiithe other hand, Aome typesidN p3g,; at.'@;j n( y*

epodeg

~ e in , %$60coenpler a6Mt .2 92 thousand incidents foran adeq,uate statement.appsur'to require severa1{ accura r can be_ expected to be comprehensive, - m r,.qeirements ' 7e o 'The moet. useful procedure for determininit" ;.detiiled3nd Tia'iidUn3this form. 'whether or not additionalincidents are needed 1gghnt r ~, for going ? W:. is to keepErsnaing' ' count on the number M ffurt,,%ap e g, y utt new critical behaviors added to the classincai'"Thah, %ae.g%g,, iM, a' ana.% i lysis ~ s: ;:n% -

3. Q.:

~

g pu% sta'geMo cukiNiirir[.igyanIdescribe} devekonunt isscedures or .f ~; 9j 'y r M.y~.4 m sent, manner.. v the job effectnieneess For.. the data in ane s. .ao:. he most appropriate classa.6caten s .~ t i ha,it can; a. . used f$,, 'p.ychologicatonahThe usin headinenniiMi! _33 t v ac c..

3 do with typiis hi psychological traita'tlisit are {

v.r? ~ i jn disc lo' llows,'it utilized in time selection procese. For ' training % M 'Tf l should bP$e. serymnde that uses, the beje clamincation system follows sh + thej set of headings that :s casdy relge l PJ$CK Of M.CFiSPh.@iM . W$9 Eses or hI'ddlrsNifrig71ms.4to triuningm c i tor pro-pleted. Yhe specif5C res to bi ficiency measurement, the headings tend to be 1 discussed are not conc'e'rned with im - similar to those for training except thai there proving on the $0mpr'ehensiveness, is lp a ttention to pmible course organizatic..

.g 8pecificity,of detail, Or v4gty o{ gEh ponents of the job'as it is aictually ' ericrmed. - a 15 statement of thegitutrtyents of the. For the development of ps. arm for'evalu p

activity.. Na~ther, they are concerned ating on the-job e#ectiveness tolatablishla@; "1 with making it easier toFreport these ' criterion of ruccess;theclami6caition ^

^

p,., rc<Iuirements' to[ draWE ' inferences "'"*earHy directed at, presen,ti p. from thetr, and k come -pare the $h,p: job behaviors under headi~ng0t ~ . 4 either weH ensrked' ~ 1 tivity ~with other activitics.P P Vide a simple frainswork. ~ 1 The aim in to'iit' crease'.the usef& iobactivities ~ ness tsthydd$#aMaie#%cidMd[%NM i little as possible of their compre* Smdarly, I n ponvocate..m t na see a hensiveneal, specih,eity, and validity.- frame of reierence d6iis'ndentiid ~ la 2 ~ It ap, pears 1that there are thrce pri-for the 6ndine.;For esempisliy" 'F irg made eo de6ne immaturity?. marv problems involved : (a) the ' tailitary perennel, the GiiissE6fduQ / seleciion of the generahdrame of' """Id d*P'" t mnal denen.4 *""*h*3 "" ". beg![gunc 4 ptmn is to be uaQ pr y inP m r eference that will be mq. s.g uwful for 3 dewcribing the incidents:-(61 the in-identif y per.onnel showing this type ormaladC i instmeni or whether the principal.dr 'typevo(n f- -use.w.ill be dinfive devekspmen ts.d.a set oF to try to pertanrw M eions I W m.ijor area and shh.irca 'h'cadinga; ~ i ' "d ' i""

in which immaturity reactiomi.C.

and (c ) t he wiertiots of one or mnre. wendd not lead to serious dif5 cult en. 9 y 9,,,, u m The indmion # leveis aloijg the.ppeyiticg.gygeraht.y.a caseseries from'the ba ie data in1he form of I e onlinuum to m.c "in ~ feporttlig 'the ' incidents is e ta k' requiring insight, experi-t et put emen t x l{arh of thenc prob. ence, and judenkni. 1 unfortunately. thi pro-lems will he diocyssed beloitri w ure so. in the pment stage ul psychological knowledge. more.uh ertive than objective, 7.p. l Frame of . There are counricas No simpic ruten are available, and the quality o. w ays in which a gi of incidenta can be i and unability of the nnal product are largely...[y[ e laandied, in ne . thef general nature of dependent on the nkill and sophisticatiott oil.. t he dannihcation, rise principal run nleration the formulator One rule in to submit the ten.

}

hould usually be that of the une, to be mad 2 tative categorien to othern for review. Al-l2. ' I of t he data. The preferred categorica mill be though there in no guarantee that results p5i-t huac believed to be moes valuable in using the agreed on by neveral workern will be more useg, .tatement of regidreviwita. Other runaiderar ful than those obtained from a" single worker, J nuns are came and accuracy ed cina.ifying the the con 6rmation of judgments by a number'of i deta. relatiun 1o ad@dd ped deAni g;. persons is usualfy 'renmuringJ7be "' ' f tiJns~or MsMficii gy tions of interpref(aten%5d reporting, which ~considerad*cedur'e is to sort a relativel h incidents in'to piles thatMelsiteditot franseof referenceseiscted? A'ffe/ihilie;%tF A will be discue ed in a'la6er section. For job activities, the choice of a frame of dennitiorm of them are mad @Flife'dditti tive categories have' beeni"estiablishid, br i reference is unusily dominatosirby consedera-iaU -a tions of whether the priescipal use of the re-incidents are cleseinedje "ThediEthidi <iuirernent's will be in$ relation 'to.' selection, procese. needs for re' eini "" far- "d d iraining. mes.urennent<a,t pronciency. or the. velopseent si seer ca 1.' lasiiW "?? E - ?.... [hdh Jg m.... ~. s.m,.m%mm.. 5 a t n

CkWIN tentative categories are snodified as mdicatedTSRbeerp*refidg andR

gs'.c..

T 4 $ the ~ 'disatilinuid stir aupiMMEmi?;:'i 52O '-W; esi 7 M *It,is. never possible ;m practice to ~ ShThel M@edre sutxhlvided intoa.obtairi an ideal' solution fdr, each of. & dentshave l hha d P(a,,cticairproblems involved in M p taining a funttional descript,on of Mplacedtogether'. The de6aitions for all the i an act,y,ty. Therefore, the statement M % categories and majar headings should then be1 'of reg;ta,remente as.obtained nee i re4xainined in' terms o( the actual incidents i gcidssi6ed under.each! N' interpretation 'if it is to be ysed ~,-l5Fe. Gescral behaviorsf The last step is to de, p;.Miy. InTihany cases, the 'real 49ehmne the m.. os.t approp. ate level of specifi~ - ..made* not.in the collection c errors: o.d...o _are.%.f.,.th, data,but. 3 ... w... ....m .~ ity-generanty to use in.reportsng the. data .an aanaly61.s-o e, m the This is the problem of weighing the advan'.. Qtages of the speci6 city achieved in specific inci- "failbireft'6" interpret"them pro ~perly. .jdents against.the simplicity of<a relatively' ;Eachh[ithe fbdr preced,in'g steps, (a) t The level chosen r m am - r~ 3 small number.pf headings. m -. 1the determm.at on of the Eenerattim might be only a dozen very general behaviors v m s ww.9gggrg.d } w .Ji9,i; mishy beyveral hun _ dred g#pecific$. i tion .m~behaverst Pracocal considerations m tlw wih ndYoh r .hEinediate situation usually det6 rhine the op% ns tSI inak, h ' d at'aT " p.e d4 N ~ ME { %f "rnal level of generality' to be**used. w3 N .gSeveral considerations.phould'.be}kept' fig 2 M h piases ye-Nmind in establishing headings for maior areast 2been% g.., A are listed be " ado"y%.%y,oduq hyith,agpr ,s nt r - and in stating critical requirements at the sp w" met'e + lfth'eiiFisMdi"" on<inf. lit " -lected level o( ge y)- . go,. e-m vymm (i) % head. 1 -mgs and requiremensa.shoulek opmmn as t+o the gen.+ l aim and-o era r mv/.~ Ob the competmg, alma is sele.cted, indican, a clear-cut and logical organization. s th. )hould be made very clear in the is They should had z tttmmtMMr and easily re. membered structure. report. If the groups on whom~ the (ii) The title, nhould convey meanings i" observations ~are made are not repre- , themselves without.the necessit,y$reiitiatiisk. acntative of the re1evant. groups in-of detailed explanafiim.7rPdiff

dennition, This does not mean that they Asuld not be.. Velved, they must be described as O recisely as possible. The aim of the defined snd explained. It does mean that these p

titles, wit hout the. detailed expl$ nation. study is usually not a flincfional de-should till be meaningf ul to the reader. scription of the activity as carried on (iii) l'he hat of statements should be b-mogenaeu.;peqhe headi7ss for either areas by th.is anmple but rather a state-or requireniensishould be paralid in tiontent mem .ing to all grou'ps of -this \\ and structun./not defining either unnatin - Headings for major aream type. In order to avoid faulty in-should be neusal fere 'es and generalizations, the limi- .lactory or outstanding behaviors. Critical re-

n8 mPOmg g>y t}a gmup must quirementa thould ordinarily be stated in pomi.

be brought, ta clear focus. Sign-m tive terms. . (iv) The headings.of a given tyg h>uld all larly, the nature of judgments made '".be r(the name general mignitiade'on level ofR in' collecting and analyzing the* data importance. Known biases in the data causing must be carefully reviewed; - one area or one requirement to have a dupro-While *h"M'm'itations need g~' be ~ Wportionate numberof:i~ncidentaishguld not haS..# I W ~ res cted in the heedfiig'sg "%Fstcb wWear.lywreported,.(the value of. the W -M.# h) The headinga usedik cleidellIrftion andl 7es6ltiMah6uk8 Walso be emphasized. Ef5 regn>rting of the data shiiiMbe such that find irTodoften the resca~rch worker shirks, ~ ings in terma of t hem wilt be eneity applied and g Oresponsibility for tendering 7 a maximally useful. i, udrw (vi) The list of headingNhould turcompggmen 9concermng the degree of hensive and cover,an.incidena ha+iq sisam6.?Tdibility, W^eich sh.ould be attached . _ +.' %.

&nd'

~ .L:t?is a difficult task, Rl 'it m.e 4:; coat frequencies. ,.m. .=. d . b 3 d T S,w,,, b%2 ' %.c A ' T"M wth"% .:. ~ -.

.a q e usedf andbec511st odomaM125M i y' tiut if-the resalis someone wil!Jgg to make such a of procedure for evaluating the typiP l o judgment, aiuMRidoriginal investi-cal performance of persons engag2d v ator is bestl ered to' make the' in this activity. If an observational saa y eva natT..-..-r$.E..t.,fied..for.the check lis(that includes all the im-r 'prtant inhaviors for the' ttvity is w .a case or, r certain stypic f 7. MiEld, the performance of the in-genera specific %iaiffpW%f@W dividual can be objectively evaluated 1 i USESjOF THE Chir!CAI. INCIDENT,!.and recorded by merely, making a. 4 l i single tally mark for each observa-TECHNfQUEF i The variety of 'situations'in which tion. Such records provide the essen-the collection of efitical~iecidiiniswillf tial. basis for criterion dat sufficiently detailed and.; specific for prove of value h;as only been par-special pur' poses but at the same time tially explored. In the approximately eight yeads since the writer and his. can be com ' began"a ~ systematic for-- evaluatiorig w en thid[ irs tinisiedhN colleagucskf*ptincipfes itnd proc (dures Such a sie.wa's"f mulation to be fulloped (Qkethg Aisstyis MiethfiIricMNii3E witlF of data, a ! airly'largd:nu glRr of ap gn' velopmqntit!"stagigi t%gMme

1*g;.

.:-4 f plications_hae,been maIk the appli: Tnstitute" for ResearchM1%e N$f ~ # catinns wetebe discussed under the:.cludedrPrfstoids"st'udythe'IR ~ (s)Diean-followidh hine headings: ures of typs al performance (critcria);. Napsy'istudy. ons air ~ route traffic ' '? (b) m'casures of proficiency (sandard ' E8ntrollers for ttECivi1 Aeronautic (d) selection Administration (49)'; and M. H. sampics); (c) training: and clasniriratinn; (r)% MgttwF Wt%f5WYttrdy. oli"research per-sonnel for the Office of Naval Re-pu rifica t io n, (f) opera t ing proced u res ; search (69L Another American In-(g) equipment design; (h) motivation stitute for Research study was.re-and Icadership (attitudes); (i) coun-seling and psychritherap?? poried by R. B. Miller and the Sp.a e n mit available hery to de-present author (21). This was a per-acribe these various applicafinns in formance record form for hourly detail. Ilow "'f a 'briff' description wage employees developed in coopera-of the type > application that have tion with perscmnet of the Delco-g 'with brief illustra-Remy Division of the General Motors twen made,- tive examples iind references, will be Corporation, the Employment Prac-presen t ed. Some of the studies in-tices Division of that corporation, volve several of the types of applica-and the Industrial Relations Center of the University of Chicago. The tions to tw discussed. The presenta-tion is not intended to be emnplete,.. same authors have developed similar l but rather togalgrpader intesg. performance records for salaried em-further study some. oriental playees, and forymen and superviso ested m, .;.21 (22;423). .The. principles ' iriid~ pro-tion and guidance.1of ' typical performance Jeedures. underlying th Measures $ tion of^ performance have beEn' pub-The simplest and most (criteria). natural application of a systematic-dished elsewhere (14,.15,17). ally collected set of critical incidentsWMnumeiF of' imp 16ftant contribu-is in terms of the preparation of 'a stkms Sto thEYdeveldbmEnFor fune-of critical requiremenisJFtionhfdesefistions a'nd<staitdards of , s.; statement

y. m y:gg

u,g.v. _ m 2 . mwy 7

performance h'av, e. been mad 2 by

-they en c means of sharing thFmont essen _ ' I other gro sing the critical inci-ti*l di'cipline of examining i C"-M! expo. n:nce, fonmng hysmthews ebout pmfemirnal q,gnyL - gn2 og tKtmost conduct, and testing these hypotheses by rei-m.. ' 'notablis' os;the development erence to the welfare of the people afected by 'by Tob if(37 31), o F Ethical them (sip. v). " *" ' ~ .' WStandardsW4*lipcholetisw.TVMeire "'*"" + than 1,000 critical incidents involving In addition t6 the study by Smit psychologists mentioned in a previous section (58), , ethical problems. of he meinbers of were contributed by t several other studies on the use of the 'the American Psychological Associa-critical incident fprocedures as a E tion. It is bdlieved.that thiogepre-basis]for, evaluating teaching effec-sents the first attempt to use smpiri-tivenps,have been reported. One'of ~ cal methods to establish ethical these wjas. a study condyted under standet de. .Because.of the impor-the joi6t sponsorship ofthe.Educa-tance of thiestudy,and the generality tionahResearch Co'rporatigiMnd the of some of+the problems involved, Harvald Unive' uIste. School. ionf " pinvided by - . +.Strtajag[tpe., conclusions reported by_ of E,d thd Committee on Etliical' Standards the New Sc il~ D e p '. Milton FundthTliii.was% 'F.- for Pynch' ology in their introductory merFCouncil statement will be quoted here.) 'an ex bra-tory study of[telichei$c6mhetenc'e First, it is clear that psychologisen believe that ethics are important; over two thousand reported by Domas.(6)?.Npproxi-psychologists were auf5cieiitly concinied with mM$ 1,000 criti6al'indidents were the ethical obligations of their profession t colledted from. teachers, principals, contribute substantially to die' formulation of and other supervisors. Although this these ethical stancjarad.AcondtR*do39gt48 believe that the ethics of a profession cannot be wasTiiexploratory study, it was felt prescribed by a committee: ethical standards that it made an important contribu-must emerge from the day by day value com-tidn to the general problem of relat-r pr es n. y ho ogi E ""0".ry to teacher competenge. Irhe sec,ond of these studies was , hare a conviction that the problems ~of men. even those involving values. can be. studied conducted as part of the tehcher objectively ; this document summarises the re-characteristics study sponsored by suits of an efort to apply some of the tech. the American Council on Education behavior of. .. science to the study of ethical niques of and subsidized by the Grant Founda-hologists. Fourth, psychoto-gists are a ' t a good code of ethics must tion. Th.is study is reported by be more than a description of the current sta-Jen n (32). Teachers, administra-tus of ethics in the profession; a code must em-tors and teachers in training in the body the ethical aspirations of psychologist >' Los ageles area contributed--more and encourage changes in behavior, brmamg performance ever closer to aspiration. Fifth, than 1,500cr..itical m.eidentsof teacher . u... Psychologiata appreciate _that pmcess is often behavior. Theincidenta were classi- ~ more importadt thsd prcsiuct in infuencin's fled uniser personal, professi6nal, and ~e human behavior; the four years of widely-social qualities. - The category formu-sharedworkindevelop, g thiscode are counted g.y g. M thit there.wer,e about m 'on 'to be more in5uential m changing ethical practices of psychologists than will be the pub. 20 'diit.mct critical.. ~.a.reqmrements. lication of this product of their work. Finally, These were recommended 'as'a basis psychologists tecngnise that the process of fcir teacher evaluaition and'as an aid 7tudying e.thical standards must be a continu-tNhe in-service gfowth.of teachers. ing one; occasional publications such as this Another study yas that of Sm.th i statement mark no point of conclusion in the mgoing process of defming ethical standards andN Staudoharm (59)y which deter- > w.. o + I

c,r type of application etM T ~ milied tha. critical. requirements for. 0Anoth,iirident technique to the de#p basic traini icci instructors critical f ~ in the Unised 'AiFForce. From velopmemt of bases for evaluating be-t s.130 trainipp 55jtactisarihuvi6r'it the' previously mentioned ~'~ e instructors, an "2 Meic trainees,. study oft Eilbert (7). His list of 51 i a total of 6,615 usable incidents were . types of, immature reaction based on l' f obtained. Tde authoeskromment' la collection of several hundred criti-that: .J_ ^ ical incidents describing manifesta- ! tions of emotional immaturity is be-Thet ning supervisors a predomi-nance dinne6ective Incidents in the major ,fieved to provide a useful gu,de t.o i creas oh Niets a good example and maintains further investigation and appraisal 6f ~ e#ective hersonal reistions. The tactical in-persc5ns with behavidr problems.,lt J is believed.that the resu.lts 'o.., I ceructors pport, mar, smegesve inadents in study prov.je, substantial enco f: thi the area w Makes his es tadonn rieer,. Da-i urage..g. .A nic train show a inance d ine#ec. tive in ' ate in thie= arces; Mets a good es-pteQg t,he.3pplication'oQkbbtlcalhi w 1 l cW4sie,iCMild OefleiM%sedased Main (. Tndfen(*t$hniqil to $si5nifer ' pro ~ d l tains es cjve pogretsti<= jse p. 5). lems in tkneld of clininfdiaan 3ff t Anoth)r study on the evaluation of and evaly@f 'profc'p Dr b} L d.C M easures ri standa. yp~ instructqr effectiveness was carried out by Konigsburg (33). This ettidy semples)? A closely'rEE 4 inv61vedi the developmerhjiof"an inO 5dritklal incide'nsa is to hNE atructor check. list f C 7. L in- ' lor"Evahia' ting the pEr ormIiMEIfhf" 8 ~

atructors haned on t hT*crvowal saa--passana lgt.uas olatandard siinples of ','

cident technique and a comparison behavior involving important as-of techniques for recording obarrva-pet ts of the activity. Such jvalua-t ions. 14s principal findings were the tions are called proficiency measures very low correktioneroe ' ients be-,and stedWerentiated from the evalu-t wcFn " t he total scores' om the ation of typical performance on the job *(imarily on the basis that a tes't l'ordue Rating Scale for Instruction t and the instt urtor checic list. When situation rather than a real job situa-t henc 1wo instruments were each tion in used..\\leasures o'f this sort are ~ given to half the. class oit the same estreially useful at the end of train-7 day, the aver @ correlation coeffi. ing courses as checks on the main-rient was founditi(be.29 The other tenance of proticieney, and; when the principal findidPIs that the planne I tasks auigned to participants vary a performance of a total of 46 pre. great deal in dithculty or are not di-determined behaviors were betAer rectly observed by the supervisors. retlected 'hy the results obtained on . One of the. first applications of the insiructor chNk' fin than hv the britical incidents to the development results on the Purdue Rating & ale, of pmticiency measures was Gordon's A somewhat relatedytudy has been study on the development pf a s.tand. reporte i by 'Birnhairt-d4). This - ard flight check for the airline tians - study collected ii larfntimber of J port rating (28,29); Tgigg~ ~ 0 critical incidents for the purpons of done,by the American;fhatitutetforg., establishing the critical re.quirements Research under the sponsorship of for school board membership. The the National Research CosiisciliCoin-; authar applied his 6ndings-to the mittee on Aviation Psychblogfwithb-l problem cyf evaluating the erective-funds providadaby tha2.Qvil.'Aeer s au w w:.. n o-n

~ WW

R-di[dh d,ata-fr. n....om. analyses of.:a.-ctrii F

i 9. accidents we ~ combined with' riticalT" p~e a m c "**h$$ $ s airlin c 4 k)n & ar M ' J& T developidg".an 7aMi)(cDif.6fi6je'. :,i ,w..~ ^ fli' y. ; li @ s t y w " $m$, ovii.EP.nA .. c: -a. sof... pdop-.n.;profi-. c effe.ct..ive... personnela re ) ions. er eet .g . Ick consis'eifbf"MficN3cy Tesii'u*iM'ddVelopedWili' l VM,.ihriEy t O e presehts i6E. of situationPpro-connection witigl@studp' incl,uded l a#,.h. - t paper-and..:.w m,,. _ mg..un ly. ndard. ai. zed '. . e: ical w.tua ione and ...senting cri.. ned test portunit....i.(,orm.ey x.r.the...,op ecrdical,e t. .e y,id. nw. s pre .a ~ .: requirmg th., e at ies..to w per rm 'n asp $ cts 6f fiFairl e pilot's jokas instructoFIEGIelGtXoste " of several ~ M N ^ 9.. Ano. hen.ohiti#ns.W,aw

h. 4;<

< J'n....dicate.d..s.,~',t.he, study of th,e,iacci propos.edi s.deveTo of.t is type t .w al,m, e. fents reported. 3 . t c.~ dents and cri e u 7 The new chece wk found to-yieldL88: 1.. carried on aht ~ (Ins.titute - r ~ f c struc.t...i.o.,,n: of,. e.d.,. y. to sv o.r Res'e,ar. ent' . mem%c e n n examined onj@wl.,. n r r pi t, enfrests for1.,.. p y pp: - pass r' flights on successive ificiency in? Epii6(r,! ~ .e...y.e"hv., d..i." {..n,t E,h.e.,ck ~ pilo.ts.t 1 %g sheck~ when used.,i.$Ehe ~ O. ffi.P.-.. W . T. h i s w =. . ona M.. H ' liis.

r; 4{d,,pievious fl s es

~ L' c13.4 f,, . <gt '~1*- ., ? N ? 5 e same flig . ave only 63 per centa, was-wi r, kreementMhicl%as r. lp k tterg y g g ~~ hu e a: than chance under the moe).of Aa.pnt oun - ite

f

-4 L g w ' tu y;. h. C.M '* " Y A the study. J;'.%- z-s ofic ere t ps# {

Similar studiesehede_velopmen t,@QescribEi _ (syd gj.Me on d items ical 'research of flight checkg at the American~ int stitute for Research have been carried situation in cons

! detail ~ and eE' 3 "'is c'oncern-tent hv Mafley (36.s.37), G. S. hiiller#yes$~ined five sp'EkthEbeint'hifig (39),[ind Ericksen (9). These studies, Ing such matte dai nest, suggestimIToYiiiiihibvigg 7 spo sored by the lloited Staten?. Air Force ami the Civil Aeronautics Ad-the procedure as EepMed, ete!.The minist rat ion, were concerned re-critical behaviors ' tested in theltems spectively with. objective tflight were taken directik from the critical checks for B-29 boYnbing crew mem-incidents. The method of developing bers, B-36 bombing crew members, tests through the use of comprehen-and pri " ' a flying light c' ilian sivq rationales has been discussed w aircrafi' n also developed a genbrally in another paper (16).- ~ light pla , iciency check to pre-Three studies have been reported dict military flying success (10) on by the AmericanE!nstitute for Re-a sinillar$rbjEct sponsored by the ' search in which %rifical' incidsnts F United States Air Force iluman.Re-were used as a basis fod~)levelosiing situational performance ~ tests F for mources Research Center. A"sirnilarist of praticiency?Id.n6A-measuring certhiri"spe(s 6f thd$ro- ~ ~ i ~ ures was' developed by Krumm. for ficiency of m.ilitaryTpefsonnel. These u GAir Fordiiif$iipinstructors (34;i35),w included the'ag "of' ~~ iid Ilhge ~ istf l REfetio~jEtive . ~ also?dsikctfIU@msorship ~'of^ilie ~ on "thfdevdo idTest HumLi 1(esources Research Ceitter. form %f' thy These me'asu'rps!were based on more for thd Persof MEEBf6Bidth, DepE mMt1 fArQ('5%,This than 4,000 critiEat incidents collected 3 from student pilots, flight instructors, test includidr20" 'fi6naliproblems u ami superg. pas.,2The criticaLMdis. based? .rdsissie'speniiof ^' ~r a g my

.. J. m mg y g g. g ;3 y y 4,

q..;.. n. s m. aw s.g

w., 4_.m .. &., W.,.,6 '

t% %,

b i eencomm' ned; comfat' infan-x.. Air., orpe Saman Factorippera @:n 4 Caboratory. ~ Oh 't., _

m. j]

i = try leadec. ~ sained on th(basis;M.s9yAthousand+inckWnts( grg ngicomba,IN .II I ',;.9.CH t ported b, y a,ircrew$ personnel regar. - -> v. i taryj operatiess a ...m. Korea." A_ ing emergencies, three evaluation: 2' profi',e, cy0 measure tolya devices, ere preppred. These ink.1 ~' ! secodd.. ci n eveloped h volved.a;w. conventional type multiple- ~ ~ somewhat similar sort , idiother typesW* ly?Uif. choice 3est;?i spec'ial multiple-choice : J ~ WeTaloni (75f8[5 (61) for,i the...s ciu. fried._of stddy ffa' die' ~ t den $Fd to measure the indi.' ' J i v man Re..t.e? by S.u,ttelF.,.v idual"sl,. inform. atlon concern ~ this t. ype wa u ~C ,u -w w u .Hu.~. e sources, Rearmportant4 cues,m theJemergency search (iinteel This'enly was basei$.$lisjtuatI6N appropriate $tions'to' b on criticht incidents i:pliected in,, pref 4.be tak'ei@and.the basic t ~ YiousM4 dies ' o f :the-Ar6Erican "InistiQ ciuses "s emerg{" l-tute W ReseaI0cWaQd+ryotN4l. development and prehminary eval]ua QQ . jbR i 't ance' 9ff 7 he OfficerI,SitEitihnIT t was -designed ;lm..m_el & r. tion.of tv.. ...m

.n p

This tes cts'd,o. m,e.as, w nonintellectual:aspe.. g er me f6rpiarg theistigh(( % Nd gjJ "' ~ situational problems requ . d,,:......!.i. v. J C.L u 5 D Because of the great '.difficuliy in.1iikg maYeri.de$m#s?7@?W I " six hours of test'ing tim-r' obtaining! valid and reliable ' measures A recent study by Collihiij5 siiesi critical incidents as a basis f6ite{fals of typical performance, accurate measures of.proficieg.ar,e.9% fappgg flectiveness of<aftraif(isg.P for many types of activn$s. It is program.- . types of mciden ts.re- - ~ ~ apparent that a compr'ehinilVe set of. ported hy mothers af ter a two-week critical incidents cantheinf' ' great e training course were significantly dif-value in constructing such meassr.es. ferent from those reported at 'the Training. Mans $hejpplica-beginning of the program in a number tionsof tbecriticalincident techisiqse of aspects relevant to the objectives to training ' ' ms liase bdk car-of the program. The critical inci-ried out ' f [ military in special dents appeared to provide a much nit uations?. t.7 the reports-are. more sensitive basia for revealingL classified security" information. In. changes than other procedures used.. addition to work by l'reston, Glaser,- Selection and classification. Uritil and R. I.c Weisloseli Ru B. Mille #andWrde'ntlyr the niistomary apipkach' f / o Folley have utilized' critical inci.-,.,the research-psychologist to the 4 - ~ 9 dents in establishing training require ~ 'velopr'nen. t of test's fhr sele.c. tion,.YrW m ~ fo~r specific. types of mags. cla:.. d. ic; ationipurposes has. be. ments ss e ..a. r,.. ..e~ - 3 ~ ' ;......, $nk 4-5:." tenance m ses 47 in a study, "fa< -re w yA v*.~r ery;.'s.,r. b..r. - .* :w - s' r ry

.*w a,.o

-.?

y t h.e H um.t %

nW. sl~' 9f = o .t -a e n w.mm+n. :n.. ..a Similarly, Ronan h'as und. q,., ca wide vars...~ critical fwas:adminis. ~ U N., - ~ s ety o se ection. ures e. ter. incidents as s3 basis for. develop.idg%....da..,n. pi.ica. - k s/sindjoiksif.upo.

w. c program of tra,imagj.fo.. w.r emergency <

tywerggathety,a Sg...cth y,.,p,L va procedures m; multi-enguie aircraft.. search.psycholog .litt W (54). in a et.u.d..y for.t.he.:p. nit.e. d4(< ate.sS. r-w -Ma A ) C U ed$iin a j c. g ..a.aY$?0$b .* YW - 2Y* %.... C OWY'

N lh pay ological' iits

b,,3kerea,waggendyjobjelement:were inductiveIf[

in-)- - ....qncy; . rpthing 'that,.Lwas lated from theicritical incidents arni^,' 'Taiaifi..., einRgven; remotely-were ' grouped under.the four aria-drIelated3. Isinvolved5This headinks: (a) learning and thinking; ' MasJ beeii-Iled the " shotgun ap-(b) observation and ' visualization; i.dh{ch,g]thyrasbpe(tjiiMwitga;,;((c) sensorj.motorjoordination;' and I gwEle. scatter at Nast a ikw of d) motives, temprament, and lead-ts. .sdould pay off. The criticai'incid t ership. '.i. ac; t.e.c..l nique has lent su.bstantial sup r T. h.e dev.<el.op' ment of more than 100 e i. sj qport to ihe Meore thorough, ~ stud,y,of, proficiency tests to measure eac

v. ~

. of ' the job prior so initiating testing pro-thpvarious. critical behaviors I ~Yedurds.' Tliere is increasing feeling cluded in ihf 24 tedEisel r6 posed '?TMfthMpnserWtifrfedthstMidEidh@j6b1EfinisiitFFdiff0ii 'v p'y. Hhhn 'b "!arger ' percentage of the 'investiga-(30) for the:. School bf*A' icii Medi-J k for'stimefaho'uld bA sfent on deter- ~cin'. W iedsET"r.,

TiisthFdt g

~ e r +s s ~ minmg the cr.~.itical requ.ireme.nts of to,a groug7ap y. m m.. . J.tfjob, so...that;the psychologist,will ;schoolym he.:in, 2 tentative conclusions as to tk._ure .igpereaus efor j..c " haves sufficiente confidencegmyg' the tentatIEM&%m rw a of the importan't' selection procedures attidy justieftsplefedggayl' 2)! to permit their use on a tentative. for the-HG"nianLR$liources3 %a'rch-R basis prior to the collection of em, Center, the"rssultsjoff applyihg an pirical follow-up data. This is espe-analytical 'procedureideveloped by .cially importgnt in,, those sit,uafionay., Herst..to study thekinteprelationships' where the fairow-up requires a'very involvnd are reporte4 This analysis long period. of time or where the Idi to the formulation of'a nei set of'/ number of cases that can be followed 20 job elements f5Feach oFwhich aA up is so small that definitive findings selection test has been developed. cannot be anticipated. These tests have been administered One of the most important re. to several hundred aviation cidets quirements for developing a system and follow-up data on their success in of job analysis that will facilimte a airc)ew training should be available relativelp4acc, urate identification of sooA. s the import [nt.. job elements. for a A similar project based on critical specific tasFii to establish a clear and incidents collected from various ci-specific'se'"6f definitions for'these job' vilian ' jobs has been reported by' the t elements in behavioral terms. The present author (2,18, 19). The American Institute for Research has Flanagan Aptitude Classification carried out a series of projects on this Test Series, published in.1953, pro-problem. The first of these was a vides aptitude measures for.14 criti- ~ ag.Thedpplicant , n. stSdy undettaken by Wagnerlu'nder" cal. job elei((le.puidisfiEdm . - the spons' rsh'ip of the United States Inventor {ffa o " Air F6tce School of" Aviation Med( measur@s'dtitude(pfedictNeJof job adjustme%s~..for.diiihrly' cine to define the requirements of airs crew jobs in' terms of specific job An.ifffr d elements (67, 68). Several thousand , critical incidenp were gatheredfromp..cidentjaircrew me'mberg and ~ ~ 'jce examinations w. y,m

2. p p.b E; i

.y* 7 ' $'$ * " y - o'

' y gyrf. ~ . :q,.. c

a. +.,,,;g y

c xihDBg%t.ed R.k.%' Mgs4virab9asksyare invoheddpj[dQL[,, y (64); ~ la in, {retop.atudh they jobs have been studied by use of thM ^ collected abOtt Qcidents regards critical incident technique, it may be ing on-the-jhavi5(s[of Emaint possible to select and train each team tenance technicians. These incidents member for _only two or three of the i l critical j[ob elements..This tend were used as a! basis for' determining niahimiz theeffectiveness of per-joliIestiirdin5Ms inieYrki of b6 I formanceiwith respect to each of the havior,jand test items were developed 3 various-XypesL of. tasksc Although such procedures' have nearly always for une in selecting applicants who could.be expected to meete these re-quiremdnts, Wy been informjdly used. i1 planning the l Anotl)er study that used critical work of ~ teams; the (ritical~ incident" ;, ineldsts aN basiss fdrafevelopinitr teWhniciOMfiicilitii;ref thRrbilntion ofk ~ tests to predict performance was rar-the data (eidsntiAI.to t@ift71p( jdlAH 24 f.K < purificatiii@$[hWi$yEd ried out!by O'Dnnnell151.);l liin test; Somef un nayt. . j.)li' -2 designed ta prodirt surcess in den-tistry, was based on triycal incidents prnblem p,ag fat.t colheted by %' saner.N The eent in. America E ssearc P cludes itema designed in hiedict, in' RecomE ions fine Isik fr 1.c denumntratmg 'tet4mic:dM $Mn.CfMu$$numifed JiiG following three general thAie iludiFs~forisiti f. i pa rt, the (i) .. ream: proficiency; (b) handling patient 're-njmmon 'mainte5$rdeljobi ~a~ ret la tionships, and (r) at i epting pro-pet ted to lead to a saving of millions..., femional respgmsibilityr A. foliow o p (danars in raining.r.osts as well-as, study indicated moderate Validitv for to im ing the effectivenene of job l t heisF noterials, performam.. 'H One of the few studies known to Opefat:Ing prochAnother ap. f t he author m whi< h the critical inci-plication of critical inrillEiidhi/h j dent tes hmque was ured in a project has not been adequately exploited is l rarried on outside the United States the studs of operating proced ures. l in Emons' doctor's diuertation (8). 1)etailed fac tual data on successes l This studv. carried out at the IIni-and failures that can be systematic. versity of 1,i6ge; investigated the ap-ally analyzed are of great imimrtance j titudes of effective sales perwnnel in in improving the etTertiveness and a farge departnEnt store. A grmrp of etliciency of operations. Such infor-40 supervisors"provided 228 critical mation can be efficiently collected by incidents. Nine rategories were for-means of the cr.itical incident tech-mulated from this group of incidents nique. and recommendations made for an ' Examples of such studies are pro-aptit ude test to improve current vided by a ser.ies of three projects ~ carried ouAby the American institute. seleetion ' procedures. - (

5. Job design and' purification. in-for Research under the sponsorship adequate attention has been given to of the. United States Air Force 4.:

the scientific design of jobs to pro-School oCdAviation Medicine.' The ~ mote over all efficiency. Where a first of these involves the collection of.1 team has several different. types of critical! incidents relatingTro near_ac? tasks to perform.it is frequently pos. cidentilaj.ftying reported by.Vasilas,, sible to design each of the team mem-Fitzpatric,1@.DuBoistan'd Youtz (63)a ~ m,um mm. m.a

.. n ,.wNb bnditl6F "repf5sfit$i IINhib7 m. [hd ence provides a sound basis fdirhdiliC po I~ ' by procedures d:- ' ' %jircrd, mem d,. These inci fying existing cquipment and desigrF eloped.;f' lents$m '_ble. improve-ing new models.. men ts"i" job design and ain the study by Fitts and Jones (12), ' mentioned above, which was 'ing prSci ures.T '".l'a~s'in~operat? " N dip carried out at the Aero-Medical

The second1of these.. studies was

. Laboratory, 270' critical incidents spefiScafly $cerned;with the effect relating to errors in reading and in-of the. age of pilots and othe,r crew .terpreting aircraft instruments were members on6irciew operations. This collected and analyzed.. These led to . 4 uils wayipgrted Ay..ShriverJ56h.. a number..of.sp'ecific. suggestions re-ind included ' tentative suggestions garding modifications gstrument regarding:. various modifications in displays. M Other recent;jtudies conducted!at ' hperating ' procedures /# the American Inftitute lovResearch. -( , Th' thirdi study in this series, re-have use'd'ditMrWtliWcNdcal.iEt e .'. DWWAN.NdQhdenough and 'juttell 126)CmvoivedphIcollEioifoiEitil ~ cident ' te'c@'depisp3 t sources ktoe . nress g3 yalgiricidents $garding the impair-menf;of hnman efficiency in emer-designing jobii lier on t er icEc. iVipbratitins ~These incidents'"pbiiRtr"ar "... y .. P.. m ili .4 ~ .a c security reas6n,w pe.%hs .~ - arovide a detailed statement of both e 3 the, types of 8 tresses that impair 1y *', ether profic(DaDille?Aiurica'nl forhWinceand 'the' types' of' pei'forntA.4astitutf fin Pe%S.Yave' used thi t ance that are impaired under specific critical incident ' technique as a sup-conditions. More than 2M) critical plemental procedure for task analysis incidents were collected in which of equipment in the design stage of I impairment in performance or cpera-development (9, 10, 34, 35, 39). tional assign ment s was observed. These procedures have been found These incidents were collected in very etTective when,u d by# psy-Alaska and the Far 1:ast as well as in cho ogists working clu with 'en-operational commanda in the I.Tnited gin rs on the preparati of design States. This report contains sugges-spt. itirations for new equ! pment. tions for improving ' operations in Motivation and leaderzhip. The emergeng jations. study of attitudes has been somewhat ~ EquipmeJElign. An application limited and difficult to interpret be-clor,ely related to that just discussed cause of the almost exclusive reliance involves the collection of critical in-on verhal statements of opinions and cidents to improve the design of preferences. The critical incident l Repnrta of specific in-technique has been applied in a few . equipment. ridt rits 'froni the field have always instances to gather factual data re-i been a basis for equipment moditica-garding specific actions involying de-tions.aThe critical incident technique cisions and choices.. These studies facilitass the" collection and process-suggest that criticaliri6idents ofrthis ing of'this type of i'nforma' tion: Too ' type may befv'ery valuable supple-often in the past action was taken on mentary tool for' the' study of-atti-the basis of informal reports from rudes. operating personnel. The coIIectinn A - recent' 'stddys. carried I out.. by of large numbkre of critical inci lents l'reston of thn A,merican Institute for 4, d) ' Q 4

y } yS*

Tdh.. t c T. JOHN CdLANS

. %$$4

D S

@Q. % .!FA v, "Jj@Research for..therAir, Force's. Human,' velopiagro 'ctive ns 2 m Resources Rese'slrch Center (53) usedJ proveminit@ut also in "exp'eriment4

critical inc

hInsis for study-studie's M the types of improvement a ....dng gleep f$d@t;%A%tig$om.the therapista' use of /j .y. proc"edures. the, Air heved that specifl.~c 't'hese speci' . tijkovide'valuZ.; A somewhat related type of study ;t cble information;'Eot contained int initiated; by.Diederich and reported ' ~ studies utilising only? data on opin.. byl Allen (1) describes the use of the. techniiuie;,to obtain critical incidents ions. j ed 7 g g.: W s.n n rpb7jfucf(55f Goin'studdsts reporting things tha l vp contains critical incidents on combat caused 1them'to like a fellow high leadersihip collected from senior offi... school $udent either more or. less cers in lthe !F5r East Air F6fcefsifthin tisifef This study is bein F These incidents proeide. a factual. tinued $ basis for the study'of motivation and of spesiMfpovide the basis forit,estsvild r leaderskhosof iAiqq 8persbrmRyfMidin$sMt%tirdy4a( in$$hefQs; engaged Bn combat 1 perations. - W i an'exarn lof >the" kind?. tide Y3 Cownsping and phe44 therapy'. An.imdesireg".[, $wn* on[d? T 3:f other fiyd in Iwhich"cuinnt teci gier con 6fithe positiv 21 if ' niques emphasize,over-alt imprek@ cent df.Y,.hMnegai:ivdb sions, opinions, and' reports of singhP;giorted"were(inI.the 'same ,l i N cases is counseling"and psychotherF the exai6Epliji~v'in',' "7 1 apy. There appears to be a trend, .P e P N however, in this field toward em-SU""i""do CONCLUSION $ ~ phasizing the collectioIri' of" factual This review has described' the'de? incidents. This suggests that the velopment of a method of studying critical incident techr{ique ma.y # c'al' ine,derW thchnique.be... activgs useful in this arealiso. f' The tech- ~ E*ploratory fworie 'has' recently nique grew out of studies carried out... l ~ been done at.epe University of Pitta. in the hviation ' Psychology Program 'j burgh with the' critical incident tech-of the Army Air ^f8nscQWgrid 1, nique to establish areas 'of change War II. The success of the mend L at i om pan ying.. psychotherapy. .A in analyzing such activities as com-t peries of thregaster's theses were bat leadership and disorientation in } . carried out by;fpeth, Goldfarb, and pilots resulted in its extension and 1 M ellet t (25, 38#40) They collected further development after the war. 243 critic:tl incidents from 11 psycho. This developmental work has been ~ therapists. These incidents were cel-carried out primarily at the American lected about patients who had showns institute for Research and the Uni-improvement and were replies to the versity of Pittsburgh. The reports of question, "What did.the patient do; this work.are reviewed briefly.;.g. that was indicative ' of improve. The 6,ve steps included-irt thk: sit- / ment ?" A&though:t$ese studiesIwera&ical~~ incident" ~prodediMe. a5%iis$t i primarily exploratoryiin.; nature,-ther cessimosityassed at 'thf"pi'eeent Jtime? 4 tentative nnding thatidifferent therac 'aife" discussed. These are'? a follows:% pists stress different Eriteria of im ' (s) Determination of-theMneraiaiim provement and nonimprovement sug-of the. activity. This. general. aine gests that the critical inciderit ap-should be.a brief statement.obtained ___aua x. n o -e u 2 --

~' ~ u355 THE CRITICAL INC 0 ~< q~ hiEh'cx%e'ss$p:.-fin sispifi$(m.~ithosAwp rx+ . i .m s m w cmp a criti - cbjectives to which most peopla e[al*Edide*nt6e~ resEn rawMia p F7dnide Qvelo ~ g jf]hydo%ofautohistEl would p ,icinIf"6Fh ectingh18td?I8ds*t'o%bfeniC owe'vEa * *Didits" factual ' egarding; the ac. procedure which assihts in collecting tivity.M ructions to. th'ei pdE-1 ~ representative sampleti of data that sons who are'io report their of>serva-arp 'directl_y. relevant to' important dions need to be as specific as possible problems s6ch as establj,shing stand, with respecbto;the' standards to be ards,l determmm requirements, or used in evaluating and classifying the evaluating resuit Id have wide behavior obiierved. (c) Collection of' applicalgility. the' data. The incident may,;be re The. applications of the critical ported in as interview or written uri. incident technique whicir have been by the observer himself. Jn either. made.to date are discussed und.er the - wy.. case-it.is;.csyntighthat thegacorting d ollpwingsnindshea' ding 1(a)eMeas-f be objective and include'alF/elevant -uresof typihalp,erforrha "riteiia); ~ The purpose.of :this an'alisiIll dafa. JbQeasureisoQgficieAEY details. (d)~ Analysis of thE s'tandard j isT latnpf5ER$)?trdinilig;T(d)' summarize 'and describe the data In' 'and slassificiGisE{(epj Pand 4 an efficient manner.so that:itican b5 purinEatiorI;3$per E6ed-h)drioti / ures;ig) (quipyneht design: (st6 effectively used for various practical variah - and"lii'dership+(di purposes. It is not usually possible to obtain as much objectivity in this (i) counsel,gg, ahd7 psych 6thcTapy. ~ step as in the preceding one:V(e) In'.

Irr summaryi the critical incident /

terpretation and reporting of the techn~ique,. rather th'ai' collecting statement of the requirements of the.. opinions, hunches, and estimates, activity.? Th51kissibiFbisses$til iHs %tains a re7dTdjf specific behaviors ~ plications of decisions and procedures from those in tIIc' best position to made-irr EcTi of the four, previous rnake the npcessary oBnegvations and steps should be clearly reported. The evaluations. The collectiori*and tabu-research worker is responsible for lation of these observations mate it pointing out not only the limitations possibie to formulate the critical but also the degree of credibility and requirements of an activity. A list of the valuc of the finai results obtained. critipal behaviors provides a sound It shou 1$E noted that the critical basp for making inferences as to re-incident tihili'nique is very flexible and quirements in terms of aptitudes, ^ the principles underlying it have training, and other characteristics. many typ;esyof applications. Jts two it is. believed that progress has been basic princi'ples may be summarized made in the development of pro-as follows: (a) reporting of facts re-cedures for determining activity re-garding behavior is preferable to the quirements with objectivity and pre-collection of interpretations, ratings cision in terms of well-defined and and opinioibased 'on general im. generalf psychological +@ategories. ~ pressions: (b) reporting sh6uld ~be MuchiremainsTt6a be ' donec it is limited to those behaviors which, hoped that the critical incident tech-according to competent observers, -nique and related. developments will make a significant contribution to provide a stable' foundation for pro-the activity. T cediepin ma6y areas of psychology. .$ l N/. M$NNdb E i .mm _}}

ML20238A428

Text

Navigation menu

Search