ML17333A551

Ltr Rept, Evaluation of Cook Ipe/Hra Matls.

ML17333A551
Person / Time
Site:	Cook
Issue date:	12/04/1994
From:	SWAIN A D AFFILIATION NOT ASSIGNED
To:	NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES)
Shared Package
ML17333A549	List: ML17333A548 ML17333A550 ML17333A551
References
NUDOCS 9609110071
Download: ML17333A551 (18)

Text

EVALUATION OFCOOKIPE/HRAMATERIALS LetterReporttoU.S.NuclearRegulatory Commission OfficeofNuclearRegulatory ResearchByAlanD.Swain,Ph.D.712SundownPlaceSEAlbuquerque, NM87108(505)265-0098File:ga/NRCQCookHRA.943 Submitted:

May10,1994Finalized:

December5,19944555S.MissionRd.//967Tucson,AZ85746(602)294-17699b09i10071 9b090bPDR*DOCK050003i5PPDREnclosure 3

EVALUATION OFCOOKIPE/HRAMATERIALS

1.0INTRODUCTION

NRCstaffaskedmetoreviewtheHumanReliability Analysis(HRA)portionsoftheCookNuclearPowerPlant(NPP)Individual PlantExamination (IPE).Ireviewedthedocumentation listedinAppendixA,visitedNRCstafftwice,andheldnumeroustelephone sessionswiththem.Specifically, astheprimarydeveloper oftheTechnique forHumanErrorRatePrediction (THERP),Iwasaskedtodetermine whetherCookHRAwasbasedonappropriate useofTHERPasdescribed intheHRAHandbookbySwainandGuttmann(NUREG/CR-1278)

(Ref.1-references areatendofthisdocument).

TheCookHRAdocumentation statesthattheHRAisbasedonTHERPandthehumanperformance modelsanddatatablesintheHRAHandbook, andalsoonNUREG/CR-2254 (Ref.2).IwasalsoaskedtojudgewhethertheHRAiscredible(believable),

andifnot,whatproblemswerethereintheHRA.Thefirst4pagesofthisdocumentsummarize myfindingsandconclusions.

AppendixApresentsmydetailedreviewoftheCookHRA.Thesefirst4pagesandtheSummaryinAppendixAareverysimilartomypreviously completed evaluation oftheZionNPPHRA(Ref.3)becausetheHRAmethodusedandtheproblemsfoundaresimilar.2.0APPROACHUSEDINTHEHRAThissectionparaphrases someofthemainpointsintheapproachusedintheCookHRA,according totheIPEdocumentation providedmebyNRCstaff.TheCookHRAisbasedonwhattheanalystscalledastep-by-step taskanalysisofoperatoractionstobeincludedinsystemeventtrees.Foreachoperatoraction,theanalystsidentified ActionIdentifier, Description ofAction,TimeWindowAvailable forAction,Applicable Procedures, andIndication ofWhethertheActionwasSimulated inTraining.

Genericmedianhumanerrorprobabilities (HEPs)foreachactionwereobtainedfromvarioustablesintheHRAHandbookandconverted togenericmeanHEPsbyusingthehandbook's assumption ofalog-normal distribution andestimated 5Xand95Xerrorfactors.Assessment ofHEPswasdoneeitherbyfaulttreeanalystsorbyHRAanalysts.

Whatweredefinedasplant-specificperformance shapingfactors(PSFs)werederivedandassignedscalingvalueswhichservedasmultipliers ofthegenericHEPs.Themultipliers forstresseffectswere1.0to10.0,andthemultipliers forotherPSFs(e.g.,availability ofmultiplesupportive indicators, selection ofwrongcontrols, qualityoftraining, memorized procedure) generally wereassignedmultipliers oflessthan1.0,i.e,1or.Ol.Whenestimates ofdependence effectswererequired, thepositivedependence modelfromtheHRAHandbookwasused.Theanalystsemploydafault-tree approachratherthantheHRAeventtreeapproachdescribed intheHRAHandbook.

TheHEPsusedinthesystemeventtreesweregenerally calculated usingindividual faulttrees.Aspartofthequantification process,aplantvisitwasmadetoobtainoperatorreviewandassistance aboutthemodeledhumanactions. 3.0MAJORPROBLEMS3.1AssumtionsFollowing aresomeofthemoreimportant assumptions statedintheCookHRAdocumentation thatarerelatedtotwomajorproblems.

ThefirstproblemiswhatIconsidertobeoptimistic assessments ofthecontribution ofhumanerrorstosystem-critical events.Thesecondproblemistheacross-the-board assessment ofarelatively smallsetofHEPswithoutadequateconsideration ofthespecificsituations andtheunderlying specificPSFsforthosesituations.

Amajorassumption isthattheCookNPPoperating crewsaresowelltrainedintheuseofthesymptom-oriented EOPsthatamultiplier ofO.lwasappliedtonearlyallofthegenericHEPs.Iftheanalystsjudgedthattheprocedure tobeusedwasmemorized, thismultiplier of0.1wasincreased to0.01.Thus,therelatedHEPwouldbedividedby100.Ifindthisoptimismincredible.

Moreover, theuseofthesekindsofgenericerrorreduction factorsisapparently notsupported byaspecifictaskanalysisforthetasksinvolved.

Atleast,nosuchtaskanalysiswasincludedintheHRAmaterials Iwasgiventoreview.Considerable creditforrecovering fromoperatorerrorsisgiventowhatiscalled"checking thatinvolvesactiveparticipation (STAmonitorsplantparameters)."

TheusetheHRAHandbook's Table20-22"Estimated probabilities thatacheckerwillfailtodetecterrorsmadebyothers"isinappropriate toapplytocheckingduringanaccidentsequence.

Theconditions forusingthistablespecifically includeastatement thatitshouldbeappliedonlytonormaloperating (pre-accident) conditions.

Application ofthistabletopost-accident conditions (i.e.,thesituation following someinitiating event)couldresultineithertoohighortoolowestimates ofHEPs,depending onthesituation involved.

Also,themeannominalHEPof1.6E-2thatisbasedonTable20-22isfurtherreducedbyamultiplier of0.5,togiveamodifiedofSE-3.Asthisnumberisusedtomultiplyalltherelevantoperatoractions,itisaquitesizeableerrorrecoveryfactor.UnliketheZionHRA(Ref.3),itwasassumedthatdiagnosis errorswereimportant intheoperators'esponses toaccidentsequences.

However,thegenericHEPsfromTable20-3intheHRAHandbookweremodifiedupwardsbyafactorof5formoderately highstressandthenreducedbyafactorof0.1fortraining.

Table20-3alreadyincludestheeffectsofstress.Althoughthroughtheappropriate useofTable12-5intheHRAHandbook, itispossibletoreducediagnosis HEPsbyafactorof10,theconditions forsuchreduction asstatedinthistablearestated,andfurtheramplified andclarified inNUREG/CR-4772 (Ref.4),whichisamorerecentapplication ofTHERP.Inmyopinion,theCookHRAdocumentation doesnotprovideanadequaterationale fortheiruseofareduction factorof10.3.2ModelinofHumanBehaviorForafull-scale HRA,whichtheCookHRApurportstobe,itisnecessary tomodelthehumanbehaviorcorrectly foreachtaskorsetoftasksthatcouldhaveamaterialnegativeimpactonthesystemfailurecriteriaofinterest.

IntheTHERPapproachtoHRA,thisalwaysmeansathoroughtaskanalysisforeachrelevanttaskinitscontext,asisdescribed inChapter4oftheHRAHandbook.

AllofthePSFsthatarelikelytohaveamaterialeffect,onataskshouldbeincluded.

Thetaskanalysisbasically stateswhatleadstowhat,andmovesforwardwithtime. Itisaninductive method,ascontrasted withthefault-tree methodology whichisadeductive methodworkingbackwards inasequenceofevents.Whileitistheoretically possibletoperformanHRAusingtheinductive fault-treemethodology, thoseofus'withconsiderable experience intheHRAfieldhaveelectedtousethedeductive event-tree approach.

Eventtrees,especially theHRAeventtreesusedintheTHERPHRAmethodsince1961,makeiteasiertographically represent thenecessary underlying taskanalysis, andtorepresent allhumanerrorsandhumansuccesses asconditional probabilities inwhichdependence effectsarerepresented directlyinthetrees.Theinfluence ofimportant plant-specific PSFsisincludedineachbranching intheeventtree,andisbasedontheparticular contextinwhichthepotential errorcouldoccur.Inshort,theassessment ofconditional humanerrorandsuccessprobabilities foreachrelevanttask(orstepinatask)is.basedonafullconsideration ofthecontextunderwhichtaskmustbeperformed.

Thisapproachcanbecontrasted withthatusedintheCookHRAinwhicharelatively smallsetofgenericHEPsandgenericPSFmodifying factorswereuseddespiteapparentdifferences incontext.Ifoundnoevidenceofanunderlying taskanalysisofthetypethatIwouldfindacceptable forafull-scale HRA.Iwouldcharacterize theCookHRAasacomputerized, mechanistic approachthatdoesnothavesufficient flexibility tofullyincorporate theeffectsofplant-specificPSFs.3.3AlicationofTHERPTherewereobviousmisapplication ofTHERP.Theseincludeuseintheirpost-accidentHRAoftablesfromtheHRAHandbookintendedonlyforHRAofpre-accidenttasks,useofgenericmultipliers ofO.loreven0.01asadjustments toHEPstakenfromtheHRAHandbook, andadjusting thehandbook's nominaldiagnosis modelforstresseffectswhenthatmodelalreadyincludestheeffectsofstress.Themostseriousmisapplication ofTHERPwastheapparentassumption thathumanbehaviorcanbepartitioned intosmallunitsofactions,theHEPsofwhichcanbeincorporated intofaulttreesasindependent basicevents.Ingeneral,itappearstomethattheseveralmisapplication ofthefull-scale approachtoHRAdescribed intheHRAHandbookindicatethattheanalystsdidnothavemuchunderstanding oftheTHERPmethodorthehumanperformance modelsintheHRAHandbook.

3.4Traceabilit Oneofthemostfrequentcriticisms ofrecentHRAsisthatwhattheanalystsdidisnottraceable.

IhavespentagooddealoftimetryingtofigureoutexactlyhowtheCookanalystsarrivedattheestimated HEPsreportedinthedocumentation.

Inseveralcases,Ijusthadtogiveup.Eventhoughtheanalystsusedfault-tree technology intheHRA,nofaulttreesforspecificanalysesweredisplayed.

Onehastolookintotheequations andinducethefaulttreeinvolvedorconstruct onesowneventtree.HRAsshouldbewrittensothatindependent reviewers canreadilyandaccurately evaluatewhatwasdonee

4.0CONCLUSION

S Myreviewofmajorportionsofthedocumentation ofthehumanreliability analysis(HRA)intheCookIndividual PlantExamination (IPE)revealedseveralmajorproblems.

AlthoughtheanalystsstatethattheHRAwasbasedontheHRAHandbook(NUREG/CR-1278) anditscompanion

document, NUREG/CR-2254, manymisapplication ofdataandmodelsfromthishandbookoccurred.

Thereweremanynon-conservatisms intheHRAs,especially intheunusually largeamountofcreditassessedfortrainingandexperience andfortheemployment ofsymptom-oriented emergency operating procedures.

Therewasfailuretofullyconsidertheeffectsofwithin-person dependence inperforming tasks.Manyoftheestimated humanerrorprobabilities (HEPs)werenotbasedonsituation-specific performance shapingfactorsandcontextofthetasks,especially tasksthatareperformed morethanonceinthesameaccidentsequence.

Thisproblemisexacerbated bytheuseofglobalorgenericestimates ofHEPs,whichresultedinarathermechanistic approachtoHRA.Finally,traceability ofwhatwasdoneintheHRAwasmademoredifficult becausethenecessary information waslacking,notclearlywritten,ormostlydisplayed inequationformonly.Myoverallconclusion isthatmanyoftheestimated HEPsarenotcredible.

Ihavenoconfidence thatanoverallsystemsanalysisincorporating theresultsofthecurrentCookHRAwouldproperlyassesstheinfluence ofpotential humanerrors.

AppendixADETAILEDEVALUATION OFCOOKIPE/HRAMATERIALS (Draft1)Thisappendixprovidesmoredetailedcommentsrelatedtomyreviewoftheapplication ofTHERP(Technique forHumanErrorratePrediction

-seeRefs1,2,and4)intheHumanReliability Analysis(HRA)portionsofIndividual PlantExaminations (IPEs)ontheCooknuclearpowerplant(NPP).Thisreviewisrestricted totheCookHRAandrelatedmaterialmadeavailable tomeforreview.TableofContentsAcronyms&Abbreviations I.SummaryII.DetailedCommentsA.CommentsonSection3.3.3HumanFailureDataB.CommentsonSectionI.TimingConsiderations inHumanReliability 9AnalysisC.CommentsonAppendixtotheAttachment toAEP:NRC:1082F 10References 13Acrons&Abbreviations ANNASEPEFEOPFTHEPHRAIPENPPNRCOAPRAPSFRFROSTATHERPVTTAnnunciator AccidentSequenceEvaluation ProgramErrorFactorEmergency Operating Procedure FaultTreeHumanErrorProbability HumanReliability AnalysisIndividual PlantExamination NuclearPowerPlantNuclearRegulatory Commission OperatorActionProbabilistic RiskAssessment Performance ShapingFactor[Error]RecoveryFactorReactorOperatorShiftTechnical AdvisorTechnique forHumanErrorRatePrediction Technical Institute ofFinland A-2~I.Summar1.OneoftheNRCquestions tomewaswhethersomeusesofTHERPintheCookHRAwerenotinagreement withtheprocedure described intheHRAHandbook(NUREG/CR-1278, SwainandGuttmann, 1983-Ref.1).TheNRCstaffidentified whattheyjudgedtobe"atypical applications" ofTHERP,andfurtherstatedthat"Thestaff...cannotconcludewhetherthepotential for'erroneous results'omes fromtheparticular wayTHERPwasappliedbyanindividual consultant

...orbythelackofunderstanding ofTHERPbysomelicensees."

AfterreadingthematerialontheCookHRAthatNRCprovidedme,Iconcludethattherewereindeedseveralinappropriate applications oftheTHERP/Handbook approachtoHRA.Itappearstomethattheseveralmisapplication indicatealackofunderstanding bytheanalystsofTHERPandthehumanperformance modelsintheHRAHandbook.

Thefollowing threeexamplesandothersnotedlaterillustrate thispoint:Thenominaldiagnosis model(Table20-3oftheHRAHandbook) wasincorrectly used.Theanalystsdidnotunderstand thatthetableisbasedontimeavailable fordiagnosis, whichisobtainedbysubtracting post-diagnosis actiontimefromtotaltimeavailable forthehumandiagnosis andactionscombined.

b.ThespecialrulesusedtoadjustthejointHEPsfordiagnosis ofeventsfoundinthenominaldiagnosis model(Table20-3oftheHRAHandbook) arenottakenfromtheHRAHandbook, and,asnotedinitem2below,application oftheserulesresultinsubstantial and,inmyopinion,optimistic downwardadjustments ofdiagnosis HEPs.HRAHandbook's Table20-22"Estimated probabilities thatacheckerwillfailtodetecterrorsmadebyothers"isinappropriate toapplytocheckingduringanaccidentsequence.

Theconditions forusingthistablespecifically includeastatement thatitshouldbeappliedonlytonormaloperating (pre-accident) conditions.

Application ofthistabletopost-accident conditions (i.e.,thesituation following someinitiating event)couldresultineithertoohighortoolowestimates ofHEPs,depending onthesituation involved.

Forexample,inSectionR."Operator FailstoRestoreControlAirThroughUseofthePlantAirCompressor DuringLossofOffsitePower($672),"errorQ6is"Operator failstonoticelackofcompressed airthroughsubsequent actions(i.e.,additional valveswillnotopen)."ThemeannominalHEPof1.6E-2isbasedonTable20-22,butthenisfurtherreducedbyamultiplier of0.5,togiveamodifiedQ6of8E-3.Asthisnumberisusedtomultiplyalltherelevantoperatoractions,itisaquitesizeableerrorrecoveryfactor.UnliketheZionHRA,theanalystsintheCookHRAdidattempttoassesstheeffectsofincorrect diagnosis ofvariousaccidentsequences.

InthethreecasesIreviewed, theinitialdiagnosis HEPwasselectedfromthenominaldiagnosis model(Table12-4or20-3)intheHRAHandbookandthenwasreducedbymakingnon-conservative modifications.

TheHEPwasfirstincreased byusingamultiplier of5forstress,butthenwasreducedbyamultiplier of0.1oreven0.01fortraining.

TheO.lmultiplier was A-3appliedto"RestoreControlAirinLOOP"andthe0.01multiplier wasappliedto"PBF-PrimaryFeedandBleed"andto"OA5-SteamGenerator Depressurization andCondensate Feed."Thus,thecombined"stress"and"training" adjustments resultedinmultiplying thediagnosis HEPfromTable20-3byeitherafactorof0.5or0.05.Thereareseveralproblemswiththeaboveapproach.

First,itisnotappropriate toadjustthenominaldiagnosis modelforstresseffects,astheHEPsinnominaldiagnosis modelalreadyincludesucheffects.Thesecondproblemisthatthefactorof0.01reduction isnotpartoftheHRAHandbook, and,inmyopinion,represents acompletely unjustified reduction.

Thereisprovision intheHRAHandbookfora0.1reduction factor.ThiscomesfromTable12-5"Guidelines forAdjusting NominalDiagnosis HEPsfromTable12-4."Table12-5providespossibleadjustments tothenominaldiagnosis modelasafunctionofpracticebyoperating crewsofthespecificeventbeinganalyzed.

Additional rulesforloweringthenominalHEPsfromTable20-3arepresented inTable8-1inNUREG/CR-4772(Ref.4).Thethirdproblemisthattheanalystshaveapparently completely misunderstood howTable20-3istobeapplied.Theestimated HEPsinthetablearenotforestimated timestoperformadiagnosis.

Instead,asexplained inChapter12oftheHRAHandbook, theHEPsaretimeavailable fordiagnosis notactualdiagnosis time.IfanalystsuseTable20-3,theywouldhavetodetermine thetotalamountofactiontimeplusanyoperatorrecoverytimeassessed, andsubtractthisfromthetimewindow(theTminmyterms).ThiswouldbemyTd,orthetimeavailable fordiagnosis.

ThenoneentersTable20-3withthistimetogetthediagnosis HEPforthatparticular available time.Oneofthebasicanalytical toolsofTHERPisthetaskanalysisinwhichthepotential forhumanerrorisidentified byidentifying thosePerformance ShapingFactors(PSFs)thatarenotfullycompatible withthecapabilities, limitations, andneedsoftaskperformers.

IntheCookHRA,itappearsthatHEPsareassignedtotaskswithoutfullconsideration ofthecontextsunderwhichthetasksmustbeperformed.

Thus,thesameHEPmaybeassignedtoataskthatisdoneunderdifferent circumstances.

IntheCooksystemeventtrees,thesametaskappearsindifferent locations andthesameconditional HEPisapparently assignedwithoutconsideration ofitscontext.TheapproachtakenintheHRAappearstobequitemechanistic andnon-situation specific.

OneofthemaintoolsoftheTHERPHRAmethod(whichtheanalystsstatetheyused)istheHRAeventtree.Thistreeisagraphicformoftaskanalysisthatenablesananalysttoidentifyinteractions betweentasksperformed byonepersonandinteractions betweendifferent persons.Inthiswayconditional probabilities ofsuccessandfailurecanbeassignedtothesuccessandfailurelimbsineachbranching inthetree.Thisgreatlysimplifies anHRAandgreatlyincreases thechangesofcorrectly considering dependence effects.Theuseof'equations withoutsuchtrees,ortheuseoffaulttreesinplaceofHRAeventtrees,greatlyincreases thechancesofoverlooking orincorrectly assessing dependence effects.

A-44.Theestimated valuesforadjustedHEPsareoftenverysmalland,inmyopinion,unjustifiably so.Forexample,theuseofamultiplier of0.1,orinsomecaseseven0.01,toapplytoHEPsbecauseof"goodtraining" isnotjustified inthedocumentation.

Thisisaprimeexampleofoptimisminassessment offinalHEPs,anditisoneoftheseveralmisusesoftheHRAHandbook.

IhavealwayssaidinmyHRAtrainingcoursesthatthetablesofestimated HEPsinthedatatablesintheHRAHandbookarenotsetinconcrete.

Asthedesignofman-machine interfaces improve(including improvements inwrittenprocedures),

theseHEPscouldbereduced.(Theconcluding chapterintheHRAHandbooksaysthesamething.)Butreductions intheseHEPsbyafactorof10(andcertainly byafactorof100)inmyopinionrepresent agrossmisapplication of,thedatatablesandshowahighdegreeofunsubstantiated optimism.

5.Onthepositiveside,theconsiderable useofsubject-'matter experts(e.g.,reactoroperators) intheHRAsprovidesagoodmeasureoffacevaliditytotheunderlying taskanalysis.

However,theanalystsapparently haveacceptedtheusualoptimismofhighlytrainedspecialists withoutquestion.

Ireferespecially tooptimistic statements abouttheirbeingnodebilitating stresseffectsoncopingwithhypothesized accidentsequences.

A-5II.DetailedCommentsontheD.C.CookHRACommentsaremadeonthevarioussectionsoftheD.C..Cook HRAdocumentation listedbelow.Someofthecommentsreflectthedifficulty Ihadintryingtodetermine whatwasdone.Provisions foreasytraceability ofwhattheanalystsdidandassumedwerelessthanadequate.

Becauseofmytimelimitations, Iwasunabletomakeasthoroughanevaluation oftheD.C.CookHRAasIdidfortheZionHRA.A.CommentsonSection3.3.3HumanFailureDatapp3-145to3-1591.AsintheZionHRA,theD.C.CookHRAestimates ofconditional HEPsstartoutwithmedianHEPsandEFsfromtheHRAHandbook(NUREG/CR-1278, whichisReference 22intheD.C.Cookdocument).

ThesemedianHEPsandEFsareconverted tonominalmeanHEPsandvariances foundinTable3.3-2(p3-148).Subsequently thesemeanHEPsaremodifiedbyotherfactorssuchasoperatortraining, existence ofprocedures, operatorstresslevel,etc.Theseotherfactorswereusedtocalculate acomposite PSFwhichwasusedasamultiplier onthenominalmeanHEPtocalculate "aplant-specific value."AlthoughtheHEPsmaybeplane-specific, manyofthemdonotappeartobesituation-specific.

NRCstafftransformed someofthesystemfaulttreesintosystemeventtrees,andnotedthateventhoughagiventaskappearedmorethanonceinatree,thesameHEPwouldbeassignedtoit.Thisisnotappropriate becauseforagiventask,thepreceding eventstoeachlocationofthetaskinthetreewerenotsame.Insomecases,thepreceding eventsmightincludeapresumedfailureofsomeequipment thatoperators dependon.Inothercases,thepreceding eventsmightincludeapreviousoperatorerrorinthesequenceofoperatoractions.Thisoversight canleadtooptimistic assessments ofHEPs.p3-145,3.3.3.1,lastsentence:

"TheHEPassociated withthegeneraloperatoractionwasquantified usingengineering calculations orfaulttreemodels(provides thesamecalculational resultsasTHERPtrees),asappropriate."

Theoretically, thisisatruestatement.

However,thisstatement overlooks oneofthefundamental advantages oftheTHERPHRAeventtreesoverfaulttrees.Useoffaulttreesrequiresanassumption ofindependence amongthebasiceventsleadingtoatopevent.IntheHRAeventtree,alltheprobabilities assignedtoeachbinarybranching inthetreeareconditional probabilities.

Thus,forproperapplication oftheTHERPmethodusingtheHRAeventtreesitisnotassumedthatsequential actionsdepictedinanHRAeventtreeareindependent.

Generally, somenon-zerolevelofdependence isassessed.

Anothermajordifference betweentheTHERPandfaulttreeapproachtoHRAisthattheformerisadeductive approachtoanalysiswhereasthelatteremploysadeductive approach.

TheHRAeventtreeusedinTHERPisagraphicrepresentation oftheresultofataskanalysis, whichisthebasictoolforacompleteHRA.Taskanalysisisaninductive methodusedtoidentifyhumanbehaviors andunderlying performance shapingfactors(PSFs).TaskanalysisanditsrelatedHRAeventtreesanalyzehuman A-6ac'tivities andrelatedsystemeventsandcontextsinaforwardmovingtimeframe.Botharealsohighlyflexibleintheirstructure andmoreamenabletoanalyzing thecomplexities involvedinhumanbehaviorinasystemcontext.Finally,anunpublished studybytheTechnical Institute ofFinland(VTT)identified errorsmadebyanalystsperforming anHRAwhenusingfaulttrees,HRAeventtrees,andcause-consequence diagrams.

ByfarthemosterrorsweremadewhenusingfaulttreesforHRAandthefewestwhenusingHRAeventtrees.Oneofthemostseriouserrorsinusingfaulttreesisthefailuretoadequately represent dependencies amongdifferent humantasks.Therefore, Imustconcludethat.thefailuretouseHRAeventtrees(oratleastsomeformofeventtree)fortheHRAdetailsislikelytohaveresultedintypeoferrorsdescribed above.p3-146,topofpage:"Guidelines usedincalculating PSFsareidentified inReference 22andinTable3.3-3."Thisstatement incorrectly impliesthatthevaluesforthePSFscomefromtheHRAHandbook; itsaysnothingabouttheuseofmodifiers notintheHRAHandbookwhichwereusedtomaterially reducetheHEPsusedintheHRA.Table3.3-3(p3-152)liststhe"Descriptive HRAScalingGuides."SeveralofthemarenottakenfromtheHRAHandbook, andIcannotdetermine theirbasisinthatdocument.

Following aresomedetailedcommentsonthattable:a.PSFof10forExtremely HighStress:Ifanydynamicaspectsordiagnosis ordecision-making isinvolved, Table20-16itemjj7listsabasicHEPof.25.Ifthetaskcanbeclassified asstep-by-step, themultiplier of5isusedperitemjj6inthetable.Depending ontheaccidentsequenceandthetaskinvolved, themultiplier of10couldbeeitherconservative ornon-conservative.

Butwhenananalystusesgenericfactorsratherthanasituation-specific

analysis, someinconsistency isunavoidable.

Theabbreviation w.r.t.isused.Ihavenoideawhatthatmeans.b.PSFof5foraStep-by-Step TaskUnderModerately HighStressfora"TypicalTransient":

Againthew.r.t.isstated,andavalueof2isrecommended.

Butapparently theanalystsusedthemoreconservative multiplier of5.Buteventhisfactorcouldbenon-conservative fordiagnostic activities laterinatransient ifthisdiagnosis musttakeplaceunderthedisruption ofmany,manyannunciators competing fortheoperator's attention.

c.PSFof0.1asa"Generalvalueforresponseforoperators whoarewell-trained intheappropriate procedures:

Asageneralacross-the-board rule,Ijudgethistobetoooptimistic.

IntheHRAHandbook, thereisallowance forusingthelowerboundofanominalHEPgivenunusualpositiveinfluences.

ThelowerboundformostofthetabledHEPsintheHRAHandbookisthemedianHEPdividedby3.Butinmyexperience, eventhisdownwardadjustment hasseldombeenused.

adA-7d.PSFofO.lforMedianTimeFrameforResponse:

Ihavefoundnodefinition ofwhatismeantbya"mediantimeframe."AndIthinkthatthistypeofgenericmultiplier withoutconsidering theunderlying PSFsforeachtaskinquestioncanleadtooptimism.

However,Idounderstand thedesireofanalyststodevelopgenericfactorstoreduceanalysistimeandeffort.e.PSFofO.lforAvailability ofMultipleSupportive Indicators:

"Multiple Supportive Indicators" donotnecessarily helpanoperator.

Icanenvisioncasesinwhichmultipleindicators mightwelldefineaheavytaskload(i.e.,moderately highstress)foranoperator.

SoratherthandividethenominalHEPby10,insuchcasesamultiplier ofsomewholenumber>1.0mightbeappropriate.

Also,thereisastrongtendencyonthepartofoperators tolookatonlyoneoftwoormoreindicators thatprovidebasically thesameinformation.

Thismaynotbeagoodpractice, butitistheall-too-commonmodeofoperation.

f.ThePSFsonpage3-153:IcanfindnobasisfortheseintheHRAHandbook.

Amultiplier of0.01ismostoptimistic.

5.p3-147,Section3.3.3.2,paragraph 2:Itisnotedthatforsomeoperatoractions,"nodetailedprocedures areavailable, andonlygeneraldirection isprovidedtotheoperator.

Insuchcases,thesuccessoftheoperatoractiondependslargelyonthetrainingandmemoryoftheoperator, andthecalculated HEPmaybeunreasonably high.Interviews withtheoperators wereconducted todetermine iftheoperators hadtheknowledge andtrainingtodealwiththesesituations."

Myconcernherewouldbethattheanalystscouldbemotivated togivetoomuchcreditfor"training andmemory."However,onp62ofAttachment toAEP:NRC1082F,discussion ofasensitivity analysisofsomerecoveryactionsdonewithoutawrittenprocedure indicates tomethatthecreditfor"training andmemory"isnotunreasonable.

AmedianHEPof.05(fromTable20-7item/j5)wasassessedforseveralsimple,routinerecoveryactionsnotcoveredindetailintherecoveryprocedures.

Theanalystsreportthattheresultant increaseintheprobability offailingtherecoveryactionofaboutafactorof5hadanegligible effectintheoverallsystemsanalysis.

6.p3-154,Table3.3-4Dependence LevelDefinitions:

Theequations forthefailureequations aretakenfromTable20-17intheHRAHandbook.

Theseequations represent theconditional probability ofhumanerrorgivenfailureontheprevioustask.Butthesuccessequations fromthistablehavebeenchangedtofailureequations, givensuccessontheprevioustask.Mathematically, thesechangesarecorrect,butIamuncertain astohowtheseequations areused.Typically, thesuccessequations inTable20-17wouldbeappliedtosuccessive tasksinaseriessystem,asdefinedintheHRAHandbook.

Andthefailureequations inTable20-17wouldbeappliedtosuccessive tasksinaparallelsystem.

A-87.p3-155,Table3.3-5SummaryofHumanErrorProbabilities:

SomeofthesemeanHEPsseemverysmalltome,e.g.,"manualvalverestoration aftertestandmaintenance" of2.1E-5,and"airormotor-operated valverestoration aftertestandmaintenance" of4.2E-7.Apparently theseHEPsandtheothersinthetableareappliedwithoutregardtosituation-specific PSFs.

A-9B.CommentsonSect~onI.TiminConsiderations inHumanReliabilit Analsis5unnumbered pages1.p1,paragraph on"SuccessTime":Basedoninterviews oftheanalystsbyNRCstaff,theinitiation ofanabnormaleventbysomecompelling signalsuchasanannunciator definesthestarttime.Thiscorresponds totheTOintheASEPHRAProcedure (Ref.4).Thetotaltimeavailable, Tminmyterminology, consistsofdiagnosis timeandpost-diagnosis actiontime.2.p1,paragraph on"Diagnosis Time":Itisstatedthattheuseofanydiagnosis HEPprovides"addedconservatism" becausesymptom-oriented EOPsareused.Itisfurthersaidthat"Thediagnosis errorratefoundintheHRAHandbook...isbasedonthediagnosis ofbasicinitiating eventswithlittleformalguidanceavailable

.totheoperators."

Apparently, theanalystsdonotunderstand thattheNominalDiagnosis Model(Table20-3)isbasedontimeavailable fordiagnosis, notactualdiagnosis time.IfanalystsuseTable20-3,theywouldhavetodetermine thetotalamountofactiontimeplusanyoperatorrecoverytimeassessed, andsubtractthisfromthetimewindow(theTminmyterms).ThiswouldbemyTd,orthetimeavailable fordiagnosis.

ThenoneentersTable20-3withthistimetogetthediagnosis HEPforthatparticular available time.It.istruethatforthe1983HRAHandbook, Table20-3wasdeveloped withevent-based EOPsinmind.ButTable12-5(seelastfootnoteinTable20-3)canbeusedtoassessalowerboundHEPasthenominalmedianHEPiftheproperconditions aremet.IntheASEPHRAProcedure (Ref.4),Table8-1item9.dpermitsthelowerboundtobeassessedasthediagnosis HEPifcertainconditions aremetinthedesignanduseofsymptom-oriented EOPs.Foravailable diagnosis timesupto30minutes,theuseofthelowerboundwouldbetantamount toassessing thenominaldiagnosis HEPdividedby10.Thisis,ofcourse,amultiplier ofO.l,asissometimeusedintheCookHRA.3.p1,paragraph on"ActionTime":Itisstatedthatthetimerequiredtoremotelyopenafewvalvescanbeignoredintheanalysis.

Therearetwoproblemswiththisstatement.

First,itmayrequireamaterialamountoftimeforanauxiliary operatortobetold(byphone)togotosomelocation, andthentraveltothatlocation.

InsomeHRAsIhaveperformed orreviewed, thistimecouldnotbeignored.Second,asnotedinmycommentsontheparagraph on"SuccessTime,"correctuseofthenominaldiagnosis model,Table20-3,fromtheHRAHandbookrequiresthatactiontimebeestimated.

4.p2:InotethattheHRAassumestheShiftTechnical Advisor(STA)willnotbeavailable until10minutesintoanabnormalevent.Thisappearstobeaconservative assumption.

C.CommentsonAendixtotheAttachment toAEP1NRC:1082F pp41-43;78-1011.pp41-43,R.OperatorFailstoRestoreControlAirThroughUseofthePlantAirCompressor DuringLossofOffsitePower:Itisdifficult toenvisionthehumanactionsintheequationfortheHEP.Itwouldbemucheasierforareviewer, especially onewithahumanfactorsbackground oroneusedtotheTHERPHRAmethod,toemployanHRAeventtree.Following isasketchofanapproximate HRAeventtreeinwhichonlythefailurelimbsarecompletely drawn.Onecannotethateachfailurepaththroughthisseriessystemendswithapotenterrorrecoveryfactor(RFHEP8E-3).TheQ6erroris:"Operator failstonoticelackofcompressed airthroughsubsequent mitigating actions(i.e.,additional valveswillnotopen)."Idon'thavesufficient information toevaluatethisRF,butitdoesresultinasubstantial reduction inFT,thetotalfailureprobability forthisevent.NotethatwereitnotforthisRFHEPbeingANDedwithQlinthefirstfailurepaththroughtheHRAeventtree,the.135HEPforQlfordiagnosis errorwouldresultinatotalfailureprobability, FTforthiseventbeingafactorof125higher.01.135FailtorespondtoANN05.00SFailtonoticelackoiconpzessed airthzu<<subsedzuent initiating actions.Fl.00108~Or,00065 Failtostart+plantaircospzessoz d+dtdttdttttiftdt<<ttcontzolaizilov/pzessuze

+Ol.0065Failtonanuallyloadair06.Ooscorpzessoz

/Xdt.dtit06.OOO65Failtoreopencontrolairisolation valves05~0019Fl<<61'~06.000~.F4-6 A-11c.Thereareseveralmisapplication oftheHRAHandbookinthisevent:Qlisamodification ofitemjj4inTable20-3(theNominalDiagnosis Model),butthenominalHEPismodifiedfortheeffectsofstress.Thistablealreadyincludestheeffectsofstress.Finally,Ifindthemultiplier ofO.lneedstohavebelievable justification.

(Seeespecially thediscussion initem9.dinTable8-1ofRef.4)2)Q2usesthecorrecttabledHEPfromtheHRAHandbook, butthe0.1multiplier for"trained" isquestionable.

Normally, ifoneallowsextracreditforacombination offamiliarity andhighskill,thelowerboundofthemedianHEPwouldbeused.Inthiscase,itwouldbe.001/3<<3E-4.Well,thatseemstoworkoutOK,assumingtheuseofthelowerboundcanbejustified.

ThemeanHEPwouldthenprobablybearoundthe6.4E-4assessed.

3)Q3soundslikeanECOM,butTable20-7isusedasthestartingpoint,andthistableisforEOMs.Icouldn'ttellifthisactionisdoneinthecontrolroomorisitdoneoutsidethecontrolroom.4)Q4seemsOKexceptforthe0.1multiplier fortraining.

5)Q5seemsOKexceptforthe0.1trainingmultiplier.

Butwhatdirectstheoperatortoperformthisstep26)Q6likealltheothertermsismultiplied by0.1for"trained."

Thisiscertainly aconvenient wayofreducingestimated HEPs,buttheuseofsucha"correction factor"acrosstheboardisquestionable.

AlsoIwonderhowmuchtimeisavailable forthisgenericRF.7)Nodependence isassessedamonganyoftheoperatoractions.ThismaybeOK;Idon'tknowenoughaboutthedetailstojudgewhetheritisOKornot.Buttypically ananalystwillassessindependence betweenEOMsforwrittenstepsinaprocedure.

Thereareexceptions, however.NRCstaffdrewasystemeventtreetobetterindicatethesequenceofoperatoractions.Thistreealsocallsintoquestiontheassessment ofa1.9E-3HEPforQ5regardless ofwherethetaskisperformed.

Thislimitation illustrates myearlierpointthatwhiletheHRAmaybeplant-specific, itcertainly isnotsituation-specific.

Italsoillustrates thepoint(asdoesmyHRAeventtree)thattheuseoffaulttreesfortheHRAoftenresultsinfailuretoconsidertheeffectsofdifferent PSFs,including dependence.

2.pp7888,2.3PBF-PrimaryBleedandFeed:a~ThisHRAincludesamultiplier of0.01foranoperatorfailingtoentertheappropriate EOP,andtheusual0.1fortraining.

Basically thesamemisapplication oftheHRAHandbookdescribed initem1abovewerealsofoundintheHRAforthisevent.NRCstaffdrewasystemeventtreewhichillustrated theassessment ofoneHEPtothesametaskregardless ofwherethattaskoccurredintheaccidentsequence.

A-12pp89-101,2.4OA5-SteamDepressurization andCondensate Feed:ThisHRAisalmostidentical tothePrimaryBleedandFeedHRAabove,including misapplication oftheHRAHandbook.

Itcanbenotedthat"FollowsProcedures" occursseveraltimesinthefaulttree.Thereisnoexplanation ofwhatismeantbythesewords.Wastheerrorthefailuretouseavailable writtenprocedures or,morelikely,thefailuretodowhatwasinthewrittenprocedure.

Inanyevent,whydoesthisappearseveraltimesintheerrorsequences?

Itmaybethatthismeansverification bytheSTA,butthiswouldimplythattheSTAverifiesdetailedoperatoractions,afunctionnotappropriate totheSTA.

References 1.Swain,A.D.andH.E.Guttmann, Handbookof'umanReliability AnalysisVithEmphasisonNuclearPowerPlantApplications, NUREG/CR-1278, U.S.NuclearRegulatory Commission, Washington DC,August1983,128pp.2.Bell,B.J.andA.D.Swain,AProcedure forConducting aHumanRe'liability AnalysisforNuclearPowerPlants,NUREG/CR-2254, U.S.NuclearRegulatory Commission, Washington DC,May1983,700pp.3.Swain,A.D.,Evaluation ofZionIPE/HRAMaterials, U.S.NuclearRegulatory Commission, Washington DC,June$994,33pp.4.Swain,A.D.,AccidentSequenceEvaluation ProgramHumanReliability AnalysisProcedure, NUREG/CR-4772, U.S.NuclearRegulatory Commission, Washington DC,February1987,152pp.