ML17332A712

Evaluation of Cook Ipe/Hra Matls.

ML17332A712
Person / Time
Site:	Cook
Issue date:	12/05/1994
From:	Swain A External (Affiliation Not Assigned)
To:
Shared Package
ML17332A711	List: ML17332A712
References
NUDOCS 9504050248
Download: ML17332A712 (20)

Text

EVALUATION OF COOK IPE/HRA MATERIALS Letter Report to U.S. Nuclear Regulatory Commission Office of Nuclear Regulatory Research By Alan D. Swain, Ph.D.

712 Sundown Place SE 4555 S. Mission Rd. j/967 Albuquerque, NM 87108 Tucson, AZ 85746 (505) 265-0098 (602) 294-1769 Submitted: May 10, 1994 Finalized: December 5, 1994 File: ga/NRCQCookHRA.943

'P504050248 941205 PDR ADOCK 05000315 PORE Attachment

N J

EVALUATION OF COOK IPE/HRA MATERIALS

1.0 INTRODUCTION

NRC staff asked me to review the Human Reliability Analysis (HRA) portions of the Cook Nuclear Power Plant (NPP) Individual Plant Examination (IPE). I reviewed the documentation listed in Appendix A, visited NRC staff twice, and held numerous telephone sessions with them.

Specifically, as the primary developer of the Technique for Human Error Rate Prediction (THERP), I was asked to determine whether Cook HRA was based on appropriate use of THERP as described in the HRA Handbook by Swain and Guttmann (NUREG/CR-1278) (Ref. 1 - references are at end of this document). The Cook HRA documentation states that the HRA is based on THERP and the human performance models and data tables in the HRA Handbook, and also on NUREG/CR-2254 (Ref. 2).

I was also asked to judge whether the HRA is credible (believable), and what problems were there in the HRA.

if not, The first 4 pages of this document summarize my findings and conclusions.

Appendix presents my detailed review of the Cook HRA. These A first 4 pages and the Summary in Appendix A are very similar to my previously completed evaluation of the Zion NPP HRA (Ref. 3) because the HRA method used and the problems found are similar.

2.0 APPROACH USED IN THE HRA This section paraphrases some of the main points in the approach used in the Cook HRA, according to the IPE documentation provided me by NRC staff.

The Cook HRA is based on what the analysts called a step-by-step task analysis of operator actions to be included in system event trees. For each operator action, the analysts identified Action Identifier, Description of Action, Time Window Available for Action, Applicable Procedures, and Indication of Whether the Action was Simulated in Training. Generic median human error probabilities (HEPs) for each action were obtained from various tables in the HRA Handbook and converted to generic mean HEPs by using the handbook's assumption of a log-normal distribution and estimated 5X and 95X error factors. Assessment of HEPs was done either by fault tree analysts or by HRA analysts. What were defined as plant-specific performance shaping factors (PSFs) were derived and assigned scaling values which served as multipli.ers of the generic HEPs. The multipliers for stress effects were 1.0 to 10.0, and the multipliers for other PSFs (e.g.,

availability of multiple supportive indicators, selection of wrong controls, quality of training, memorized procedure) generally were assigned multipliers of less than 1.0, i.e , .1 or .01. When estimates of dependence effects were required, the positive dependence model from the HRA Handbook was used.

The analysts employed a fault-tree approach rather than the HRA event tree approach described in the HRA Handbook. The HEPs used in the system event trees were generally calculated using individual fault trees. As part of the quantification process, a plant visit was made to obtain operator review and assistance about the modeled human actions.

3,0 MAJOR PROBLEMS 3.1 Assum tions Following are some of the more important assumptions stated in the Cook HRA documentation that are related to two major problems. The first problem is what I consider to be optimistic assessments of the contribution of human errors to system-critical events. The second problem is the across-the-board assessment of a relatively small set of HEPs without adequate consideration of the specific situations and the underlying specific PSFs for those situations.

A major assumption is that the Cook NPP operating crews are so well trained in the use of the symptom-oriented EOPs that a multiplier of 0.1 was applied to nearly all of the generic HEPs. If the analysts judged that the procedure to be used was memorized, this multiplier of 0.1 was increased to 0.01. Thus, the related HEP would be divided by 100. I find this optimism incredible. Moreover, the use of these kinds of generic error reduction factors is apparently not supported by a specific task analysis for the tasks involved. At least, no such task analysis was included in the HRA materials I was given to review.'onsiderable credit for recovering from operator errors is given to what is called "checking that involves active participation (STA monitors plant parameters)." The use the HRA Handbook's Table 20-22 "Estimated probabilities that a checker will fail to detect errors made by others" is inappropriate to apply to checking during an accident sequence. The conditions for using this table specifically include a statement that it should be applied only to normal operating (pre-accident) conditions. Application of this table to post-accident conditions (i.e., the situation following some initiating event) could result in either too high or too low estimates of HEPs, depending on the situation involved. Also, the mean nominal HEP of 1.6E-2 that is based on Table 20-22 is further reduced by a multiplier of 0.5, to give a modified of 8E-3, As this number is used to multiply all the relevant operator actions, it is a quite sizeable error recovery factor.

Unlike the Zion HRA (Ref. 3), it was assumed that diagnosis errors were important in the operators'esponses to accident sequences. However, the generic HEPs from Table 20-3 in the HRA Handbook were modified upwards by a factor of 5 for moderately high stress and then reduced by a factor of O.l for training. Table 20-3 already includes the effects of stress. Although through the appropriate use of Table 12-5 in the HRA Handbook, it is possible to reduce diagnosis HEPs by a factor of 10, the conditions for such reduction as stated in this table are stated, and further amplified and clarified in NUREG/CR-4772 (Ref. 4), which is a more recent application of THERP. In my opinion, the Cook HRA documentation does not provide an adequate rationale for their use of a reduction factor of 10.

3.2 Modelin of Human Behavior For a full-scale HRA, which the Cook HRA purports to be, it is necessary to model the human behavior correctly for each task or set of tasks that could have a material negative impact on the system failure criteria of interest. In the THERP approach to HRA, this always means a thorough task analysis for each relevant task in its context, as is described in Chapter 4 of the HRA Handbook. All of the PSFs that are likely to have a material effect on a task should be included. The task analysis basically states what leads to what, and moves forward with time.

It is an inductive method, as contrasted with the fault-tree methodology which is a deductive method working backwards in a sequence of events.

While it is theoretically possible to perform an HRA using the inductive fault-tree methodology, those of us with considerable experience in the HRA field have elected to use the deductive event-tree approach. Event trees, especially the HRA event trees used in the THERP HRA method since 1961, make it easier to graphically represent the necessary underlying task analysis, and to represent all human errors and human successes as conditional probabilities in which dependence effects are represented directly in the trees. The influence of important plant-specific PSFs is included in each branching in the event tree, and is based on the particular context in which the potential error could occur.

In short, the assessment of conditional human error and success probabilities for each relevant task (or step in a task) is based on a full consideration of the context under which task must be performed.

This approach can be contrasted with that used in the Cook HRA in which a relatively small set of generic HEPs and generic PSF modifying factors were used despite apparent differences in context. I found no evidence of an underlying task analysis of the type that I would find acceptable for a full-scale HRA. I would characterize the Cook HRA as a computerized, mechanistic approach that does not have sufficient flexibility to fully incorporate the effects of plant-specific PSFs.

3.3 A lication of THERP There were obvious misapplication of THERP. These include use in their post-accident HRA of tables from the HRA Handbook intended only for HRA of pre-accident tasks, use of generic multipliers of 0.1 or even 0.01 as adjustments to HEPs taken from the HRA Handbook, and adjusting the handbook's nominal diagnosis model for stress effects when that model already includes the effects of stress.

The most serious misapplication of THERP was the apparent assumption that human behavior can be partitioned into small units of actions, the HEPs of which can be incorporated into fault trees as independent basic events.

In general, it appears to me that the several misapplication of the full-scale approach to HRA described in the HRA Handbook indicate that the analysts did not have much understanding of the THERP method or the human performance models in the HRA Handbook.

3.4 Traceabilit One of the most frequent criticisms of recent HRAs is that what the analysts did is not traceable. I have spent a good deal of time trying to figure out exactly how the Cook analysts arrived at the estimated HEPs reported in the documentation. In several cases, I just had to give up. Even though the analysts used fault-tree technology in the HRA, no fault trees for specific analyses were displayed. One has to look into the equations and induce the fault tree involved or construct ones own event tree. HRAs should be written so that independent reviewers can readily and accurately evaluate what was done.

4.0 CONCLUSION

S My review of major portions of the documentation of the human reliability analysis (ERA) in the Cook Individual Plant Examination (IPE) revealed several major problems. Although the analysts state that the HRA was based on the HRA Handbook (NJREG/CR-1278) and its companion document, NUREG/CR-2254, many misapplication of data and models from this handbook occurred. There were many non-conservatisms in the HRAs, especially in the unusually large amount of credit assessed fer training and experience and for the employment of symptom-oriented emergency operating procedures. There was failure to fully consider the effects of within-person dependence in performing tasks. Many of the estimated human error probabilities (HEPs) were not based on situation-specific performance shaping factors and context of the tasks, especially tasks that are performed more than ence in the same accident sequence. This problem is exacerbated by, the use of global or generic estimates of HEPs, which resulted in a rather mechanistic approach to HRA. Finally, traceability of what was done in the HRA was made more difficult because the necessary information was lacking, not clearly written, or mostly displayed in equation form only.

My overall conclusion is that many of the estimated HEPs are not credible. I have no confidence that an overall systems analysis incorporating the results of the current Cock HRA would properly assess the influence of potential human errors.

Appendix A DETAILED EVALUATION OF COOK IPE/HRA MATERIALS (Draft 1)

This appendix provides more detailed comments related to my review of the application of THERP (Technique for Human Error rate Prediction - see Refs 1, 2, and 4) in the Human Reliability Analysis (HRA) portions of Individual Plant Examinations (IPEs) on the Cook nuclear power plant (NPP). This review is restricted to the Cook HRA and related material made available to me for review.

Table of Contents Acronyms & Abbreviations I. Summary II. Detailed Comments A. Comments on Section 3.3.3 Human Failure Data S. Comments on Section I. Timing Considerations in Human Reliability 9 Analysis C. Comments on Appendix to the Attachment to AEP:NRC:1082F 10 References 13 Acron s 6 Abbreviations ANN Annunciator ASEP Accident Sequence Evaluation Program EF Error Factor EOP Emergency Operating Procedure FT Fault Tree HEP Human Error Probability HRA Human Reliability Analysis IPE Individual Plant Examination NPP Nuclear Power Plant NRC Nuclear Regulatory Commission OA Operator Action PRA Probabilistic Risk Assessment PSF Performance Shaping Factor RF [Error] Recovery Factor RO Reactor Operator STA Shift Technical Advisor THERP Technique for Human Error Rate Prediction Technical Institute of Finland

~ ~

A-2

~Z ~ S Qllllll8 I

1. One of the NRC questions to me was whether some uses of THERP in the Cook HRA were not in agreement with the procedure described in the HRA Handbook (NUREG/CR-1278, Swain and Guttmann, 1983 - Ref. 1). The NRC staff identified what they judged to be "atypical applications" of THERP, and further stated that "The staff ... can not conclude whether the potential for 'erroneous results'omes from the particular way THERP was applied by an individual consultant ... or by'he lack of understanding of THERP by some licensees."

After reading the material on the Cook HRA that NRC provided me, I conclude that there were indeed several inappropriate applications of the THERP/Handbook approach to HRA. It appears to me that the several misapplication indicate a lack of understanding by the analysts of THERP and the human performance models in the HRA Handbook. The following three examples and others noted later illustrate this point:

The nominal diagnosis model (Table 20-3 of the HRA Handbook) was incorrectly used. The analysts did not understand that the table is based on time available for diagnosis, which is obtained by subtracting post-diagnosis action time from total time available for the human diagnosis and actions combined.

The special rules used to adjust the joint HEPs for diagnosis of events found in the nominal diagnosis model (Table 20-3 of the HRA Handbook) are not taken from the HRA Handbook, and, as noted in item 2 below, application of these rules result in substantial and, .in my opinion, optimistic downward adjustments of diagnosis HEPs.

HRA Handbook's Table 20-22 "Estimated probabilities that a checker will fail to detect errors made by others" is inappropriate to apply to checking during an accident sequence. The conditions for using this table specifically include a statement that it should be applied only to normal operating (pre-accident) conditions.

Application of this table to post-accident conditions (i.e., the situation following some initiating event) could result in either too high or too low estimates of HEPs, depending on the situation involved. For example, in Section R. "Operator Fails to Restore Control Air Through Use of the Plant Air Compressor During Loss of Offsite Power ($/672)," error Q6 is "Operator fails to notice lack of compressed air through subsequent actions (i.e., additional valves will not open)." The mean nominal HEP of 1.6E-2 is based on Table 20-22, but then is further reduced by a multiplier of 0.5, to give a modified Q6 of 8E-3. As this number is used to multiply all the relevant operator actions, it is a quite sizeable error recovery factor.

2. Unlike the Zion HRA, the analysts in the Cook HRA did attempt to assess the effects of incorrect diagnosis of various accident sequences. In the three cases I reviewed, the initial diagnosis HEP was selected from the nominal diagnosis model (Table 12-4 or 20-3) in the HRA Handbook and then was reduced by making non-conservative modifications. The HEP was first increased by using a multiplier of 5 for stress, but then was reduced by a multiplier of 0.1 or even 0.01 for training. The O.l multiplier was

~

~

A-3 applied to "Restore Control Air in LOOP" and the 0.01 multiplier was applied to "PBF - Primary Feed and Bleed" and to "OA5 - Steam Generator Depressurization and Condensate Feed." Thus, the combined "stress" and "training" adjustments resulted in multiplying the diagnosis HEP from Table 20-3 by either a factor of 0.5 or 0.05.

There are several problems with the above approach. First, it is not appropriate to adjust the nominal diagnosis model for stress effects, as the HEPs in nominal diagnosis model already include such effects.

The second problem is that the factor of 0.01 reduction is not part of the HRA Handbook, and, in my opinion, represents a completely unjustified reduction. There is provision in the HRA Handbook for a 0.1 reduction factor. This comes from Table 12-5 "Guidelines for Adjusting Nominal Diagnosis HEPs from Table 12-4." Table 12-5 provides possible adjustments to the nominal diagnosis model as a function of practice by operating crews of the specific event being analyzed. Additional rules for lowering the nominal HEPs from Table 20-3 are presented in Table 8-1 in NUREG/CR-4772 (Ref. 4).

The third problem is that the analysts have apparently completely misunderstood how Table 20-3 is to be applied. The estimated HEPs in the table are not for estimated times to perform a diagnosis. Instead, as explained in Chapter 12 of the HRA Handbook, the HEPs are time available for diagnosis not actual diagnosis time If analysts use Table 20-3, they would have to determine the total amount of action time plus any operator recovery time assessed, and subtract this from the time window (the Tm in my terms). This would be my Td, or the time available for diagnosis. Then one enters Table 20-3 with this time to get the diagnosis HEP for that particular available time.

One of the basic analytical tools of THERP is the task analysis in which the potential for human error is identified by identifying those Performance Shaping Factors (PSFs) that are not fully compatible with the capabilities, limitations, and needs of task performers. In the Cook HRA, it appears that HEPs are assigned to tasks without full consideration of the contexts under which the tasks must be performed. Thus, the same HEP may be assigned to a task that is done under different circumstances. In the Cook system event trees, the same task appears in different locations and the same conditional HEP is apparently assigned without consideration of its context.

The approach taken in the HRA appears to be quite mechanistic and non-situation specific. One of the main tools of the THERP HRA method (which the analysts state they used) is the HRA event tree. This tree is a graphic form of task analysis that enables an analyst to identify interactions between tasks performed by one person and interactions between different persons. In this way conditional probabilities of success and failure can be assigned to the success and failure limbs -in each branching in the tree. This greatly simplifies an HRA and greatly increases the changes of correctly considering dependence effects. The use of equations without such trees, or the use of fault trees in place of HRA event trees, greatly increases the chances of overlooking or incorrectly assessing dependence effects.

A-4

4. The estimated values for adjusted HEPs are often very small and, in my opinion, unjustifiably so. For example, the use of a multiplier of O.l, or in some cases even 0.01, to apply to HEPs because of "good training" is not justified in the documentation. This is a prime example of optimism in assessment of final HEPs, and it is one of the several misuses of the HRA Handbook. I have always said in my HRA training courses that the tables of estimated HEPs in the data tables in the HRA Handbook are not set in concrete. As the design of man-machine interfaces improve (including improvements in written procedures), these HEPs could be reduced. (The concluding chapter in the HRA Handbook says the same thing.) But reductions in these HEPs by a factor of 10 (and certainly by a factor of 100) in my opinion represent a gross misapplication of the data tables and show a high degree of unsubstantiated optimism.

5. On the positive side, the considerable use of subject-matter experts (cog., reactor operators) in the HRAs provides a good measure of face validity to the underlying task analysis. However, the analysts apparently have accepted the usual optimism of highly trained specialists without question. I refer especially to optimistic statements about their being no debilitating stress effects on coping with hypothesized accident sequences.

A-5 II. Detailed Comments on the D.C. Cook HRA Comments are made on the various sections of the D.C. Cook HRA documentation listed below. Some of the comments reflect the difficulty I had in trying to determine what was done. Provisions for easy traceability of what the analysts did and assumed were less than adequate.

Because of my time limitations, I was unable to make as thorough an evaluation of the D.C. Cook HRA as I did for the Zion HRA.

A. Comments on Section 3.3.3 Human Failure Data pp 3-145 to 3-159 As in the Zion HRA, the D.C. Cook HRA estimates of conditional HEPs start out with median HEPs and EFs from the HRA Handbook (NUREG/CR-1278, which is Reference 22 in the D.C. Cook document). These median HEPs and EFs are converted to nominal mean HEPs and variances found in Table 3.3-2 (p 3-148). Subsequently these mean HEPs are modified by other factors such as operator training, existence of procedures, operator stress level, etc.

These other factors were used to calculate a composite PSF which was used as a multiplier on the nominal mean HEP to calculate "a plant-specific value."

2. Although the HEPs may be plant-specific, many of them do not appear to be situation-specific. NRC staff transformed some of the system fault trees into system event trees, and noted that even though a given task appeared more than once in a tree, the same HEP would be assigned to it. This is not appropriate because for a given task, the preceding events to each location of the task in the tree were not same. In some cases, the preceding events might include a presumed failure of some equipment that operators depend on. In other cases, the preceding events might include a previous operator error in the sequence of operator actions. This oversight can lead to optimistic assessments of HEPs.

3. p 3-145, 3.3.3.1, last sentence: "The HEP associated with the general operator action was quantified using engineering calculations or fault tree models (provides the same calculational results as THERP trees), as appropriate." Theoretically, this is a true statement. However, this statement overlooks one of the fundamental advantages of the THERP HRA event trees over fault trees. Use of fault trees requires an assumption of independence among the basic events leading to a top event. In the HRA event tree, all the probabilities assigned to each binary branching in the tree are conditional probabilities. Thus, for proper application of the THERP method using the HRA event trees it is not assumed that sequential actions depicted in an HRA event tree are independent. Generally, some non-zero level of dependence is assessed.

Another major difference between the THERP and fault tree approach to HRA is that the former is a deductive approach to analysis whereas the latter employs a deductive approach. The HRA event tree used in THERP is a graphic representation of the result of a task analysis, which is the basic tool for a complete HRA. Task analysis is an inductive method used to identify human behaviors and underlying performance shaping factors (PSFs). Task analysis and its related HRA event trees analyze human

A-6 activities and related system events and contexts in a forward moving time frame. Both are also highly flexible in their structure and more amenable to analyzing the complexities involved in human behavior in a system context.

Finally, an unpublished study by the Technical Institute of Finland (VTT) identified errors made by analysts performing an HRA when using fault trees, HRA event trees, and cause-consequence diagrams. By far the most errors were made when using fault trees for HRA and the fewest when using HRA event trees. One of the most serious errors in using fault trees is the failure to adequately represent dependencies among different human tasks.

Therefore, I must conclude that the failure to use HRA event trees (or at least some form of event tree) for the HRA details is likely to have resulted in type of errors described above.

p 3-146, top of page: "Guidelines used in calculating PSFs are identified in Reference 22 and in Table 3.3-3." This statement incorrectly implies that the values for the PSFs come from the HRA Handbook; it says nothing about the use of modifiers not in the HRA Handbook which were used to materially reduce the HEPs used in the HRA. Table 3.3-3 (p 3-152) lists the "Descriptive HRA Scaling Guides." Several of them are not taken from the HRA Handbook, and I cannot determine their basis in that document.

Following are some detailed comments on that table:

a. PSF of 10 for Extremely High Stress:

If any dynamic aspects or diagnosis or decision-making is involved, Table 20-16 item f/7 lists a basic HEP of .25. If the task can be classified as step-by-step, the multiplier of 5 is used per item $/6 in the table. Depending on the accident sequence and the task involved, the multiplier of 10 could be either conservative or non-conservative. But when an analyst uses generic factors rather than a situation-specific analysis, some inconsistency is unavoidable.

The abbreviation w.r.t. is used. I have no idea what that means.

b. PSF of 5 for a Step-by-Step Task Under Moderately High Stress for a "Typical Transient":

Again the w.r.t. is stated, and a value of 2 is recommended. But apparently the analysts used the more conservative multiplier of 5.

But even this factor could be non-conservative for diagnostic activities later in a transient if this diagnosis must take place under the disruption of many, many annunciators competing for the operator's attention.

c. PSF of 0.1 as a "General value for response for operators who are well-trained in the appropriate procedures:

As a general across-the-board rule, I judge this to be too optimistic. In the HRA Handbook, there is allowance for using the lower bound of a nominal HEP given unusual positive influences. The lower bound for most of the tabled HEPs in the HRA Handbook is the median HEP divided by 3. But in my experience, even this downward adjustment has seldom been used.

A-7

d. PSF of O.l for Median Time Frame for Response:

I have found no definition of what is meant by a "median time frame." And I think that this type of generic multiplier without considering the underlying PSFs for each task in question can lead to optimism. However, I do understand the desire of analysts to develop generic factors to reduce analysis time and effort.

e. PSF of O.l for Availability of Multiple Supportive Indicators:

"Multiple Supportive Indicators" do not necessarily help an operator. I can envision cases in which multiple indicators might well define a heavy task load (i.e., moderately high stress) for an operator. So rather than divide the nominal HEP by 10, in such cases a multiplier of some whole number >1.0 might be appropriate. Also, there is a strong tendency on the part of operators to look at only one of two or more indicators that provide basically the same information. This may not be a good practice, but it is the all-too-common mode of operation.

f. The PSFs on page 3-153:

I can find no basis for these in the HRA Handbook. A multiplier of 0.01 is most optimistic.

p 3-147, Section 3.3.3.2, paragraph 2: It is noted that for some operator actions, "no detailed procedures are available, and only general direction is provided to the operator. In such cases, the success of the operator action depends largely on the training and memory of the operator, and the calculated HEP may be unreasonably high. Interviews with the operators were conducted to determine if the operators had the knowledge and training to deal with these situations."

My concern here would be that the analysts could be motivated to give too much credit for "training and memory." However, on p 62 of Attachment to AEP:HRC 1082F, discussion of a sensitivity analysis of some recovery actions done without a written procedure indicates to me that the credit for "training and memory" is not unreasonable. A median HEP of .05 (from Table 20-7 item g5) was assessed for several simple, routine recovery actions not covered in detail in the recovery procedures. The analysts report that the resultant increase in the probability of failing the recovery action of about a factor of 5 had a negligible effect in the overall systems analysis.

6. p 3-154, Table 3.3-4 Dependence Level Definitions:

The equations for the failure equations are taken from Table 20-17 in the HRA Handbook. These equations represent the conditional probability of human error given failure on the previous task. But the success equations from this table have been changed to failure equations, given success on the previous task. Mathematically, these changes are correct, but I am uncertain as to how these equations are used. Typically, the success equations in Table 20-17 would be applied to successive tasks in a series system, as defined in the HRA Handbook. And the failure equations in Table 20-17 would be applied to successive tasks in a parallel system.

A-8

7. p 3-155, Table 3.3-5 Summary of Human Error Probabilities:

Some of these mean HEPs seem very small to me, e.g., "manual valve restoration after test and maintenance" of 2.1E-5, and "air or motor-operated valve restoration after test and maintenance" of 4.2E-7. Apparently these HEPs and the others in the table are applied without regard to situation-specific PSFs.

A-9 B. Comments on Section I. Timin Considerations in Human Reliabilit Anal sis 5 unnumbered pages

1. p 1, paragraph on "Success Time": Based on interviews of the analysts by NRC staff, the initiation of an abnormal event by some compelling signal such as an annunciator defines the start time. This corresponds to the TO in the ASEP HRA Procedure (Ref. 4). The total time available, Tm in my terminology, consists of diagnosis time and post-diagnosis action time.

2. p 1, paragraph on "Diagnosis Time": It is stated that the use of any diagnosis HEP provides "added conservatism" because symptom-oriented EOPs are used. It is further said that "The diagnosis error rate found in the HRA Handbook ... is based on the diagnosis of basic initiating events with little formal guidance available to the operators." Apparently, the analysts do not understand that the Nominal Diagnosis Model (Table 20-3) is based on time available for diagnosis, not actual diagnosis time.

analysts use Table 20-3, they would have to determine the total amount of If action time plus any operator recovery time assessed, and subtract this from the time window (the Tm in my terms). This would be my Td, or the time available for diagnosis. Then one enters Table 20-3 with this time to get the diagnosis HEP for that particular available time.

It is true that for the 1983 HRA Handbook, Table 20-3 was developed with event-based EOPs in mind. But Table 12-5 (see last footnote in Table 20-3) can be used to assess a lower bound HEP as the nominal median HEP if proper conditions are met. In the ASEP HRA Procedure (Ref. 4), Table 8-1 the item 9.d permits the lower bound to be assessed as the diagnosis HEP certain conditions are met in the design and use of symptom-oriented EOPs.

if For available diagnosis times up to 30 minutes, the use of the lower bound would be tantamount to assessing the nominal diagnosis HEP divided by 10.

This is, of course, a multiplier of 0.1, as is sometime used in the Cook HRA.

p 1, paragraph on "Action Time": It is stated that the time required to remotely open a few valves can be ignored in the analysis. There are two problems with this statement. First, it may require a material amount of time for an auxiliary operator to be told (by phone) to go to some location, and then travel to that location.,In some HRAs I have performed or reviewed, this time could not be ignored. Second, as noted in my comments on the paragraph on "Success Time," correct use of the nominal diagnosis model, Table 20-3, from the HRA Handbook requires that action time be estimated.

4. p 2: I note that the HRA assumes the Shift Technical Advisor (STA) will not be available until 10 minutes into an abnormal event. This appears to be a conservative assumption.

C. Comments on A endix to the Attachment to AEP:NRC:1082F pp 41 - 43; 78 - 101

1. pp 41-43, R. Operator Fails to Restore Control Air Through Use of the Plant Air Compressor During Loss of Offsite Power:

It is difficult to envision the human actions in the equation for the HEP. It would be much easier for a reviewer, especially one with a human factors background or one used to the THERP HRA method, to employ an HRA event tree.

b. Following is a sketch of an approximate HRA event tree in which only the failure limbs are completely drawn. One can note that each failure path through this series system ends with a potent error recovery factor (RF HEP - 8E-3). The Q6 error is: "Operator fails to notice lack of compressed air through subsequent mitigating actions (i.e., additional valves will not open)." I don't have sufficient information to evaluate this RF, but it does result in a substantial reduction in FT, the total failure probability for this event. Note that were it not for this RF HEP being ANDed with Ql in the first failure path through the HRA event tree, the .135 HEP for Ql for diagnosis error would result in a total failure probability, FT for this event being a factor of 125 higher.

{}3 .135 Fall to respond to }d06

// ~ 05 .OOS Fall to notice lack of coepzessed afz thzu

'sunsequent fnftfatfng actions.

F1 ~ 00108

~

02 ~ 00065 fall to scazt plant afr eoepressoz

/ ~+

Q5 .0019 Fall to verify adequate f

control afz flov/pressure

, 03 .0065 Fail to nanuaffy load afz 0 .oos eocpressor F2 ~ 6

~ 05 .0019

.00065 fall to Q6 zeopen contxol afz fsofatfon valves

/

F3 6

.>05 .OO19 06 .006

/

F6 ~ 6

c. There are several misapplication of the HRA Handbook in this event:

1) Ql is a modification of item g4 in Table 20-3 (the Nominal Diagnosis Model), but the nominal HEP is modified for the effects of stress.

This table already includes the effects of stress. Finally, I find the multiplier of 0.1 needs to have believable justification. (See especially the discussion in item 9.d in Table 8-1 of Ref. 4)

2) Q2 uses the correct tabled HEP from the HRA Handbook, but the O.l multiplier for "trained" is questionable. Normally, if one allows extra credit for a combination of familiarity and high skill, the lower bound of the median HEP would be used. In this case, it would be .001/3 3E-4. Well, that seems to work out OK, assuming the use of the lower bound can be justified. The mean HEP would then probably be around the 6.4E-4 assessed.

3) Q3 sounds like an ECOM, but Table 20-7 is used as the starting point, and this table is for EOMs. I couldn't done in the control room or is it tell if this action is done outside the control room.

4) Q4 seems OK except for the 0.1 multiplier for training.

5) Q5 seems OK except for the O.l training multiplier. But what directs the operator to perform this steps

6) Q6 like all the other terms is multiplied by 0.1 for "trained." This is certainly a convenient way of reducing estimated HEPs, but the use of such a "correction factor" across the board is questionable.

Also I ~onder how much time is available for this generic RF.

7) No dependence is assessed among any of the operator actions. This may be OK; I don't know enough about the details to judge whether is OK or not. But typically an analyst will assess independence it between EOMs for written steps in a procedure. There are exceptions, however.

d. NRC staff drew a system event tree to better indicate the sequence of operator actions. This tree also calls into question the assessment of a 1.9E-3 HEP for Q5 regardless of where the task is performed. This limitation illustrates my earlier point that while the HRA may be plant-specific, it certainly is not situation-specific. It also illustrates the point (as does my HRA event tree) that the use of fault trees for the HRA often results in failure to consider the effects of different PSFs, including dependence.

2. pp 78 88, 2.3 PBF - Primary Bleed and Feed:

This. HRA includes a multiplier of 0.01 for an operator failing to enter the appropriate EOP, and the usual 0.1 for training.

b.. Basically the same misapplication of the HRA Handbook described in item 1 above were also found in the HRA for this event.

c~ NRC staff drew a system event tree which illustrated the assessment

, ~

of one HEP to the same task regardless of where that task occurred in the accident sequence.

3. pp 89 - 101, 2.4 OA5 - Steam Depressurization and Condensate Feed:

This HRA is almost identical to the Primary Bleed and Feed HRA above, includi'ng misapplication of the HRA Handbook.

It can be noted that "Follows Procedures" occurs several times in the fault tree. There is no explanation of what is meant by these words. Was the error the failure to use available written procedures or, more likely, the failure to do what was in the written procedure. In any event, why does this appear several times in the error sequences't may be that this means verification by the STA, but this would imply that the STA verifies detailed operator actions, a function not appropriate to the STA.

References

1. Swain, A. D. and H. E. Guttmann, Handbook of Human Reliability Analysis Vith Emphasis on Nuclear Power Plant Applications, NUREG/CR-1278, U.S.

Nuclear Regulatory Commission, Washington DC, August 1983, 128 pp.

2. Bell, B. J. and A. D. Swain, A Procedure for Conducting a Human Reliability Analysis for Nuclear Power Plants, NUREG/CR-2254, U.S. Nuclear Regulatory Commission, Washington DC, May 1983, 700 pp.

3. Swain, A. D., Evaluation of Zion IPE/HRA Materials, Unpublished Report, U.S. Nuclear Regulatory Commission, Washington DC, May 1954, 33 pp.

4. Swain, A. D., Accident Sequence Evaluation Program Human Reliability Analysis Procedure, NUREG/CR-4772, U.S. Nuclear Regulatory Commission, Washington DC, February 1987, 152 pp.

4

/

l~ 1l