ML18065A479

Plant TER on IPE Submittal Human Reliability Analysis Final Rept.

ML18065A479
Person / Time
Site:	Palisades
Issue date:	05/31/1995
From:	Haas P CONCORD ASSOCIATES, INC.
To:	NRC
Shared Package
ML18065A476	List: ML18065A478 ML18065A479 ML18065A480
References
CON-NRC-04-91-069, CON-NRC-4-91-69 CA-TR-93-019-15, CA-TR-93-19-15, NUDOCS 9602120376
Download: ML18065A479 (38)

Text

• CA!fR-93-019-15 PALISADES NUCLEAR PLANT TECHNICAL EVALUATION REPORT ON THE IPE SUBMITTAL HUMAN RELIABILITY ANALYSIS FINAL REPORT By:

P. M. Haas Prepared for:

• U.S. Nuclear Regulatory Commission Office of Nuclear Regulatory Research Division of Safety Issue Resolution Draft Report June, 1993 Final Report May, 1995 CONCORD ASSOCIATES, INC.

Systems Performance Engineers 725 Pellissippi Parkway Knoxville, TN 37932 Contract No. NRC-04-91-069 Task Order No. 15

• 9602120376 960207 POR AOOCK 05000255 G PDR

• E. l TABLE OF CONTENTS E. EXECUTIVE

SUMMARY

Plant Cbaracterization . . . . . . . . . . . .

1 1

E.2 Licensee IPE Process . . . . . . . . . . . . . . .. .. .. . . . . . . . .. . . . . .. . . . 1 E.3 Human Reliability Analysis . . . . . . . . . .. .. .. . . . . . . . .. . . . . .. . . . 1 E.3.1 Pre-Initiator Human Events . . . . .. .. .. . . . . . . . .. . . . . .. . . . 1 E.3.2 Post-Initiator Human Actions . . . .. .. .. . . . . . . . .. . . . . .. . . . 2 E.4 Generic Issues and CPI . . . . . . . . . . . . .. .. .. . . . . . . . .. . . . . .. . . . 3 E.S Vulnerabilities and Plant Improvements . .. .. .. . . . . . . . .. . . . . .. . . . 4 E.6 ObservaJtiions . . . . . . . . . . . . . . . . . . . . .. .. .. . . . . . . . .. . . . . .. . . . 4 L IN'IRODUCTION * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1 HRA Review Process . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2 Plant Claaracterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 TI. 1'EOiNICAL REVIEW *.................. ~ . . . . . . . . . . . . . . . . . . . . . . . 9 21 Licensee IPE Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1.1 Completeness and Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1.2 Multi-Unit Effects and As-Built, As-Operated Status . . . . . . . . . . . 10 2.1.3 Licensee Participation and Peer Review . . . . . . . . . . . . . . . . . . . . 10 2.2 Pre-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2.1-* Types of Pre-Initiator Human Actions Addressed . . . . . . . . . . . . . 11 2.2.2 Process for Identification and Selection of Pre-Initiator Human Actions . . . . . . . * . . . . . . . . . . . . *. . . . . . . . . . . . . . . . . . . . . . 11 2.2.3 Screening Process for Pre-Initiator Human Actions . . . . . . . . . . . . 11 2.2.4 Quantification Process for Pre-Initiator Human Actions . . . . . . . . . 12 23 Post-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.1 Types of Post-Initiator Human Actions Considered . . . . . . . . . . . . 13 2.3.2 Process for Identification and Selection of Post-Initiator Human Actions . * . . . * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.3 Screening Process for Post-Initiator Human Actions . . . . . . . . . . . 14 2.3.4 Quantification Process for Post-Initiator Human Actions . . . . . . . . 15 2.4 Vulnerabilities, Insights and Enhancements . . . . . . . . . . . . . . . . . . . . . . . 22 2.4.1 Vulnerabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.4.2 Insights and Enhancements Identified by the Licensee . . . . . . . . . . 23 2.4.3 Important Operator Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Ill. CONTRACTOR OBSERVATIONS AND CONCLUSIONS . . . . . . . . . . . . . . . . . . 26 IV- DATA

SUMMARY

SHEETS . . * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 REFERENCES *. - ~ * ~ . . . * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

• E. EXECUTIVE

SUMMARY

This Technical Evaluation Report "('fER) is a summary of the documentation-only review of the human reliability analysis (HRA) presented as part of the Palisades Nuclear Plant (Palisades) Individual Plant Examination (IPE) submitted by Consumer Power Company (CPCo) to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to assist NRC staff in their evaluation of the IPE and conclusions regarding whether the submittal meets the intent of Generic Letter 88-20.

E.1 Plant Characteri7.ation Palisades is a single-unit two-loop Combustion Engineering (CE) pressurized water reactor (PWR) plant which has been in commercial operation since December, 1971. Plants with similar Nuclear Steam Supply System design include Calvert Cliffs Units 1 & 2. Significant safety features include two power operated relief valves (PORVs), and typical of most PWR PRAs, operator action to initiate bleed and feed operation is one of the important operator actions considered in the analysis. Switchover of ECCS from injection to recirculation mode, which is an important operator action in some PWRs, is automated in Palisades, and is not a significant contributor.

E.2 Licensee IPE Process The submittal discussion of the HRA was limited in detail, and was unclear regarding even basic information, such as the HRA techniques employed. As part of the licensee's response to an NRC request for additional information (RA!), a completely revised HRA discussion (Section 2.3.2 of the submittal) was submitted by the licensee, along with responses to specific questions. The revised write-up and responses to questions provided a description of the HRA methodology that was reasonably complete, though still relatively limited in detail.

NRC subsequently obtained several sample calculations of human error probabilities (HEPs) which helped to clarify further the licensee's approach to quantification of human error. The HRA addressed pre-initiator and post-initiator human actions. In general, numerical results are consistent with, or perhaps toward the conservative end of, the range typically seen in other PRAs. However, the quantification employed generic THERP evaluations or "screening" values and provided somewhat limited opportunity for plant-specific insights.

And, their were several exceptions to the conservative values which happened to be for important actions. The licensee identified some general insights regarding important sequences and contributors to plant risk, some of which were human-performance related.

However, the licensee identified no vulnerabilities and determined that no enhancements were necessary at this time.

E.3 Human Reliability Analysis E.3.1 Pre-Initiator Human Events The Palisades HRA addressed two basic types of pre-initiator human errors that may lead to

• degraded system performance or component failure/unavailability: 1) instrument

• miscalibration, and 2) failure to restore components to their proper alignment position following maintenance or testing. Pre-initiator human errors were qualitatively screened out (eliminated from consideration and not quantified) if it was judged that the error would be detected by an operational test Licensee comments in response to the NRC RAI indicate that some plant-specific evaluation was performed to verify the appropriateness of assumed credit for operational tests. No numerical pre-screening of pre-initiator human actions was performed. All pre-initiator human actions identified and selected for quantification via the qualitative review were assigned an HEP, based on a generic TIIERP evaluation or a generic screening value. The "quasi-generic" approach employed by the licensee provided limited consideration of plant-specific or case-specific performance shaping factors. Consideration was given to dependencies associated with common factors (e.g., similar equipment, same procedures, or same crew) influencing otherwise independent actions. A total of 69 pre-initiator human errors were included. Pre-initiators are included among the top ten most important human actions, based on risk achievement worth.

E.3.2 Post-Initiator Human Actions The Palisades HRA included post-initiator response-t'fPe actions directed by emergency/abnormal operating procedures, including omission of an emergency procedure step, failure to monitor/diagnose plant status, error in response to a system malfunction, and backup system alignment errors. The licensee's definition of recovery actions is limited to unproceduralized actions, and the submittal states that no recovery actions were treated.

However, some proceduralized recovery-type actions taken in response to failed equipment

• were included in the analysis.

Expected post-initiator operator actions were identified from a review of procedures and training, as well as interviews with operations personnel. In the initial quantification, HEPs were assigned a screening value of 1.0E-02 or l.OE-03. This is a relatively low screening value. (A typical value is 0.5 for post-initiator actions.) The licensee used a cutoff frequency of 1.0E-09/yr for this initial quantification. Since dependencies among multiple operator actions were not accounted for in the initial quantification, there is a concern that some potentially significant sequences were screened out and did not receive in-depth analysis. In discussions between NRC and the licensee, the licensee indicated that the sequences below the truncation frequency of 1.0E-09/yr were eliminated for reporting purposes only. They were retained in the model for subsequenct quantification. The dependency analysis, performed at the cutset level after initial quantification addressed cutsets with multiple human actions credited. It is the licensee's belief, therefore, that important operator actions/sequences were not eliminated. We do not have sufficient detailed information to independently assess the licensee's conclusion.

• The submittal and licensee responses to the NRC RAI provide only very general descriptions of the quantification process for post-initiator human actions. Sample calculations provided subsequently by the licensee helped to clarify the licensee's use of THERP. There were sixteen generic THERP evaluations performed for four categories of human actions, including

• 2

two pre-initiator and two post-initiator categories. A task analysis was performed on a "typical" procedure for each category. In a small number of cases (seven), the generic values were modified (reduced) to account for specific performance shaping factors, such as use of a checklist or independent verification. Actions that were not addressed via the generic or the specific THERP evaluations were retained at the screening value (usually 1.0E-02). This "quasi-generic" approach did involve some limited assessment of plant-specific and case-specific performance shaping factors. A number of performance shaping factors, such as procedures, training, and ergonomics were considered at a very gencal level and judged to be nominal or above average. Stress was considered directly in the 11IBU' analysis. Timing was not considered, except as a criterion for determining dependency between two actions.

The 1HERP post-initiator diagnostic model was not applied. Diagnosis error was treated as failure of operator(s) to respond to an alann or annunciator. We consider this limited treatment of diagnosis to be a weakness of the HRA.

As indicated above, a dependency analysis was performed after initial quantification by examining specific cutsets with multiple operator actions. Where dependency was identified, the 1HERP dependency model was used to estimate the quantitative impact.

Recovery actions credited were limited to proceduralized *actions. Recovery actions were incorporated back into fault trees. Usually recovery actions are highly dependent on cutset-specific influences, including other operator actions. When they are modeled in fault trees, it is difficult to account for these dependencies. If dependent actions are treated as random, independent actions and multiplied together, the credit for m:overy actions can be overly optimistic. In the discussions with the NRC staff, the licensee indicated that only in a very limited number of cases were such recovery actions identified after the initial quantification incorporated back into the model.

A total of 170 post-initiator human actions were included in the IPE model. Thirty six of these were quantified at a screening value of l.OE-02 or l.OE-03. All but two of the rest were quantified using either the generic or specific TIIERP calculations. The ASEP procedure was used for the other two. In general, the HEP values an: within the range (or toward the higher end of the range) typically observed in other HRAL It is interesting, though not necessarily significant to the overall HRA results, that one of the actions quantified using ASEP - operator opens the power operated relief valves (PORVs) and their motor operated block valves to establish once-through cooling (bleed and feed) - had the lowest HEP of all post-initiator actions. The licensee assumed the lower bound value for ASEP applies. Limited discussion is provided by the licensee regarding the justification for that assumption.

E.4 Generic I~ues and CPI The licensee addressed USI A-45, "Shutdown Decay Heat Removal Requirements."

Important functional requirements, event initiators, dominant contributors and dominant basic events were identified. The licensee's results show that the contribution to CDF due to 3

• failure of decay heat removal (DHR) is 3.0E-05/yr. Two principal mechanisms for DHR are credited: 1) secondary cooling which utilizes the steam generators as the heat sink, and 2) once through cooling. Operator actions are identified as important for both mechanisms.

E.5 Vulnerabilities and Plant Improvements Based on general screening criteria presented in the submittal and expanded discussion and clarification provided by the licensee's response to the NRC RAI, the licensee's concept, definition and criteria for identifying vulnerabilities appear to be consistent with guidance in NUREG-1335. Although no vulnerabilities were identified, The submittal discusses five "insights" - items that contribute significantly to core damage frequency and/or release of radioactivity: *

(1) Small break loss of coolant accident (SBLOCA)

(2) Single Failure of HPSI with SBLOCA (3) Loss of Offsite Power (4) Condensate Storage Tank Makeup (5) Safety Inject and Refueling Water Tank Makeup.

All five of these insights have human-performance-related aspects to some degree. In particular, makeup to the condensate storage tank (CST) was the most important operator action. A sensitivity study performed by the licensee indicated that increasing this HEP an order of magnitude increases the CDF by a factor of 3. This particular HEP (2.9E-04) is one of the lower HEPs in the model. The licensee considered these five items for enhancement, but determined that no enhancement was warranted. In our view the rationale provided by the licensee for this determination (i.e., operators have training on this action and are directed by emergency operating procedures) is rather limited, particularly for the CST makeup, which is a very important action with a relatively low HEP estimate.

E.6 Observations It is our general observation from the review of the submittal, the additional material provided by the licensee in response to NRC requests for additional information, and the responses to .

the subsequent telephone conference call between the NRC and the licensee, that the licensee's HRA is reasonably complete in scope, and did provide the licensee with some insights regarding the relative quantitative impact of human error in severe accidents, but that there are a number of weaknesses in the approach that limit the benefits obtained by the licensee from the analysis. Specific observations most pertinent to NRC's decision regarding whether or not the licensee met the intent of Generic Letter 88-20 are:

(1) The submittal and supporting documentation indicates that utility personnel were involved in the HRA, and that plant walkdowns, documentation reviews, and interviews with operations and training staff constituted a viable process for confirming that the HRA portions of the IPE represent the as-built, as-operated

• 4

• (2) plant.

The licensee performed a peer review, including an internal review and independent assessment by a consultant, that provides some assurance that the HRA techniques have been correctly applied and that documentation is accurate.

(3) Pre-initiator human actions were considered in the analysis. Both _miscalibration and restmation errors were addressed. The process for identification and selection of pre-initiator human errors included review of procedures and interviews with plant personnel. Numerical results in general were consistent with, or (in cases where "screening" values were used) slightly more conservative than, nominal values in other PRAs.

(4) The treatment of pre-initiator and post-initiator errors using the "quasi-generic" THERP models limits the degree of in-depth insights about plant-specific factors influencing human perfonnance. Such plant-specific insights are valuable products of the HRA.

(5) The treatment of post-initiator human actions included both response-type and recovery-type actions. The process for identification and selection of actions included review of procedures and discussions with plant personnel.

(6) The licensee used screening values for post-initiator human actions that are significantly lower than typically used for post-initiator actions (0.01 or 0.()01 vs.,

say 0.5) 2!Ild did not include dependencies in the initial quantification. These facts together raise a concern about the potential for screening out risk-significant actions/sequences. Based on telephone discussions between the licensee and NRC, the licensee believes that it is unlikely risk-significant actions/sequences were screened out because (a) sequences with very low frequencies were retained in the model even though they were not reponed, and (b) dependencies were treated in the subsequent model quantification. While the licensee's argument is plausible, the documentation of the licensee's approach and justification is very limited. We are not able to confirm whether or not the approach provided reasonable assurance that actions/sequences were not inappropriately screened out.

(7) The treannent of diagnosis for post-initiator actions is not consistent with most nuclear plant HRA approaches. Most post-initiator human actions in the Palisades model are quantified using THERP. Based on our limited review of several sample "generic THERP" calculations, it appears that the Ilcensee did not use the THERP diagnostic model but instead treats all post-initiator response as rule-based responses to alarms/annunciators. We recognize that THERP and other HRA techniques employ highly simplified models of the dynamic processes involved with human action in response to an accident condition. And, we recognize that there is a considerable plausibility for an argument that current use of "symptom-based"

• 5

• procedures substantially reduces the cognitive demands on the operators. However, most HR.A approaches treat the "cognitive" actions associated with diagnosis, detection, decision-making, etc. distinctly different, often using a time-based probability. None of the Palisades documentation reviewed discusses the rationale for excluding diagnosis. In our view, the limited discussion/treatment of the diagnosis portion of the response represents at least a- limitation in the documentation of the HRA, and may reflect a weakness in the licensee's understanding of human behavior in severe accidents.

(8) The licensee identified important actions and their quantitative impact on core damage and radioactive material release. And, based on the licensee's definition of vulnerabilities, none were identified and no enhancements were "necessary/appropriate." It is our opinion that some of the _potential enhancements might have been pursued more rigorously. However, the HRA process did permit the licensee to identify potential vulnerabilities and enhancements related to human performance.

(9) In general, quantitative results for posi-initiator actions are consistent with norriinal values reponed in other PRAs, and consistent with (or toward the conservative end of) the range typically generated using THERP. As indicated above, it is notable, though it may not be significant to the overall results and conclusions of the HRA, that one of the two exceptions in which ASEP was used instead of TIIBRP the calculated HEP is the lowest value of all post-initiator HEPs, and is among the most

• imponant human actions. Typically, ASEP would be expected to produce higher (more conservative) values .

• 6

I. INTRODUCTION This Technical Evaluation Repon (1'ER) is a summary of the documentation-only review of the human reliability analysis (HRA) presented as pan of the Palisades Nuclear Plant (Palisades) Individual Plant Examination (IPE) submittal from the Consumer Power Company (CPCo) to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to assist NRC staff in their evaluation of the IPE and conclusions regarding whether the IPE submittal meets the intent of Generic Letter 88-20.

• 1.1 HRA Review Process The HRA review was a "document-only" process which consisted of essentially four .steps:

(1) Comprehensive review of the IPE submittal focusing on all information pertinent to HRA.

(2) Preparation of a draft TER summarizing preliminary findings and conclusions, noting specific issues for which additional information was required from the licensee, and formulating requests to the licensee for the necessary additional information.

(3) Review of preliminary findings, conclusions and proposed requests for additional information (RAls) with NRC staff and with "front-end" and "bac~-end" reviewers

• • (4) Review of licensee responses to the NRC requests for additional information, and preparation of this final TER modifying the draft to incorporate results of the additional information provided by the licensee and finalize conclusions. Additional clarification was provided by particpation in a teleconference between the lic~nsee and NRC staff.

Findings and conclusions are limited to those that could be supported by this document-only review. No visit to the site was conducted. In general it was not possible, and it was not the intent of the review, to reproduce results or verify in detail the licensee's HRA quantification process. The review addressed the reasonableness of the overall approach with regard to its ability to permit the licensee to meet the goals of Generic Letter 88-20.

1.2 Plant Characterization Palisades is a single-unit two-loop Combustion Engineering (CE) pressurized water reactor (PWR) plant which has been in commercial operation since :December, 1971. Plants with similar Nuclear Steam Supply System design include Calvert Cliffs Units 1 & 2. Significant safety features include two power operated relief valves (PORVs), and typical of most PWR

• 7

_____________________________ _J

PRAs, operator action to initiate bleed and feed operation is one of the important operator actions considered in the analysis. Switchover of ECCS from injection to recirculation mode, which is an important operator action in some PWRs, is automated in Palisades, and is not a significant contributor.

• 8

• 2.1 2.1.1 Licensee IPE Process II. TECHNICAL REVIEW Completeness and Methodology The original submittal discussion of the HRA was limited in detail, and was_ unclear regarding even basic information, such as the HRA techniques employed. As part of the licensee's response to NRC requests for additional information, a completely revised HRA discussion (Section 2.3.2 of the submittal) was submitted by the licensee, along with responses to specific questions. The revised write-up and responses to questions provided a description of the HRA methodology that was more complete than .the original submittal. A telephone conference between NRC staff and the licensee and follow-up transmittal of sample HEP calculations provided further clarification of some of the issues identified from the initial review.

The HRA addressed both pre-initiator and post-initiator human actions. Pre-initiator actions included both restoration errors and miscalibration. Post-initiator actions included response-type actions directed by emergency/abnormal operating procedures, including omission of an emergency procedure step, failure to monitor/diagnose plant status, error in response to a system malfunction, and backup system alignment errors. In response to an NRC RAJ, the licensee states that, "The Palisades PRA does not include operator recovery

• actions." However, the licensee's definition of recovery actions provided in the revised submittal is more restrictive than for most PRAs. The Palisades HRA does treat proceduralized responses to failed equipment which other PRAs typically have termed recovery actions. The Palisades HRA does not treat actions that are not proceduralized or are not extensively covered in training. This distinction is discussed further in Section 2.3.1 below.

• In general, human error probabilities were derived using the procedures and data from the THERP handbook (Ref. 1). Sixteen "generic" THERP evaluations were performed for four broadly defined types of errors: (1) pre-initiator restoration errors, (2) pre-initiator calibration errors, (3) post-initiator errors in response to a control room alarm, and (4) post-initiator errors in aligning systems. In seven cases (unspecified as to which actions or whether they were pre- or post-initiators) the generic estimates were modified (reduced) to account for case-specific differences such as the use of a checklist. Numerical estimates, in general, are within the range typically seen in other PRAs. A "post-quantification" review was performed to identify specific cases of dependency, i.e., one human action dependent on the success/failure of a preceding action. Where a dependency was judged to exist, HEPs were modified and the model requantified.

No wlnerabilities and no enhancements were identified by the licensee .

• 9

2.1.2 Multi-Unit Effects and As-Built, As-Operated Status Palisades is a single-unit plant. The issue of multi-unit effects is not applicable.

It is evident from the text of the submittal that the licensee understood the importance of confirming that the IPE represents the as-built, as-operated plant, and that a significant effort was expended to assure that the basic input data and information used in the_ IPE was current.

It is also apparent, in particular for the back end analysis, that the licensee paid special attention to the unique features of Palisades design. The discussion of information assembly in Section 1.2 of the submittal states that "All modifications installed before July l, 1992 are included in the IPE." Section 1.2 also mentions walkdowns conducted to obtain the latest and most accurate information available regarding the as-built configuration of the plant. The text further states that "During the walkdowns, particular attention was given to the development of the human reliability analysis. Results of the walkdowns, reviews of plant procedures, and discussions with the operations and training personnel provide the basis for the IPE modeling." While there is no additional detailed discussion that permits conclusions specific to the HRA with regard to this issue, it appears that the licensee had a viable process for assuring that the IPE represents tl-ie current plant design and operating practice.

2.1.3 Licensee Participation and Peer Review Utility staff involvement and the independent review process both are discussed in Section 4 of the submittal. The IPE was performed by the PRA group at Palisades with consulting support from Tenera L.P. and ABB Impell on the front end analysis and Tenera and Gabor, Kenton and Associates on the back end. An internal independent review was led by the Plant Safety Engineering Group. Additional review by "consultants" is cited, though it is not clear whether these were the supporting consultants noted above *or other consultants. No specific results of the review or modifications to the IPE based on the review were cited. Specific HRA or human factors expertise is not identified for utility team members, consultants, or internal review group members. The submittal does state that the human error analysis team members included a human factors analyst, systems analysts and a licensed senior reactor operator. The licensee's response to the NRC RAI indicated that an independent check on the HEP estimates was performed by Tenera staff using ASEP. In the original IPE submittal, it appeared that ASEP was used as an alternative method for calculating some HEPs. However, the licensee's response clarified that ASEP was used only in this independent check by Tenera staff. (At least three post-initiator HEPs ultimately were quantified using ASEP.)

While details are somewhat limited, it appears that the licensee did participate directly in the HR.A process and that an independent review process was implemented which helped to assure the reasonableness of the HRA modeling. *

• 10

• 2.2 2.2.1 Pre-Initiator Human Actions Tvoes of Pre-Initiator Human Actions Addressed As indicated above, the Palisades HRA addressed two basic types of pre-initiator human errors that may lead to degraded system performance or component failure/unavailability: 1) instrument miscalibration, and 2) failure to restore components to their proper alignment position following maintenance or testing.

2.2.2 Process for Identification and Selection of Pre-Initiator Human Actions The revised submittal (page 2.3-15) states that, "Specific pre-accident [pre-initiator] human errors were identified for inclusion in the PRA by the potential for cc;>mmitting or detecting the error. Insignificant human errors were not included in the PRA." This comment indicates that potential pre-initiator human error contributors to degraded system performance or unavailability were evaluated qualitatively to eliminate those that were judged to have a very low likelihood of error and/or a high likelihood of detection of the error prior to demand for the affected system. Our interpretation of licensee responses to the NRC RAJ is that pre-initiator human errors were eliminated from consideration and not quantified if it was judged that the error would be detected by an operational test This is a reasonable assumption and one that is typically made in HRA. In eliminating errors from consideration, however, it is important for the analyst to verify that the operational test (or any detection mechanism credited) will have a high likelihood of being performed correctly and will identify the specific human error and component under consideration. For example, a requirement for a pump operability test may or may not assure detection of misalignment of a particular valve associated with that pump. The licensee's response to the NRC RAI regarding this issue indicated that the licensee did consider such factors. An example provided by the licensee is the case of, "A pump taken out of service for maintenance, and a full flow test through the minimum flow line is performed prior to pump operability." In this example, only the pump inlet isolation valve and the pump minimum flow valve are verified to be open, and human error in misaligning those valves was eliminated from consideration.

However, the pump discharge isolation valve may not have been tested, and if not, a human error is included in the model for that valve. Based on this limited information in the revised submittal and the licensee's response to the NRC RAI, and also based on the relatively large number of pre-initiator actions included in the model, it appears that the licensee's process for identification and selection of pre-initiator human actions was reasonably comprehensive.

2.2.3 Screening Process for Pre-Initiator Human Actions No numerical pre-screening of pre-initiator human actions was performed. The revised submittal indicates that all pre-initiator human actions identified and selected for quantification via the qualitative review discussed above were assigned an HEP, either an assumed "screening" value, or a value calculated using TIIERP. As part of the response to the NRC RAI, the licensee provided a table c:>f all HEPs included in the IPE model. A total

• • 11

of 69 pre-initiator human errors were included. All but three were quantified using TIIBRP; the other three were quantified using the screening value of 1.0E-02. This screening value is in the range typically assumed for screening values for pre-initiator human errors. For example, a value of 0.03 is recommended in ASEP (Ref. 2). All other pre-initiator HEPs are quantified at lower values, most in the range of 1. lE-04 to 2.0E-03, with a few at lower values. These values are, in general, comparable to typical values for nominal HEPs for pre-initiator actions in other PRAs. The screening value of 0.01 is at least an order of .

magnitude greater than the other calculated values. The truncation level for eliminating sequences after the initial quantification was lE-09. Thus the use of the l.OE-02 screening value for pre-initiator actions probably did not eliminate significant sequences.

2.2.4 Quantification Process for Pre-Initiator Human Actions As indicated above, HEPs for pre-initiator actions were derived using the procedures and data in the 1HERP handbook, with the exception of the three HEPs assigned a screening value of l.OE-02. Generic THERP evaluations were performed for two categories of pre-initiator human actions: restoration errors and miscalibrations. The revised submittal notes that sixteen generic TIIERP evaluations were performed for four categories of actions (including the two pre-initiator categories and two post-initiator categories.)* However, it is not clear how many of the generic evaluations were for pre-initiator actions. The revised submittal includes a general discussion of the process for the generic THERP evaluations applicable to both pre-initiator and post-initiator actions. The process included a task analysis using a "typical" procedure or human action representative of the category. The task analysis identified

• specific task errors that may prevent successful completion of the procedure or action. The submittal states that this task analysis provided a bounding model for each category, "which resulted in conservative THERP evaluations for the human errors in each category." No information is provided as lo why this approach necessarily provided bounding models or conservative estimates.

Regarding the development of HEPs, the revised submittal states (page 2.3-17) simply that, "Probability estimates for each task error were derived using the procedure and data from NUREG/CR-1278" (Ref. 1); and, that "For task errors where a range of probabilities was given. a conservative estimate was used based on PSFs." The meaning of this latter statement is unclear. We believe it is meant to say that in cases for which THERP guidance may have provided a range of values, depending on plant-specific and situation-specific performance shaping factors, the analysts selected the more conservative (higher) values in the range, presumably in lieu of case-by-case assessment of performance shaping factors.

The submittal also notes (page 2.3-17) that, 'To account for* slight differences in some actions (i.e.* use of a checklist, controlled by a procedure, use of a verifier, etc.) the generic TIIBRP models were requantified using different probabilities." Seven of these "specific THERP evaluations" were performed. No further detail is provided in the revised submittal or in the

• 12

• licensee responses to NRC RAls as to the precise factors considered or how they were quantified. There is no identification of which HEPs were generated from the generic TIIBRP model vs. the specific evaluation.

No sample calculations are provided by the licensee. No details are provided regarding specific sources (TIIERP tables) :used for the basic HEP estimates or performance shaping factors used to modify the basic HEPs are not identified. Therefore, it is not possible to verify that licensee's implementation of the THERP process was consistent With the guidance in Reference 1 and/or typical praaice in other PRAs.

In response to an NRC RAI, the licensee does discuss consideration given to dependencies in pre-initiator human actions. The :analysis identified some pre-initiator actions that are "completely dependent", i.e., failme of the first action necessarily fails the second action.

These dependencies were identified by reviewing procedures, tasks, or intended functions of the human action for similarities. An example provided by the licensee is the case of two calibrations perfonned by the same technician, using a. similar procedure and the same calibration equipment. In these cases, the two dependent errors were treated as the same basic event. which is one reasonable approach to treat such dependencies. The revised submittal (Section 2.3.3.4) discusses the general treatment of such dependencies in the assessment of pre-initiator human actions as follows:

"Human errors of calibration were carefully examined to determine the :widespread applicability to the equipmenl being examined. An analysis of the calibration procedure determines approprfiate common failure to the equipment being calibrated, with the resulting common luuna.n error events being explicitly included in the fault tree model. [Restoration errors} were examined on a similar basis. A review of operator actions required to accomplish a function determines the commonalty of the human error events modeling the failure to complete these actions. The resulting common human error events were also explicitly included in the fault tree models."

Based on this general discussion and the comments in the licensee's response, it appears that dependencies in pre-initiator actions were considered.

2.3 Post-Initiator Human A<Otions 2.3.1 Types of Post-Initiator Human Actions Considered.

As indicated in Section 2.2.1 above, the IPE model included post-initiator response-type actions directed by emergency/abnormal operating procedures, including omission of an emergency procedure step, failure to monitor/diagnose plant status, error in response to a system malfunction, and backup system alignment errors. The revised submittal states that recovery actions were not included in the Palisades PRA. The licensee's comments in response to the NRC RAI indicate that the licensee's definition of recovery actions is restricted to those that are not governed by procedures or extensively covered by training. No

• 13

• such non-proceduralized or untrained actions were included in the IPE model. Post-initiator actions that are directed by procedures - whether emergency operating procedures, off-normal operating procedures, standard operating procedures, or general operating procedures - were included in the IPE model (primarily fault trees), including actions that involve response to failed equipment or functions, and including actions that were identified from review of initial quantification results. Thus, the Palisades HRA does treat some actions that might be classified as recovery actions in other PRAs; but such actions are treated in the same way as "response-type" actions. This difference may or may not be significant. The difference in terminology is not important. However, incorporation of recovery actions into fault trees makes it difficult to account for cutset-specific dependencies. This issue is discussed further in Section 2.3.4.6 below.

2.3.2 Process for Identification and Selection of Post-Initiator Human Actions.

Specific information regarding the process for identification and selection of post-initiator actions is limited. The revised submittal and licensee responses to the NRC RAI state that expected post-initiator operator actions were identified from a review of procedures and training, as well as interviews with operations personnel. Types of post-initiator actions considered included misalignment of backup component/systems, misdiagnosis of an accident, failure to properly respond to system malfunction, and failure to properly perform a human action. Review of the systems analysis and sequence analysis sections of the submittal (Sections 2.1 through 2.4) indicates that human action was considered as an integral part of those analyses. In general, operator actions identified by the front-end reviewers as important were addressed in the HRA, and the listing of operator actions included in the revised submittal appears to be reasonably comprehensive. Based on these very general statements and indicators, it appears that the process for identification and selection of post-initiator actions was reasonably comprehensive.

2.3.3 Screening Process for Post-Initiator Human Actions.

In response to the NRC RAI, the licensee states that no pre-screening of human errors was performed in the Palisades HRA. The revised submittal (page 2.3-17) states that "Screening values of 1.0E-02 and l.OE-03 were used in the preliminary quantification." The licensee's response to the NRC RAI also notes that seql,lences were truncated at a value of l .OE-09/yr or l.OE-10/yr, which is lower than typically used. Dependencies among human actions were not accounted for in the initial quantification, but instead were examined in a "post-quantification" review. This raised a concern that even with the low truncation frequencies, some risk-significant actions/sequences may have been eliminated in the initial screening. In the telephone discussion between NRC and the licensee, it was Clarified that the sequence screening criterion of l.OE-09/yr was usedfor reponing purposes only. That is, sequences with contributions several orders of magnitude lower were kept in the model, even though they were not reponed. Because of this, and because dependencies among multiple actions

• 14

• ultimately were accounted for, the licensee believes that imponant actions/sequences were not screened out. The licensee's argument is plausible, but it is not possible for us to confirm or refute it without funher examination beyond the scope of this document-only review.

2.3.4 Quantification Process for Post-Initiator Human Actions.

2.3.4.1 General Process. The submittal and licensee responses to the NRC RAJ provide only very general descriptions of the quantification process for post-initiator human actions. As indicated previously, there were sixteen generic TIIBRP evaluations performed for four categories of human actions, including two post-initiator categories. There is no indication of the breakdown of generic models vs. categories. The grouping of human actions into a generic category was determined on the basis of similarity in "content and objective" of the human actions. A task analysis was perfonned on a "typical" procedure for each category.

The licensee contends that (but does not explain how) this task analysis provided a "bounding model for each category, which resulted in conservative IBERP evaluations for the human errors in each category." Four sample THERP calculations provided by the licensee in response to the teleconference with NRC included these actions:

(1) Failure to perfopn a function upon a system malfunction (change position of a valve, stan a pump, etc.)

(2) Failure to align a faulted safety injection actuation signal (SIAS) component

,):

\.'.' (3) Failure to align a system

_/

(4) Failure to close an atmoshpheric dump valve (ADV) on an intact steam generator.

Apparently, these actions (at least the first three) are some of the generic THERP actions used to represent groups of similar actions. Our review of those calculations indicates they employ essentially "generic" THERP values from tables (e.g.* failure to acknowledge an annunciator or failure to use a checklist) with limited evaluation of plant-specific performance shaping factors. Stress level and dependency between two operators are considered explicitly. (It should be noted that there seem to be some inconsistencies in the the citations for the IBERP tables used in the sample calculations; i.e., the tables cited do not always appear to be the appropriate tables. This appears to be simply a problem of documentation.)

The submittal indicates that in some cases, the generic values were modified (reduced) to account for specific performance shaping factors, such as use of a checklist or independent verification. As indicated previously, the submittal notes that there were seven such "specific" THERP evaluations, though it is not clear whether these seven were all post-initiator actions. Actions that were not addressed via the generic or the specific lHERP evaluations were retained at the screening value, except for three actions quantified using ASEP. A total of 170 post-initiator human actions were included in the IPE model. Thiny six of these were quantified at a screening value of 1.0E-02 or 1.0E-03. The revised

• 15

• submittal states that the l.OE-02 value was assumed for cases in which an opponunity for recovery of the human error is not probable, and the 1.0E-03 for cases in which error recovery was considered likely. One operator action, "Operator fails to reduce SWS loads after bus lD failure," was assigned a screening value of 1.0E-01 (assuming this is not simply a typographical error in the table). There is no discussion of why this was assigned a higher screening value.

A weakness of the licensee's HRA approach, in our view, is the limited treatment of the "cognitive" aspects of the operator's response to an accident, i.e., diagnosis, detection, decision-making. Our review of the sample THERP calculations indicates that the licensee did not employ the THERP diagnostic model, which is the usual approach to treating this phase of the operators' response when using THERP. Instead, the licensee treats all actions as "rule-based" responses. Diagnostic action is treated essentially as .failure of one operator to respond to a single alarm or annunciator, and a second operator is treated as a backup to the first operator (with dependence accounted for). None of the material presented by the licensee discusses the treatment of diagnosis or provides a rationale for this approach.

Arguments have been made by some analysts that the current "symptom-based" procedures have drastically reduced or eliminated the need for diagnosis or cognitive action on the pa.."'t of the operators. It is our view that while the symptom-based procedures appear to be a more effective aid to diagnosis than the old "event-based" procedures, the dynamic process of diagnosis and action following an event is not appropriately treated by the simplistic approach employed by the licensee. The absence of discussion/treatment of diagnostic action is at least a weakness in the documentation, and may indicate a weakness in the licensee's understanding of human behavior in response to an accident situation ..

2.3.4.2 Use of ASEP. The original submittal implied that ASEP (Ref. 2) was used to calculate some HEPs. In the revised submittal and the responses to the NRC RAI, the licensee clarified that ASEP was used by a consultant (Tenera) to provide an independent check on the values calculated using THERP or screening values, and that none of the TIIBRP or screening values were changed as a result of the ASEP analysis. However, two HEPs are listed in the revised submittal (Table 2.3-7) as being calculated by ASEP1:

(1) DBATTSHED - Operator fails to strip non-essential loads from station batteries (HEP= 4.50E-03)

(2) RPORVOA - Operator fails to open the PORVs (HEP= 8.lOE-05) 1 In the licensee's response to the NRC teleconference, an ASEP calculation was provided for a third action SRECIRCOX, "Operator fails to align sump suction." This third action was listed in Table 2.3-7 of the submittal with a screening value of l.OE-02. The ASEP-calculated value was 2.8E-02. The licensee noted that the difference in CDF from this change in HEP value was minimal.

• 16

These two actions were identified after the final IPE quantification by the licensee's consultant in their independent review of the PRA. The first action, DBATISHED, is credited in station blackout sequences. Load shedding extends battery life from two to six hours. Battery depletion leads to failure of auxiliary feedwater pumps. A sensitivity study performed by the licensee indicated that the failure to strip non-essential loads leads to an increased CDF of less than 1%. The calculated risk achievement worth is 2.39, which is the fifteenth highest risk achievement worth for all operator actions.

The action RPORVOA involves opening both the PORVs and the associated block valves, and is a critical action in establishing once-through cooling (bleed-and-feed). It was noted in the original submittal as being an important operator action, with the third highest risk achievement worth of all operator actions. (In the revised submittal, the risk achievement worth, which is the ratio of the CDF with this basic event at probability 1.0 to the CDF with this basic event at the assigned HEP of 8. lE-05, is 49.9. It is now listed as the fifth most important human action, based on risk achievement worth.) The fact that an operator action of this importance was not identified until review of the PRA well after the final quantification for the IPE decreases our confidence in our judgment that the process for identification and selection of human actions was comprehensive. However, it is recognized that the PRA is a continuing process, and additional changes will occur.

It is interesting to note that this value is by far the lowest post-initiator HEP in the model, one of only a few HEPs that are below l.OE-03. In comparison to TIIERP, ASEP is intended to be a simplified procedure which produces generally conservative values. In this case, the value is two to three orders of magnitude lower than most of the values calculated by TIIBRP. More significantly, based on comparison with other PRAs, the value calculated by ASEP does not appear to be conservative for the action of opening PORVs and block valves.

The licensee reports results of a sensitivity analysis on this action that resulted in "an increased core damage frequency of less than 1%. " The sensitivity calculation is not described in any further detail. This relatively low impact on CDF due to inability to establish once-through cooling is somewhat unusual. Failure to initiate* bleed-and-feed usually is a more significant contributor to CDF in PWRs. The sensitivity study could have increased the HEP significantly (say as much as an order of magnitude) and still involved an HEP substantially lower than most other HEPs, and lower than equipment failures associated with the PORVs and/or block valves. In that case, the increase in CDF would still be relatively insignificant. Since this HEP is of high importance and the HEP is obviously much lower than all the other HEPs, it would seem prudent for the licensee to carefully evaluate the calculated estimate to assure it is realistic, or use a more conservative value typical of the other HEPs and evaluate the need for potential enhancements.

2.3.4.3 "Refinement" of HEPs. The revised submittal (Section 2.3.2.3.4) also discusses a "refinement" of human error probabilities that was conducted after the preliminary quantification. This review considered the impact of human errors on the estimated CDF vs.

the resources required to conduct a more detailed analysis. No changes to HEPs were made.

The licensee contends that this is conservative because the generic THERP calculations and

• 17

• the screening values employed are conservative in comparison to what would have been calculated with a "refined" analysis. It is clear that lower HEPs could have-been calculated using TiffiRP. The question of "conservatism" in comparison to "realistic" values is an open issue, however, since there is no firm empirical basis for comparison. The generic TIIERP values are at the higher end of the range typical for detailed THERP calculations, but they are not, in general, overly conservative in comparison to nominal values typically seen in PRAs using THERP, ASEP, SLTh1 (Ref. 3), EPRI methodology (Ref.* 4) or other HRA techniques.

2.3.4.4 Consideration of Timing in Post-Initiator Actions. The submittal discussion of operator response times, performance shaping factors and dependencies was extremely limited The NRC requested additional information on the basis for estimates of timing of operator actions, on performance shaping factors considered, and on treatment of dependencies in post-initiator human actions. Regarding timing, the Jicensee's response to the NRC RAJ states that, Timing was not considered for identification and quantification of post-accident [post-initiator] human errors nor for refinement of human errors. Timing was considered in the post-quantification review of post-accident human errors." The post-quantification review involved identification and treatment of dependencies among multiple human actions in a sequence, which is discussed below. One of the criteria for dependency was the time available and/or the time frame in which the required action was performed In most HRA techniques, the probability of failure to properly diagnose an event is strongly affected by the timing, i.e., the time available vs. time required. Some HRA techniques and models, including the THERP diagnostic model, use time reliability correlations from simulator studies and/or judgment as essentially the sole means to determine the probability of failure to correctly diagnose the situation. As discussed above, the Palisades analysis did not treat diagnosis directly.

Timing was used as a key parameter in the ASEP analysis *performed by the licensee's consultant, bm those calculated HEPs were, with the exceptions discussed above, not used in the IPE model. The timing estimates used by the consultant were developed by averaging estimates obtained from interviews with operators and simulator training staff. No measurements of timing, e.g., timed walkdowns or simulator exercises were performed for the IPE.

2.3.4.5 Performance Shaping Factors Considered for Post-Initiator Actions. In the response to the NRC RAJ, the licensee stated that, "PSFs include a number of factors such as stress, crew/operator experience levels, environment (lighting, noise), workload, procedural quality, leveVquality of training, and plant human factors/ ergonomics."

The licensee cites Chapter 20 of Reference I as a basis for assuming that "temperature, noise level, lighting, and similar factors are normally above average at nuclear plants." This is probably a valid general assumption for current control room environments. It is not at all

• 18

• clear that this is a valid assumptions for out-of-control room actions. The licensee's response to a second NRC RAI indicated that there was only one recovery action that takes place outside the control room. It is not clear how many response-type actions were in- vs.

out-of-control room.

The licensee's response also states that ergonomics factors are considered to be above average at Palisades because of the Detailed Control Room Design Review and a number of other evaluations performed in response to various NRC/industry programs and coi-porate actions.

Administrative controls were considered to be above average due to improvements in EOPs and Off-Normal Procedures and continuing participation in the CE Owners Group Operations Subcommittee. Operator training is assumed to be above average due to INPO accreditation, plant-specific simulator training, and job-specific performance measures which the operators are required to demonstrate during requalification training. Since av~rage crew/operator experience at Palisades is well over six months (identified in Ref. 1 as a minimum level for "experienced" personnel) the licensee assumed that all tasks were performed by experienced operators (nominal assumption for this PSF). Stress was considered as a performance shaping factor, in most cases assuming moderately high stress. No further discussions were provided regarding the evaluation or quantification of the impact of PSFs.. In essence, all of the PSFs except stress were considered to be nominal or above average. Our review of the sample calculations provided by the licensee in response to the NRC teleconference confirmed that stress and dependency between two control room operators were essentially the only PSFs directly quantified. This generic analysis limits the ability of the licensee ro gain information

• on plant-specific influences on human behavior in severe accidents, which is one of the

• important potential products from the IPE.

The licensee's response to the NRC RAI indicates that a sensitivity study was performed to evaluate the uncertainty associated with the impact of stress on human error and, consequently, CDF. In this sensitivity study it was assumed that the screening values were adequately conservative to represent high stress, and those HEPs with based on screening values were not adjusted. The sensitivity study increased the individual HEPs for the important human errors (those with risk achievement worth greater than 1.02). An increase of a factor of ten in one human error (XOOOTCST), failure to align makeup to the condensate storage tank (CST), would result in an increase in CDF of a factor of 3.08. This human action, which is modeled in sequences using auxiliary feedwater, is listed as the most important operator action, with a risk achievement worth of 784 (a factor of 784 increase in CDF if the HEP were 1.0 instead of the estimated value of 2.9E-04). The HEP estimate is one of the lowest values of all of the HEPs in the IPE model, one of only a few below l.OE-03. The licensee notes that this action is required upon depletion of the CST, which occurs approximately three hours into the event, and contends that since support would be available from the emergency response organization by that time. the increase of a factor of ten (to 2.9E-03) is "very conservative". As indicated previously, the question of degree of conservatism in HEP estimates involves considerable judgment, and is not readily quantified.

However, the value of 2.9E-04 is not unusually conservative in comparison to typical values used in other PRAs. The licensee's argument that the time into the event is long, and that

• 19

• support would be available from the emergency response organization is valid. However, given that this action was identified as the most important human action in the model (certainly one of the most important basic events), that the HEP is approximately an order of magnitude lower than most of the other calculated HEPs, and given the uncertainty in assessing the impact of high stress in a severe accident situation, it would seem prudent that the licensee consider whether potential enhancements such as improved procedures or training are warranted, or at least perform a more in-depth assessment of the situation-specific conditions affecting this human action in order to decrease the uncertainty associated with the estimate.

The licensee's response states that factor of ten increases in two other individual actions (AOPERLOCOT and GA VOADUMP) increased the CDF by a factor of 1.2 and 1.24 respectively, and that increases in all of the other important human errors resulted in increases of less than 20% in the CDF. The licensee notes that the same conditions applicable for XOOOTCST, i.e., long time into the event and presence of the emergency response organization, would be applicable for these two actions, and that the increases in HEPs are also very conservative. The event AOPERLOCOT is operator failure to align the alternate AFW pump suction source. The HEP estimate is l.OE-03 (a "screening" value), and the calculated risk achievement worth is 23.5. The event GA VOADUMP is operator failure to open the atmospheric dump valves or turbine bypass valves in sequences using low pressure feed. The estimated HEP is 3.4E-02 (calculated from THERP).

• Note that this is an example in which the assumed "screening value" for one post-initiator action is lower than the THERP-calculated value for another post-initiator action. Obviously, either estimate could be "conservative" or "optimistic", depending on the action involved and the specific conditions associated with the action; without information on a case-by-case assessment, it is difficult to judge the realism of the individual HEPs. Note also, that the sensitivity study addressed each HEP individually. The combined effect of increasing all HEPs, perhaps much less than a factor of ten, may have a significant impact on the estimated

. CDF. One of the intents of the IPE process is for the licensee to gain an overall understanding of the quantitative impact of key factors influencing core damage and release of radioactivity. Given that essentially "generic" THERP values or "screening" values were employed for HEPs, sensitivity studies to assess uncertainties in the human error probabilities, such as the one discussed above, are important, and perhaps should be expanded to provide a better overall understanding of the contribution of human error to plant risk, and to identify potential enhancements for further consideration.

In summary, the submittal and licensee responses indicate that the licensee performed relatively little plant-specific or situation-specific assessment of performance shaping factors, and limited sensitivity studies to explore the impact of uncertainties in human error probability estimates. These factors tend to limit the licensee's understanding of the impact of human performance on risk in their plant, and their understanding of the factors that influence human performance in severe accidents .

• 20

2.3.4.6 Consideration of Dependency in Post-Initiator Actions. In response to the NRC RAJ*

the licensee identified three aspects of the treatment of dependencies in post-initiator human actions:

(1) Complete dependency between two actions. In some cases, it was determined during the qualitative review process of identifying and selecting important operator actions that more than one operator action were essentially completely dependent.

and those actions were grouped together and represented as one basic event in the IPE model.

(2) Cognitive dependence. In some cases there are alternative methods to accomplish a given goal or function which involve execution actions that are essentially .

independent but cognitive actions (detection, diagnosis, decjsion making) that are highly dependent. In those cases, all of the cognitive actions were modeled as a single separate action, which is equivalent to treating the cognitive portion as completely dependent (3) Dependencv among multiple actions in a cutset. A dependency analysis was conducted after the model was quantified to ide.ntify dependencies among multiple human actions in a given cutset. Any cutset containing more than one operator action was evaluated to identify potential dependencies. The IBERP dependency model was applied to modify (increase) the estimated HEP for dependent actions.

The revised submittal included a listing (Table 2.3-10) of results of this post-quantification dependency assessment, including the dependent actions, the assumed degree of dependence (low, moderate, or high), the original HEP estimate, and the modified HEP. The modified HEP values are consistent with the TiffiRP guidance. In some cases, the licensee noted that human actions in the IPE model have since been replaced by automated action, and the HEP has been assigned a value of 0.0.

This "post-quantification" assessment of dependencies at the cutset level is not the usual approach taken to address dependencies. As indicated in Section 2.3.3 above, failure to account for dependencies in the initial quantification could result in prematurely eliminating important sequences from consideration. In their response to the NRC RAI, the licensee recognizes this possibility and contends that it is unlikely that important sequences were eliminated because: (1) the truncation limit used in the Palisades analysis is lower than commonly used (lE-09 or lE-10 vs. lE-07 or lE-08), and is four orders of magnitude above the estimated CDF, which is on the order of lE-05; and, (2) the HEPs were generally in the conservative range of l.OE-01 to l.OE-03. (Actually, most of the HEPs are in the range of 1.0E-02 to l.OE-03.) However the combination of relatively low .screening values and failure to account for dependencies in the initial quantification could have resulted in elimination of some sequences that warranted further analysis .

• 21

• A different concept related to dependency is the fact that operator actions that are essentially the same but occur in different accident sequences may have significantly different error probabilities. Consequently, it is important to assess each action in the context of the sequence in which it occurs. It is particularly difficult to account for such situation-specific dependencies when human errors are modeled in fault trees, as they were far Palisades. The fault trees are systems related and, unless specifically modified; will be the same whenever a particular system is included in the fault tree model for a given top event ~s issue is not discussed in the submittal or in the licensee's responses to the NRC RAI. Given the overall approach of using "generic" THERP evaluations or screening values, and given that some of the HEPs were modeled in fault trees, it appears that there was limited consideration of sequence-specific differences in HEPs.

2.4 Vulnerabilities, Insights and Enhancements 2.4.1 Vulnerabilities.

The submittal does not present a precise definition of a vulnerability. The licensee cites three geneml questions as "screening criteria" for vulnerabilities:

1. Do the Palisades IPE results meet the NRC's safety goal for core damage?

2. Are the results for core damage sequences or containment performance consistent with other PRAs?

3. Does the probability of sequences characterized as having large releases exceed 10%

of the core damage frequency?

Based on these criteria, the licensee concluded that no vulnerabilities exist. No importance calculamions and limited sensitivity analysis were reported in the original submittal. As discussed above, some sensitivity analyses were performed related to human error estimates.

The licensee does identify and discuss (Section 5 of the submittal) several insights from the IPE, including some in which human performance plays a significant role. No commitments are made for enhancements, though some recent modifications and potential changes are discussed which the licensee believes reduces the significance of some of the contributions to CDF, or at least increases the conservatism of the estimated contribution.

The general vulnerability screening criteria listed above suggest areas, or perhaps sequences, in which specific vulnerabilities may exist. It is not obvious that meeting these criteria establishes that no vulnerabilities exist which deserve attention by the utility and perhaps enhancements to equipment and/or human perfmmance. Criteria 1 and 3 address gross quantitative results of the IPE. Criterion 2, which implies a more detailed comparison of results at a sequence level to other PRAs, at least suggests one rationale for identifying potential specific vulnerabilities. More specific quantitative assessment, perhaps including imponance calculations probably are more u~eful in identifying specific equipment or human

• • 22

• performance issues that may be a vulnerability or may deserve consideration for cost-effective enhancements.

In response to an NRC RAI submitted from the front-end review, the.licensee provided additional information on its definition and concept of vulnerability. The Palisades IPE used as a working definition of vulnerability; any event or condition. that because of its likelihood of occurrence or severity of its consequence could result in significant threat to the health and safety of the public for which prompt action was necessary to preclude its concurrence."

Also considered were the confidence with which the event or condition was known or could be quantified, and the relation of the event to the safety goals. The licensee notes that their concept of vulnerability is substantially different from simply those events that are the highest contributors to risk, particularly when the overall risk is determined to be "acceptable" and in line with risk determined in other studies. The licensee's concept of ._vulnerability also is different from the concept of an "insight," which is an event of condition that is not in line with the anticipated plant response or not generally known to the plant staff. The licensee stated that importance calculations were performed, and rankings of component, or basic element, using Fussel-Vesely importance and Risk Achievement Worth are available, but were not included with the submittal or with the licensee's response to the NRC RAI (except for the Risk Achievement Worth rankings for the important human actions.) Based on this response from the licensee, it appears that the licensee did have a reasonably well defined concept of a vulnerability per NUREG-1335 guidance, and did perform assessments which helped to identify potential vulnerabilities at a general level and at the level of components or basic events.

2.4.2 Insights and Enhancements Identified by the Licensee.

Section 5 of the submittal discusses several insights obtained by the licensee from the IPE results. The "insights" are items that contribute significantly to core damage frequency and/or release of radioactivity, all of which involve human action to some degree. These five

• insights are summarized below:

(1) Small break loss of coolant accident (SBLOCA). The SBLOCA initiating event was a significant contributor to CDF (26.3% ). A major contributor to the event is the high dependency on one system (HPSI) to mitigate the consequences of the accident The licensee notes several recovery actions, including recovery of the HPSI system, use of LPSI for larger size breaks, and makeup to the Safety Injection and Refueling Water Storage Tank (SIWRT), which were not credited in the IPE but which may be available. Based on these observations, the licensee concludes

• that the importance of SBLOCA, and particularly HPSI failure, are modeled conservatively, that no vulnerability exists, and no enhancements are required.

(2) Single Failure ofHPSI with SBWCA. The single failure of the HPSI pump minimum flow recirculation valves to the SIRWT contributes to failure of HPSI prior to recirculation. The dominant contributor to failure of these valves is manual

• 23

verification that the valves are open (restoration after test, maintenance, surveillance). The licensee states that this human error was quantified assuming that required quarterly verification is performed following surveillance tests, but that in actuality, verification is performed during monthly surveillance and during other quarterly tests. The licensee therefore concludes that this human error contribution is conservatively modeled, that it is not considered a :vulnerability, *and that no enhancement is required. We could not identify this pre-initiator error in the list of HEPs provided by the licensee or in the discussion of the HPSI system. Copies of the system diagrams in the submittal were unreadable, and we could not identify the valves in question.

(3) Loss of Offsite Power. The Loss of Offsite Power initiator was identified as a significant contributor (22.8%) to total CDF. The licensee identified several plant modifications which the licensee expects to reduce the assumed initiating frequency, and therefore concludes that no vulnerability exists and no (further) enhancements are required.

(4) Condensate Storage Tank Makeup. Failure to supply makeup to the condensate storage tank (CST) is one of the two major contributors to loss of secondary cooling, which is one of the methods for decay heat removal (the other being bleed-and-feed). The licensee notes that there are backup mechanisms (equipment) available for CST makeup that were not credited in the IPE,. and that therefore, the dominant failure is operator error. The licensee contends that due to training and EOP guidance, this operator failure is minimized, that the human error rate used in the IPE is conservative, and that no enhancement is necessary. This action is XOOOTCST, which was discussed above. The HEP of 2.9E-04 is one of the lowest values of all of the HEPs in the model, and in our view is not obviously conservative. The licensee does not provide any further discussion elaborating on the basis for the belief that the HEP is conservative or on the rationale used to eliminate this action from further consideration for enhancement. The general statement that failure is minimized due to operator training and EOPs does not provide strong justification for ignoring the potential for enhancement. However, we recognize that the licensee did not identify a vulnerability associated with this action, and that there is no requirement for enhancement.

(5) Safety Injection and Refueling Water Tank Makeup. Makeup to the SIRWT is required for LOCAs outside containment, ISLOCAs and SGTRs. The ISLOCA frequency is low, and ISLOCA was determined to be a minor contributor to CDF (0.5%). Steam generator tube ruptures contribute approximately 4.4%. Through these two sequences, failure to provide SIRWT makeup contributes a total of less than 3% to the CDF. The licensee states that methods to reduce the impact of failure of SIWRT makeup have been identified, but the methods are not proceduralized and not assured to work. The submittal states that, "This item will be continually reviewed for ways to identify or quantify additional methods for

• • 24

• 2.4.3 SIRWf or PCS inventory makeup following LOCA outside containment." This is the only "commitment" for enhancement identified in the submittal.

Important Operator Actions.

As indicated previously, the revised submittal provided a listing of risk-significant human errors as determined by the Risk Achievement Wonh (RAW), which is the ratio of CDF given a basic event probability of 1.0 to CDF with the basic event probability at the nominal value. Table 2.3-6 in the revised submittal lists all operator actions with a RAW of 1.02 or greater, which the licensee designates as "risk significant" human errors. An increase of 2%

in CDF would occur if the action were to always fail. For convenience, a copy of this table is ~produced in Section 4 of this TER. In response to the NRC RAI regarding the major insights obtained related to human reliability, the licensee lists the following actions with a RAW >10 (a factor of 10 increase in CDF for failure probability 1.0), which were the top six listed in Table 2.3-6 of the submittal. These are highlighted below in Table 2.4-1. Note that three of these top six actions (and a substantial portion of all of the actions with a RAW greater than 1.02) are pre-initiator actions.

Table 2.4-1 Operator Actions with High Risk Achievement Worth (>10)

• Designator XOOOTCST Operator fails to Description align makeup to CST HEP 2.9E-04 RAW 784

\ -- .~,

SLSOHSIRW SIRW tank level switches miscalibrated high l.lE-04 189 APSOHP8AC Miscalibration of all AFW pump low suet.ion pressure switches l.3E-04 137 AISOTABC Miscalibration of all flow instruments in all AFW headers l.6E-05 115 RPORVOA Operator fails to open PORVs and block valves 8.lE-05 49.9 AOPERLOCOT Operator fails to align ali.emate AFW pump SiUCtion source l.OE-03 23.5 In the licensee's response to the NRC RAI, the licensee also noted that the Fussel-Vesely importance value for XOOOTCST is 0.23, which is interpreted that this human error appears in 23% of the cutsets, and is another indicator of the high importance of this action. The other human actions with RAW> IO all have Fussel-Vesely importance values less than 0.025, with a combined total of less than 0.07 .

• 25

• ID. CONTRACTOR OBSERVATIONS AND CONCLUSIONS The intent of the IPE is summarized in four specific objectives for the licensee identified in Generic Lener 88-20 and NUREG-1335: *

(1) Develop an appreciation of severe accident behavior..

(2) Understand the most likely severe accident sequences that could occur at its plant.

(3) Gain a more quantitative understanding of the overall probability of core damage and radioactive material releases.

(4) If necessary, reduce the overall probability of core damage .and radioactive material release by appropriate modifications to procedures and hardware that would prevent or mitigate severe accidents.

The intent of our document-only review of the licensee's HRA process is to determine whether the process supports the licensee's meeting these specific objectives of GL 88-20 as they relate to human performance issues. That is, whether the BRA process permits the licensee to:

(1) Develop an overall appreciation of human performance in severe accidents; how human actions can impact positively or negatively the course of severe accidents, and what factors influence human performance.

(2) Identify and understand the operator actions important to the most likely accident sequences and the impact of operator action in those sequences; understand how human actions affect or help determine which sequences are important.

(3) Gain a more quantitative understanding of the quantitative impact of human performance on the overall probability of core damage and radioactive material release.

(4) Identify potential vulnerabilities and enhancements, and if necessary/appropriate, implement reasonable human-performance-related enhancements.

It is our general observation from the review of the submittal, the additional material provided by the licensee in response to NRC requests for additional information, and the responses to the subsequent telephone conference call between the NRC and the licensee, that the licensee's HRA is reasonably complete in scope, and did provide the licensee with some insights regarding the relative quantitative impact of human error in severe accidents, but that there are a number of weaknesses in the approach that limit the benefits obtained by the licensee from the analysis. Specific observations most pertinent to NRC's decision regarding whether or not the licensee met the intent of Generic Letter 88-20 are:

• 26

• The submittal and supponing documentation indicates that utility personnel were (1) involved in the HRA, and that plant walkdowns, documentation reviews, and interviews with operations and training staff constituted a viable process for confinning that the HRA portions of the IPE represent the as-built, as-operated planL (2) The licensee performed a peer review, including an internal review and independent assessment by a consultant, that provides some assurance that the HRA techniques have been correctly applied and that documentation is accurate.

(3) Pre-initiator human actions were considered in the analysis. Both miscalibration and restoration errors were addressed. The process for identification and selection of pre-initiator human errors included review of procedures. and interviews with plant personnel. Numerical results in general were consistent with, or (in cases where "screening" values were used) slightly more conservative than, nominal values in other PRAs.

(4) The treatment of pre-initiator and post-initiator errors using the "quasi-generic**

IBERP models limits the degree of in-depth insights about plant-specific factors influencing human performance. Such plant-specific insights are valuable products of the HRA.

{5) The treatment of post-initiator human actions included both response-type and recovery-type actions. The process for identification and selection of actions included review of procedures and discussions with plant personnel.

{6) The licensee used screening values for post-initiator human actions that are significantly lower than typically used for post-initiator actions (0.01 or 0.001 vs.,

say 0.5) and did not include dependencies in the initial quantification. These facts together raise a concern about the potential for screening out risk-significant actions/sequences. Based on telephone discussions between the licensee and NRC, the licensee believes that it is unlikely risk-significant actions/sequences were screened out because (a) sequences with very low frequencies were retained in the model even though they were not reponed, and (b) dependencies were treated in the subsequent model quantification. While the licensee's argument is plausible, the documentation of the licensee's approach an~ justification is very limited. We are not able to confirm whether or not the approach provided reasonable assurance that actions/sequences were not inappropriately screened out.

fl) The treattnent of diagnosis for post-initiator actions is not consistent with most nuclear plant HRA approaches. Most post-initiator human actions in the Palisades model are quantified using THERP. Based on our limited review of several sample

"'generic TIIBRP" calculations, it appears that the licensee did not use the TIIBRP diagnostic model but instead treats all post-initiator response as rule-based responses

• 27

• to alarms/annunciators. We recognize that TIIERP and other HRA techniques employ highly simplified models of the dynamic processes involved with human action in response to an accident condition. And, we recognize that there is a considerable plausibility for an argument that current use of "symptom-based" procedures substantially reduces the cognitive demands on the operators. However, most HRA approaches treat the "cognitive" actions associated with diagnosis, detection, decision-making, etc. distinctly different, often using a ~me-based probability. None of the Palisades documentation reviewed discusses the rationale for excluding diagnosis. In our view, the limited discussion/treatment of the diagnosis portion of the response represents at least a limitation in the documentation of the HRA, and may reflect a weakness in the licensee's understanding of human behavior in severe accidents.

(8) The licensee identified imponant actions and their quantitative impact on core damage and radioactive material release. And, based on the licensee's definition of vulnerabilities, none were identified and no enhancements were "necessary/appropriate." It is our opinion that some of the potential enhancements might have been pursued more rigorously. However, the HRA process did permit the licensee to identify potential vulnerabilities* and enhancements related to human performance.

I

'~

(9) In general, quantitative results for post-initiator actions are consistent with nominal values reponed in other PRAs, and consistent with (or toward the conservative end of) the range typically generated using TIIERP. As indicated above, it is notable, though it may not be significant to the overall results and conclusions of the HRA, that one of the two exceptions in which ASEP was used instead of THERP the calculated HEP is the lowest value of all post-initiator HEPs, and is among the most imponant human actions. Typically, ASEP would be expected to produce higher (more conservative) values.

• 28

• IV. DATA

SUMMARY

SHEETS Important Operator Actions/Errors:

The revised submittal contained a listing of human errors which had a risk achievement worth greater than 1.02. That listing (Table 2.3-6 from the revised submittal) is attached for convenience. The licensee identifies the top six of these actions, which have a risk achievement worth greater* than 10, as "risk significant"

• Human-Performance Related F.nkancements:

No human-performance related enhancements were identified. One area was identified to be under continued review for potential improvement, the operator action to provide makeup to the safety injection and refueling water storage tank.

• 29

• - '!'able 2.3-6 Risk Siqnif icant Human Errors Buman Description RAW Class Event 'l'ree or ~HEP Error Sequence Event XOOO'l'CST Operator fails to 784 Post Sequences 2.9E-4 aliqn makeup to the usinq AFW condensate storaqe (headinq-2NO) tank S~OHSIRW SIRW tank level 189 Pre Sequences l.lE-4 switches miscalibrated - .. using hiqh I containment recirculation (headinqzSIRl APSOHPSAC Miscalibration of all 137 Pl:'e Sequences 1.JE-4 AFW pump low suction usinq AFW

• pressure switches. (heading=2NO)

AISOTABC Miscali.bration of all 115 Pre Sequences l.6E-5 flow instruments in usinq AFW all AFW headers (headinc;z2NO)

• • RPORVOA Operator fails to open PORVs and their associated motor operated isolation valves 49.9 Post Sequences usinq once throuqh coolinq (headinq=POR)

N/A **

AOPERLOCOT Operator tails to 23.5 Post Sequences l.OE-3 aliqn alternate AFW usinq AFW

• pump suction source (heading=2NO)

SXVOM3178 MV-3178 left closed 7.92 Pre Sequences 2.0E-3 after maintenance usinq HPSI (heading=SII)

SXVOM3187 MV-3187 left closed 7.88 Pre Sequences 2.0E-3 after maintenance usinq HPSI (heading=SII)

AISOHAB Miscalibration of all 6.44 Pre. Sequences l.lE-4 flow instruments on. usinq AFW AFW train A/B headers (headinqs2ND)

IXVOM245 Nitrogen station 3.75 Pre Sequences 2.0E-3 manual valve MV-245 using AFW left closed (headincr-=2NDl

• Revision 1, 7/11/1994 2.3-43

Table 2.3-6

~sk Siqnif icant Buman Errors DCVOM244 Nitroqen station 3.75 Pre Sequences 2. OE-J manual valve MV-244 usinq AFW left closed (headincr-2NDl IPSOH1203 Pressure switch PS- 3.72 Pre Sequences l.3E-4 1203 miscalibrated .* usinq :tA

. hiqh (support svstem)

IPSOH1201 Pressure switch PS- 3.72 Pre Sequences l.3E-4 1201 misca.librated usinq IA hiqh (support  :

system)

IXVOMNS'l'A2 Nitroqen station t2 2.46 Pre Sequences l.OE-2 left isolated after usinq AFW maintenance (headincr-2ND)

DBATl'SHED Operator fails to 2.39 Post Transient N/A**

strip non-essential Event Tree loads from station (LOOP events) batteries SISOISIRW SIRW tank level 2.29 Pre Sequences 1.3E-4 switches miscal.il::lrated usinq low containment recirculation (headinci-BIR)

HISOB0442 Level switch LS-0442 2.25 Pre Sequences l.lE-4 miscalibrated usinq BPA (support system)

HLSOH0440 Level switch LS-0440 2.01 Pre Sequences l.lE-4 mis calibrated usinq HPA (support svstem)

GAVOADOMP Operator fails to open 1.77 Post Sequences 3.4E-2 the ADVs or the TBV usinq low pressure feed (headinqa2ND)

• IXVOACA677 Operator fails to open 1.53 Post Sequences 2.0E-3 valve to bypass air usinq IA dryer (support svsteml Revision l, 7/11/1994 2.3-44

• GXV081234 t1PMOEPtJMP Operator fails to close AJJV isolation block valve Operator fails to 1.s2 1.so Post Post SGTR or Transient Event Trees Sequences 2.0E-3 3.4E-2 manually start service usinq SWS water pump {support system)

SXVOL3225 MV-3225 left open 1.44: Pre Sequences 2.0E-3 after maintenance usinq containment spray (headinq:sCHR)

AOPTRIP4 Operator tails to trip 1.35 Post Sequences 1.0E-2 all primary coolant usinq AFW pumps for P-SC (beadinq-2HD) operation ICSOB2A2B Operator tails to l.29 Post Sequences l.OE-2 manually start usinq IA instrument air (support compressor system)

AAVOBAFW-A Operator fails to 1.24 Post SG'l'R Event 3.4E-2 isolate AFW flow to E- 'l'ree

• QHSOLC7137 SOA (SG'l'R)

Control on Panel C-137 not in auto 1.14 Pre Sequences usinq fire protection (support system) 2.0E-3 AOPADJFL Operator fails to 1.12 Post Sequences 1.7E-3 adjust AFW flow qiven usinq AFW failure of one header (headinqs2NDl GXVOM101 Manual valve.MSlOl 1.10 Pre Sequences 2.0E-3 left closed usinq AFW or LPF (headincr-=2HD)

GXVOM103 Manual valve MS103 1.10 Pre Sequences 2.0E-3 left closed usinq AFW or LPF (headinc:r=2HDl IXVOM0781 Manual valve 0781 left 1.10 Pre Sequences 2".0E-3

,\ closed usinq AFW or  !

LPF '

(headinc:=2NDl

• 2.3-45

• IXVOM0782 Manua1 valve 0782 left closed 1.10 Pre Sequences using AFW or LPF (headinc:r=2ND) 2.0E-3 SOOOTSDCS Operator fails to 1.08 Post Sequences 1.0E-2 align shutdown cooling using shutdowm

. cooling (headincr-CHR)

SPSOH0104 PS-0104 pressure 1.08 Pre Sequences l.3E-4 switch miscalibratecl using high shutdown

• cooling (headincr-CBR)

POPREOt1ClD Operator fails to 1.07 Post Transient l.OE-1 reduce SW loads on bus Event Tree 10 failure (loss of bus.

10)

AOPONISA Operator fails to 1.06 Post Transient l.OE-2 unisolate flow to Event Tree**

isolated SG (MSLB) or SGTR Event Tree AOPC?aSB Operator fails to l.06 Post Transient l.OE-2 unisolate flovtc Event Tree isolated SG (MSLB) or SGTR Event Tree GXVOM102 Manual valve MS102 1.03 Pre Sequences 2.0E-3 left closed us~ng AFW or LPF (heading=2ND)

GXVOM104 Manual valve MS104 1.03 Pre Sequences 2.0E-3 left closed using AFW or LPF (heading=2ND)

IXVOM0779 Manual valve 0779 left 1.03 Pre Sequences 2.0E-3 closed using AFW or .

LPF (heading=2ND)

IXVOM0780 Manual valve 0780 left 1.03 Pre Sequences 2.0E-3 closed using AFW or

- LPF (headincr-2NDl

.;. Revision 1, 7/11/1994 2.3-46

• QXVOAOl2 Operator fails to align FPS to compressor 1.03 Post Sequences usinq IA (support system) 3.4E-2 FOOOTLPF Operator fails to 1.02 Post Sequences l.OE-3 supply condensate to usinq low depressurized SG pressure feed

.* (headinc:i-2ND)

SRECIRCOX Operator fails to 1.02 Post Sequences l.OE-2 align sump suction usinq containment .:.~

recirculation (heading.SIR)

• • 'l'his human error was identified after final quantification. The RAW was calculated usi~q a sensitivity analysis with the ASEP value developed duri.nq the independent review of the BRA.

NOTE: Scme human error events have been added and RAW values chanqed since the oriqinal submittal. 'l'wo factors are the major contributors to this: 1)

RAW values were recalculated to include RPORVOA and DBA'l"l'SHED: and 2) some RAW values were incorrectly translated (off by an order of magnitude) or incorrectly calculated. All caleulations are now done by computer code *

.t, Revision 1, 7/11/1994 2.3-47

• REFERENCES

1. Swain, A.D., and H.E. Guttmann, "Handbook of Human Reliability Analysis Procedures, with EmphaSis on Nuclear Power Plant Applications (Final Report)," NUREG/CR-1278-F, August, 1983.

2. Swain, AD..,. "Accident Sequence Evaluation Program Human Reliability Analysis Procedure," NUREG/CR-4772, February, 1987.

3. Embrey, D..E.. et al., "SLIM-MAUD: An Approach to Assessing Human Error Probabilities; Using Structured Expert Judgment, Vol. I: Overview of SLIM-MAUD,"

NUREG/CR...,3518, March, 1984. .

4. EPRI TR-100259, "An Approach to the Analysis of Operator Actions in Probabilistic Risk Assessment,,... Electric Power Research Institute, June, 1992.

• 35

APPENDIX C PALISADES NUCLEAR PLANT INDIVIDUAL PLANT EVALUATION TECHNICAL EVALUATION REPORT (BACK-END ANALYSIS)

• Enclosure 4