ML20138K062

TER on IPE Submittal Human Reliability Analysis, Final Rept

ML20138K062
Person / Time
Site:	Arkansas Nuclear
Issue date:	11/30/1995
From:	Haas P CONCORD ASSOCIATES, INC.
To:	NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES)
Shared Package
ML20138F088	List: ML20138F081 ML20138F092 ML20138G522 ML20138K043 ML20138K056 ML20138K062
References
CON-NRC-04-91-069, CON-NRC-4-91-69 CA-TR-94-019-37, CA-TR-94-19-37, NUDOCS 9705120107
Download: ML20138K062 (46)

Text

i l

CA/TR 94-019-37 ARKANSAS NUCLEAR ONE UNIT 2 TECHNICAL EVALUATION REPORT ON THE l IPE SUBMITTAL i HUMAN RELIABILITY ANALYSIS f i

FINAL REPORT l J

i By:  ;

P. M. Haa3 l l

1 Prepared for:

U.S. Nuclear Regulatory Commission .

I Office of Nuclear Regulatory Research  ;

Division of Systems Technology i

I l

Draft Report December,1994 Final Report November,1995 .

CONCORD ASSOCIATES. INC.

Systems Performance Engineers

- 725 Pellissippi Parkway Knoxville, TN 37932 Contract No. NRC-04-91-069 Task Order No. 37 l

ENCLOSURE 4B 9705120107 970505 PDR ADOCK 05000313  !

P PDR

TABLE OF CONTENTS E. EXECUTIVE

SUMMARY

. ..... . . ... .. ................. 1 E.1 Plant Characterization . .......... ........................ 1 E.2 Licensee IPE Process . . ... ... .......................... 1 E.3 Human Reliability Analysis . .. .... ....................... 2 E.3.1 Pre-Initiator Human Events . . . ....................... 2 E.3.2 Post-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 E.4 Generic Issues and CPI ................................... 4 E.5 Vulnerabilities and Plant Improvements . . . . . . . . . . . . . . . . . . . . . . . . 4 E.6 Observations . . . . . . . . ................................ . 5

1. INTRODUCTION . .......... ........ ........... ............. 7 1.1 HRA Review Process . .......... ......... .. ........... 7 1.2 Plant Characterization . . . . . . . . . . . . . . ...... ............... 7

2. TECHNICAL REVIEW . ........... .................. ...... . 9 2.1 Licensee IPE Process ... . ....... .......... ... ....... 9 2.1.1 Completeness and Methodology . . . . . . . . . . . . . ........... 9 2.1.2 Multi-Unit Effects and As-Built, As-Operated Status . . . . . . . . . . 9 2.1.3 Licensee Participation and Peer Review . . . ......... . . . . . 10 2.1.3.1 Licensee Participation . . . . . . . ................. 10 2.1.3.2 Peer Review . . . . . . . . . . . . . . . . . . ............. 11 2.2 Pre-Initiator Human Actions . . . . . . . ... ........... ..... . 12 2.2.1 Pre-Initiator Human Actions Considered . . . . ........ . . . . 12 2.2.2 Process for Identification and Selection of Pre-Initiator Human Actions . . . . . . ............... ... .......... . 13 2.2.3 Screening Process for Pre-Initiator Human Actions . . . . . . . . . . . 13 14 2.2.4 Quantification of Pre-Initiator Human Actions . . . . . . . . . . . . . .

2.3 Post-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . ........ 16 2.3.1 Types of Post-Initiator Human Actions Considered . . . . . . . . . . . 16 2.3.2 Process for Identification and Selection of Post Initiator Human Ac tio ns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........... 18 2.3.3 Screening Process for Post-Initiator Response Actions . . . . . . . . . 19 20 2.3.4 Quantification of Post-Initiator Human Actions . . . . . . . . . . . .

2.3.4.1" Time Independent Quantification Technique . . . . . . . . . 20 2.3.4.2 Time-Dependent Quantification Technique . . . . . . . . . . . 21 2.3.4.3 Basis for Time Estimates . . . . . . . . . . . . . . ...... . 24 2.3.4.4 Treatment of Dependencies in Post-Initiator Actions . . . . 26 2.3.4.5 Equipment Failures Associated With Recovery Actions . . . 27 2.3.4.6 Quantification of Human Actions in the Flooding Analysis 28 2.3.4.7 Human Actions.in the Level 2 Analysis . . . . . . . . . . 29 Table of Contents (Continued) 1

(

Table of Contents (Continued) i 2.3.5 GSI/USI and CPI Recommendations . . . . . . . . . . . . . . . . . . . . . 29 ,

2.4 Vulnerabilities, Insights and Enhancements . . . . . . . . . . . . . . . . . . . . . . 30 2.4.1 Vulnerabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.4.2 Insights Related to Human Performan~ce . . . . . . . . . . . . . . . . . . . 30 2.4.2.1 Important Operator Actions . . . . . . . . . . . . . . . . . . . . . 30 2.3.4.2 Sensitivity Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.3.4.3 Credit for Operator Recovery Actions . . . . . . . . . . . . . . 34 2.4.3 Human-Performance-Related Enhancements . . . . . . . . . . . . . . . . 35

3. CONTRACTOR OBSERVATIONS AND CONCLUSIONS . . . . . . . . . . . . . . . . . . 37 ,

4. D ATA

SUMMARY

S HEETS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 REF5RENCES ..................................... ............ 42 i

?

l t

i l

L i

l

.D 2

e b

t

E. EXECUTIVE

SUMMARY

This Technical Evaluation Report (TER) is a summary of the documentation-only review of the human reliability analysis (HRA) presented as part of the Arkansas Nuclear One Unit 2 (ANO-2) Individual Plant Examination (IPE) submitted by Entergy Operations, Inc. (Entergy) to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to assist NRC staff in their evaluation of the IPE and conclusions regarding whether the submittal meets the intent of Generic Letter 88-20.

E.1 Plant Characterization The ANO-2 plant is a Combustion Engineering (CE) pressurized water reactor (PWR) with two heat transfer loops, each loop containing a steam generator and two reactor coolant pumps. Design features noted by the front-end reviewer as having a significant impact on the estimated core damage frequency (CDF) and which significant from a human performance perspective include: a) the ability to perform feed-and-bleed (in the absence of power-operated relief valves); b) automatic switchover of ECCS from injection to recirculation; and, c) service water backup to EFW pumps. In general, operator actions were found to be relatively important to plant risk in the ANO-2 analysis. Six of the ten most important basic events were operator actions, in particular, failure to align offsite power, failure to trip reactor coolant pumps (RCPs) after a loss of component cooling water, and failure to realign DC buses to the swing battery charger.

E.2 Licensee IPE Process Licensee personnel were involved in the IPE/HRA process. The HRA methodology and guidance for its use was provided by a contractor (SAIC), but the analysis was performed by ANO-2 staff. The licensee's process included review of procedures and other plant documentation, discussion with plant personnel, and plant walkdowns to help assure that the ,

IPE/HRA represents the as-built, as-operated plant. A reasonably comprehensive in house review process was in place to help provide assurance that models and data were properly l implemented. Multi-unit effects, in panicular human action to align cross ties to ANO-1, were addressed in the HRA. The HRA approach employed by the licensee addressed both .

i pre-initiator human actions (actions during maintenance, test, etc.) that could cause failure of important equipment on demand during an accident, and post-initiator human actions (those taken in response to an accident event). The licensee identified important human actions based on risk increase, risk reduction, and uncertainty measures. Several human-performance related enhancements were identified and credited in the IPE.

f  ;

l

E.3 Human Reliability Analysis E.3.1 Pre Initiator Human Events.

The ANO-2 HRA addressed pre-initiator errors in maintenance, test and surveillance actions by incogor .ng human error into the systems analysis (fault trees) as a specific cause for system une <ailability. Both restoration errors and calibration errors were addressed. The licensee's process for identification and selection of pre-initiator human actions was performed as an integral part of the development of fault trees. The process included review of procedures and the human interface, and plant personnel were involved in the process. These findings and the fact that a significant number of pre-initiator human actions included in the model indicate that a reasonable process was employed and that important actions to assure that appropriate pre-initiator actions were included in the model. A numerical screening value of 0.003 was used by the licensee for pre initiator errors in the initial quantification. This HEP value is consistent with nominal values (not screening values) for errors of omission / commission in THERP. An HRA screening cutoff of 1.0E-08/yr was used to screen out unimportant cutsets. This is an " acceptable", but not particularly low, truncation value.

In our view, more conservative (higher) values are appropriate for screening. However, it is not possible to determine from this document-only review whether or not the use of this relatively low screening value for pre-initiators had a significant impact on the quantitative results.

The quantification process used for the pre-initiator human actions was a simplified technique developed by the licensee or the licensee's contractor which was based on concepts from THERP/ASEP. The basic HEP value was multiplied by several possible factors to account for dependencies in actions performed on multiple equipment trains / channels and to take credit for potential error recovery by other crew members. Plant-specific assessment was very limited in scope and depth. In general, we find the licensee's approach to be relatively crude and mechanistic. Use of a " generic" screening approach limited the ability of the licensee to cbtain insights about the potential impact of this type of human error on plant risk and certainly about the underlying factors influencing human error. Those insights could lead to simple, cost-effective ways to improve risk. However, the approach in general is not inconsistent with treatment of pre initiator human errors in other PRAs that have been accepted.

E.3.2 Post-Initiator Human Actions.

The pst-initiator HRA was ahpropriately broad in scope in that it addressed both respons type and recovery type actions (both in the control room and out of the control room). The licensee used different nomenclature than most PRAs to classify post-initiator actions. The licensee's use of the term " recovery action" or " recoveries" was broader than typically used and included actions typically treated as response-type actions in other PRAs. Recovery actions could include both proceduralized and non proceduralized actions (though in the ANO-2 analysis non proceduralized actions were screened out). Recovery actions in the 1

2 1 l

control room were quantified using a different "model" (actually different parameters for the same model) from the model used for actions outside the control room. The process employed by the licensee to identify and select the post initiator actions to be quantified ,

included review of procedures and discussion with operations / training staff. A numerical screening values of 0.4 and a cutset cutoff value of 1.0E-09/yr were employed to eliminate unimportant actions /cutsets from detailed analysis. In the final quantification, all screening values were replaced with nominal values determined from one of the HRA quantification 4

techniques. This numerical screening approach appears to have been a reasonable approach to eliminate unimportant sequences from consideration without eliminating important actions / sequences.

Post-initiator response type actions were quantified using the SAIC system of time reliability correlations (TRCs). These correlations are based on published simulator data. As is the case with all TRCs, th: primary determinant of the HEP value is the relative time for operator action, i.e., tiro required relative to the time available. In the ANO-2 HRA, estimates of available time were made from thermal hydraulic calculations, simulator data, and other sources. Estimates of time required were based primarily on operator judgment. The interview process for obtaining time estimates appears to have been reasonably well structured, and the time estimates in general appear to be reasonable based on comparison with other PRAs.

The SAIC TRCs have several additional parameters that provide an analyst with a convenient mathematical means to apply subjective judgment to alter the HEP predicted by the base correlation. These parameters can serve as an aid to guide the analyst's judgment and assessment of plant-specific factors influencing each operator action. In general, the ANO-2 analysis appears to have properly and effectively used the SAIC approach to obtain HEP estimates in a reasonably systematic, though still fundamentally subjective, manner. The

! licensee's discussions and examples indicate a reasonably plant-specific assessment of performance shaping factors was made as a part of the analysis.

Some of the response-type actions were modeled in fault trees. Usually, modeling of human actions in fault trees is less desirable than modeling in event trees or specific cutsets, because it is difficult to account for sequence specific influences on the probability of error. The licensee states that such sequence-specific influences were accounted for in the ANO-2 analysis by treating each action as a unique basic event in the fault tree models. Thus the "same" human action was sometimes modeled as different actions with different HEPs in two different sequences. This is reasonable approach to treating these dependencies, given that it was fully implemented for all cases in which the difference in HEPs was significant.

Dependencies among multiple human actions appearing in the same cutset were not treated directly in the ANO-2 model, but me licensee's judgment based on review of cutsets was that such dependencies were negligible. This judgment was not supported by detailed analysis or discussion. While it is not possible for us to determine from this review whether or not the impact on the IPE results was significant, the lack of direct consideration of such dependencies must be considered a weaknesa.

3 i

i l

l I

Another weakness in the post-initiator analysis is the use of an unrealistic model for crediting error recovery by other crew members. The model could lead to highly optimistic estimates of overall human error. Since very few HEPs were affected in the ANO-2 analysis, the impact on the IPE results probably is not significant. However, use of the model in our view indicates a weakness in modeling human performance.

Assessment of human action in the flooding analysis was li'mited to credit (HEP =0.01) for operator action to identify and isolate the source of large floods within 20 minutes. The HEP value of 0.01 is an " arbitrary" screening value that has been used in other PRAs.

Human action was not treated explicitly in the Level 2 analysis.

E.4 Generic Issues and CPI The front-end reviewers discussed the licensee's treatment of decay heat removal (DHR), USI A-45. The licensee employed a classification scheme based on mean CDF due to DHR failure. The licensee concludes that no unique DHR vulnerability exists for ANO-2. The licensee considered diverse means of decay heat removal, including: use of the power conversion system (PCS), feed-and-bleed cooling, auxiliary feedwater, and emergency core cooling system. The licensee also addressed GSI-23 " Reactor Coolant Pump Seal Failures" and GSI-105 " Interfacing Systems LOCA at LWRs" in the IPE. Both types of events were determined by the licensee to have small contributions to CDF for ANO-2.

The licensee addressed recommendations of the Containment Performance Improvement (CPI)

Program for PWR large dry containments that relate to effects of global hydrogen combustion on containment and containment equipment. The Level 2 reviewer concluded that the licensee's analyses was responsive to the CPI recommendation for PWR large, dry containments regarding these vulnerabilities.

E.5 Vulnerabilities and Plant Improvements The licensee defines a vulnerability as a sequence group with a mean core damage frequency (CDF) greater than IE-04/yr or a containment event tree (CET) endstate group with a mean CDF with containment failure / bypass greater than IE-05/yr. This definition is based on, and partially adopts, NUMARC 91-04 guidance. Specific exceptions to the NUMARC guidance were identified by the licensee. No vulnerabilities were identified per this definition.

Human-performance-related plant improvements were identified by the licensee and credited in the IPE, including:

(1) Loss of Service Water (SW) Procedure - Potential improvements to the loss of SW ,

procedure were identified which would secure one train of CS and LPSI eliminating unnecessary use and overheating and providing a " backup" if one train fails.

4 l

n

(

(2) Shutdown Cooline Svstem Procedure - An additional verification check is proposed to be added to the shutdown cooling system procedure to verify suction line isolation valves are closed once the system has been secured after startup. This check is intended to reduce the potential for an interfacing system loss of coolant accident (ISLOCA) through this path.

(3) Station Blackout Procedure - A change to the SBO procedure is proposed to close a manual valve which would eliminate one potential containment leak path.

(4) Emereency Feedwater Flow Control - Procedure changes implemented are intended to increase assurance that the EFW discharge valves are manually controlled prior to a loss of DC power, which improves response to extended loss of power events.

(5) Deeraded Power Procedure - An improvement to the degraded power procedure is proposed to manually close containment vent header valves or associated valves in series to help assure containment isolation.

E.6 Obsen ations The following observations are pertinent to NRC staff's determination of whether the licensee's submittal meets the intent of Generic Letter 88-20:

1) The submittal and supporting documentation indicates that utility personnel were involved in the HRA, and that the walkdowns and documentation reviews constituted a viable process for confirming that the HRA portions of the IPE represent the as-built, as-operated plant.

1

2) The licensee performed an in-house peer review that provides some assurance that the HRA techniques have been correctly applied and that documentation is accurate.

3) The HRA was appropriate in scope, and a reasonable process was in place to assure that important actions were identified and included in the quantification. in the model.

4) The licensee identified operator actions important to risk using importance calculation (Fussel-Vesely) and sensitivity studies. Human actions appear to be generally of somewhat greater importance in the ANO-2 analysis than " typical" for PWR studies.

Six of ten most important basic events are human actions, and the overall contribution of credit for operator action is responsible for a reduction of approximately three orders of magnitude in CDF (compared to what the CDF would be if no credit were taken for operator action).

5) Overall, the licensee's approach to quantification of post-initiator response actions l l l properly implemented the HRA techniques selected (the SAIC TRCs), though the level of detail of plant-specific assessment is limited, especially in the pre initiator analysis.

5 i

t l

6) The licensee employed a systematic process to screen for vulnerabilities and identify potential enhancements. The process identified a number of human performance-related (prucedure) enhancements expected to reduce the likelihood of human error, and consequently reduce the estimated CDF.

7) Weaknesses identified in the HRA process included the following:

The quantitative screening process for pre-initiator actions used a non-conservative value, which could have eliminated some important sequences. The licensee assessed this possibility and concluded that their analysis was appropriate and that no significant were likely to have been missed.

1

- The quantification process used for the pre-initiator HRA was essentially a l

" generic" assessment with relatively limited plant-specific evaluation of factors )

influencing human performance. This generic approach limits the opportunity for 1 the licensee to identify and understand factors influencing human performance m I pre-initiator events. l

- Quantification of post-initiator " slips" using the SAIC time-independent technique I employed what we believe to be non-conservative assumptions regarding credit for  !

I error recovery by other crew members. While we do not have details sufficient to evaluate the impact on overall results, the model was used for very few actions, and therefore its use is unlikely to have had a significant quantitative impact. ,

- Dependencies among multiple actions (HFEs) modeled in fault trees was not accounted for. The licensee presented qualitative arguments that the effect of such (

dependencies was negligible, but did not present supporting analysis to justify this assertion.

l l

I 6

i

j 1

i l 1. INTRODUCTION j 1 This Technical Evaluation Report (TER) is a summary of the documentation-only review of  !

i the human reliability analysis (HRA) presented as part of the Arkansas Nuclear One Unit 2 l (ANO-2) Individual Plant Examination (IPE) submitted by Entergy Operations, Inc. (Entergy)  !

to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to assist NRC staff in their evaluation of the IPE and conclusions regarding whether the submittal  !

l meets the intent of Generic Letter 88-20.  ;

3 )

i 1.1 HRA Review Process -

, i The HRA review was a " document-only" process which consisted of essentially four steps:

(1) Comprehensive review of the IPE submittal focusing on all information pertinent to HRA.

1 i (2) Preparation of a draft TER summarizing preliminary findings and conclusions, noting j l specific issues for which additional information was required from the licensee, and formulating requests to the licensee for the necessary additional information. )

l

! (3) Review of prelimhary findings, conclusions and proposed requests for additional j information (RAls) with NRC staff and with " front-end" and "back-end" reviewers e

l (4) Review of licensee responses to the NRC requests for additional information, and

preparation of this final TER modifying the draft to incorporate results of the i additional information provided by the licensee and finalize conclusions.

l l Findings and conclusions are limited to those that could be supported by the document-only review. No visit to the site was conducted. In general it was not possible, and it was not the intent of the review, to reproduce results or verify in detail the licensee's HRA quantification j j process. The review addressed the reasonableness of the overall approach with regard to its 1 ability to permit the licensee to meet the goals of Generic Letter 88-20.

l 1.2 Plant Characterization The ANO-2 plant is a Combustion Engineering (CE) pressurized water reactor (PWR) with i two heat transfer loops, each' loop containing a steam generator and two reactor coolant 5

pumps. The power rating is 2,815 MWt and 912 MWe Commercial operation began March, 1980. The reactor and nuclear steam supply systems are similar to San Onofre Units 1 and 2.

l Design features noted by the front-end reviewer as having a significant impact on the i estimated core damage frequency (CDF) and which significant from a human performance

perspective include:

i A

7 i

e

Ability to nerform feed and bleed. The CE design employed in ANO 2 does not include power-operated relief valves. However, once-through feed-and-bleed cooling ,

can be accomplished using either Emergency Core Cooli'g System (ECCS) vent valves or Low Temperature Overpressure Protection (LTOP) valves. Operator action l is required to align for feed-and-bleed operation. (This operator action is significant in the ANO-2 analysis, but does not appear among the.most important operator actions per the Fussel-Vesely importance calculations.)

l Automatic switchover of ECCS from iniection to recirculation. Typically, operator action to align for the switchover is a notable contributor to plant risk in PWR designs I for which manual switchover is required. The automatic switchover tends to decrease CDF.

. Service water backun to EFW oumes. The service water system provides and additional means of supplying water to the emergency feedwater pumps. Operator action is required to align the service water system.

In general, operator actions were found to be relatively important to plant risk in the ANO-2 analysis. Six of the ten most important basic events were operator actions, in particular, failure to align offsite power, failure to trip reactor coolant pumps (RCPs) after a loss of component cooling water, and failure to realign DC buses to the swing battery charger. A  !

sensitivity study performed by the licensee indicated that credit for operator recovery actions reduced the total CDF from an estimated 1.5E 02/yr to 3.4E-05/yr.

8

2. TECHNICAL REVIEW 2.1 Licensee IPE Process 2.1.1 Comoteteness and Methodologv.

The scope of the licensee's HRA analysis was generally complete, though the analysis was limited in depth in some areas, in particular the pre-initiator HRA. The analysis was performed by ANO personnel using methodology and guidance provided by a subcontractor, i Science Applications International Corporation (SAIC). The HRA addressed both pre initiator  !

human actions (actions during maintenance, test, etc.) that could cause failure of important ,

equipment on demand during an accident, and post-initiator human actions (those taken in l response to an accident event). Pre-initiator human actions were quantified using a simplified i approach developed by the licensee or SAIC that employs some of the concepts and basic l human error probability values from ASEP (Ref.1) and THERP (Ref. 2). Post-initiator  !

human actions were quantified using a time reliability correlation (TRC) approach developed l by SAIC. Both response-type actions (anticipated actions in response to an accident event such as those designated in emergency operating procedures), and recovery-type actions (those 4

involving alternative responses or recovery of failed equipment) were addressed. The post-  !

initiator quantification included consideration of several performance shaping factors.

Sequence-specific influences were considered, but dependencies among multiple operator actions in a cutset were not addressed for the human actions included in fault trees (the  !

HFEs).

2.1.2 Multi-Unit Effects and As-Built. As-Ocerated Status Arkansas Nuclear One Unit 1 (ANO-1), a B&W PWR, is co-located on the ANO-2 plant site.

Shared facilities or equipment identified by the front-end reviewer include Startup i

Transformer 2, the control room, the emergency cooling pond, and a number of non safety-related structures and facilities. Operator action to align emergency feedwater suction to the alternate ANO-1 source is one of the ten most important human actions. l l

The summary of documentation development in Section 2.4.3 of the submittal includes normal operating procedures, emergency and abnormal operating procedures, and system training manuals as sources of information on current plant operations and expected operator response to accident conditions. The di,scussion does not specifically identify maintenance procedures j as part of the documentation, though technical specifications were identified as providing information on operating limits and surveillance frequencies. The submittal (page 2.4-1) notes i that the use of controlled drawings that are updated after plant modifications helped to assure i that the most recent and accurate information on plant configuration was incorporated into the IPE model. However a " cutoff date" of August 31,1988 was used for plant changes. This j cutoff date is four years before the IPE submittal date of August 28,1992. The submittal I 1

notes that several other programs underway at ANO helped provide assurance that the design and licensing basis documentation (and therefore the IPE) are accurate reflections of the actual 9

i t

l plant configuration. These efforts include: 1) the design configuration documentation l program; 2) the isometric drawing update program; 3) the electrical drawing update program; and,4) the system training manual upgrade program.

The submittal identifies several types of plant walkdowns and interviews conducted to support !

the Level 1 and Level 2 PRA analysis, including:

System level walkdowns by systems analysts, performed as needed j

• Walkdowns and interviews to assess recovery actions  !

- Walkdowns for the flooding analysis

- Containment walkdowns during a refueling outage in support of the Level 2 analysis.

The walkdowns related to the recovery actions were performed for several recovery actions which occur outside the control room. Specific actions walked down were not identified. However, the submittal states that the PRA teams confirmed or modified the actions based on interviews with plant operators. The interviews consisted of characterizations to the operators of the sequence of failures of concern and the operators identifying annunciators, actions and procedures used to respond to the accident conditions. Maintenance personnel also were interviewed to gain an understanding of component restoration, testing and maintenance timing (pertinent to assessing pre-initiator human errors and component unavailability due to test and maintenance activities).

2.1.3 Licensee Particination and Peer Review.

2.1.3.1 Licensee Particioation. As part of the IPE planning and development process, the licensee formed a PRA group in the Nuclear Engineering Design Department. The IPE l was performed by this group, supported by contractors - Science Applications International Corporation (SAIC), and ERIN Engineering and Research, Inc. The submittal states that I ANO personnel were involved in all aspects of the PRA model development, l quantification, refinement and interpretation. At the beginning of the project, SAIC provided planning and direction for the work and were assigned as Task Leaders with the ANO PRA Project Team members as assistants to facilitate technology transfer. The ,

submittal indicates that by the midpoint of the project, ANO staff were performing the  ;

majority of the work, and by the end of the effort, ANO staff had assumed full  !

responsibility for the PRA development at ANO, with the contractors providing only supplemental suppon and review. The submittal (page 1.4-1) states that over 50% of the total engineering effort applied to the project was contributed by Entergy personnel.

The PRA group, located on site at ANO, includes two safety analysis supervisors and ten engineers. The submittal provides brief biosketches of the individuals involved in the IPE.

The HRA was led by an ANO Senior Staff Engineer with 19 years experience at ANO.

The submittal states that the HRA was performed by ANO personnel with participation by individuals from operations, training, and engineering. Details are not provided on the 10 f

involvement of these plant individuals in the IPE development, but the submittal contains several general references to interviews with plant operators and maintenance personnel in support of the HRA. SAIC staff provided review of the ANO staffimplementation of SAIC HRA techniques, in particular the analysis of recovery actions.

2.1.3.2 Peer Review. The submittal (Section 5.2) describes an independent review process conducted by the licensee and subcontractor staff which helped to provide assurance inat the IPE analytic techniques were correctly applied and documentation is accurate. independent technical review by ANO and SAIC staff was conducted iteratively as the ind vidual task products were developed. These reviews included: (1) technical review of individual system work packages, (2) a Level I cutset quantification review, (3) a Level I human recovery action cutset review, and (4) a Level 2 input and quantification review. The submittal states that informal cutset reviews were performed by interested Operations and System Engineering staff on multiple occasions.

In addition to these internal and informal reviews, an Independent Review Team was formed that consisted of personnel from ANO Operations, Design Engineering, Training, and Licensing supplemented by staff from ERIN with expertise in PRA methodology.

The review was coordinated by ERIN. Two training sessions in the IPE process and PRA techniques were conducted for ANO members by ERIN. The review process included seven review meetings in which one or more technical areas were summarized. ERIN PRA experts had reviewed the analysis prior to the meeting and presented their findings and results. ANO team members then reviewed material in their respective areas and provided comments to the PRA Project team. The review process, therefore, included review by personnel with PRA expertise and with plant operations / engineering experience.

The major areas of review were:

• Overall PRA methodology

- Initiating Events and Accident Sequences

- Systems Models

- Data Analysis

- Human Reliability and Recovery Analysis Model Quantification

• Containment Analysis and Release Characterization The submittal (Section 5.3) provides a brief summary of findings and results in each of these general areas. The HRA andRecovery Analysis review involved both a general methodology review and detailed review of quantification of sample human error probabilities (HEPs). The general review addressed methods for identification of human actions, development of factors which influence the likelihood of successfully completing the actions, and techniques for quantifying HEPs. The detailed review of HEP quantification was performed for a representative set of human actions, including in-control room, ex-control room, pre-initiator, post-initiator and recovery actions. The submittal (page 5-9) states that the review concluded that, "The HRA analysis methodology is adequate for producing all HEPs necessary for

~

11

quantification of the ANO IPE studies," and that, "For the most part, quantification of human actions was performed consistent with the methodology." Minor discrepancies were noted with subsequent correction by the ANO PRA group. Detailed review comments are not provided in the submittal.

2.2 Pre-Initiator Human Actions j Errors in performance of pre initiator human actions, such as failure to restore or properly i align equipment after testing or maintenance or calibration of system logic instrumentation, '

may cause components, trains, or entire systems to be unavailable on demand during an accident, and thus may significantly impact plant risk. Our review of the HRA portion of the IPE examines the licensee's HRA process to determine what consideration was given to  !

pre-initiator human events, how potential events were identified, the effectiveness of quantitative and/or qualitative screening process (es) employed, and the processes for  !

accounting for plant-specific performance shaping factors, recovery factors, and dependencies among multiple actions.

l 2.2.1 Pre-Initiator Human Actions Considered.

The ANO-2 HRA addressed pre-initiator errors in maintenance, test and surveillance actions by incorporating human error into the systems analysis (fault trees) as a specific cause for unavailability of equipment on demand (equipment not normally operating). The submittal indicates that the ANO-2 IPE addressed pre initiator " slips," i.e., unintended actions made during test and maintenance or normal operations. It also states that pre-initiator " mistakes" l were nqt addressed. Mistakes are errors in higher level cognitive action, such as decision making. Mistakes in pre-initiator actions are not treated, according to the submittal (page 3.4-1), "because the decision making of maintenance like actions is procedurally controlled, is assumed to be reliable and has traditionally been considered out of the scope of PRA." This classification of human errors according to causal mechanisms of slips and mistakes is not a part of all HRA techniques. THERP (Ref. 2), for example, classifies errors as " commission" l

' I or " omission." The licensee's response to an NRC RAI identified seventeen specific human errors included in the final model (i.e., included in cutsets that were not truncated), and an additional fifteen that were included in the model initially, but appeared in cutsets that were truncated. Both restoration and miscalibration errors are included in both lists.

2.2.2 Process for Identification and Selection of Pre-Initiator Human Actions.  !

~ \

The key concerns of the NRC staff review regarding the process for identification and  !

l selection of pre-initiator human events are: (a) whether maintenance, test and calibration procedures for the systems and components modeled were reviewed by the systems analyst (s),

and (b) whether discussions were held with appropriate plant personnel (e.g., maintenance, training, operations) on the interpretation and implementation of the plant's test, maintenance and calibration procedures to identify and understand the specific actions and the specific components manipulated when performing the maintenance, test, or calibration tasks.

12 l

I

The licensee states (in a response to an NRC RAI) that pre-initiator human actions were identified and selected, "During the development of the system fault trees by reviewing the various failure modes of the systems and accounting for human induced failures. ... The rationale t;ed to develop these events was to ekamine the human interface to equipment and evaluate the potential failure modes and incorporate these into the system fault tree.

Examples of references used in developing the system fault trees include operating procedures, calibration procedures, and surveillance procedures." Specific criteria for including or eliminating a particular action, were not provided by the licensee. The submittal includes general stater 9ents that normal operations procedures (not specifically maintenance procedures) were reviewed and that interviews were conducted with both operations and maintenance personnel.

The summary information provided by the licensee in this document-only review is not sufficient to determine the level of rigor employed in identifying and selecting pre-initiator actions to include in the model. However, the licensee's statements indicate that the assessment was performed as an integral part of the development of fault trees, that the process included review of procedures and the human interface, and that plant personnel were involved in the process. All of these factors are positive and suggest that, at a general level, a reasonable process was employed. The number of pre initiator human actions included in the model also suggests that a reasonable process was employed and that important actions to assure that appropriate pre-initiator actions were included in the model.

2.2.3 Screenine Process for Pre-Initiator Human Actions.

The submittal states that the screening process for pre-initiator actions employed a twofold strategy: (1) to identify pre-initiator human failure' events (HFEs) no lower than the system-important train level, and (2) then to screen them at 0.003, with a beta factor of 0.1 for multiple train events. The value of 0.003 is consistent with nominal values (not screening values) for errors of omission / commission in THERP. The licensee notes that the use of nominal values rather than a conservative screening value was intended. This is beceuse the human errors are believed by the licensee to be already included in the component failure data, and directly modeling the human errors is, in effect, double counting the impact of the human errors. In our view, more conservative values are appropriate for screening. The value selected for screening should represent a balance between completeness and economy in modeling. Given the degree of uncertainty in models and data for human performance, it is desirable to error on the conservative side. The cutset truncation value employed in the i ANO-2 model was 1.0E-08/yf. This is an " acceptable", but not particularly low, value. In {'

our view a higher screening value for pre-initiator HEPs would have been more appropriate, particularly since pre-initiator HEPs are incorporated into fault trees, where it is possible that multiple human actions may be multiplied inappropriately to reduce the CDF. Furthermore, the licensee states in response to an NRC RAI that the beta factor of 0.1 noted above was not applied during the screening. This increases the non-conservatism of the resalts and increases the potential that some significant cutsets could have been omitted. The licensee asserts that 13 l

l I

t

the impact of not using the beta factor is negligible. We do not have detailed information nccessary to evaluate this assertion.

2.2.4 Ouantification of Pre Initiator Human Actions.

The submittal (page 3.4-4) states that the technique applied to quantify slips is a time-independent technique which is a " variant" of THERP and is "similar" to ASEP. The same technique is applied for pre-initiator and post-initiator slips. The model consists of multiplying the following four factors:

(1) A basic human error probability (BHEP) of was selected, either from tables in Chapter 20 of the THERP Handbook (Ref. 2), or a default value of 0.003. In the ANO-2 analysis the value of 0.003 was used for the BHEP in all cases.

(2) A multiple component beta factor, ranging from 0.01 to 1. The default value of 1 is applied when only a single train or channel is involved. In the case of two redundant trains or channels, the beta factor is 0.1; for more than two, the value of 0.01 is used. The ANO-2 analysis used the default value of I for all pre initiators except two:

AHF2CNTHIL - Operators miscalibrate containment high pressure sensors; the factor of 0.01 was used because it would be necessary to miscalibrate 3 redundant channels for this event to occur (four channels with 2 out of 4 logic).

AHFCALRASL - Operators miscalibrate RWT level sensors; the factor of 0.01 was used for the same reason.

(3) A dependency factor intended to account for redundancy (checking) by another person. This dependency factor is based on a combination of two concepts from THERP: 1) inter-person dependency, and 2) recovery of error by a second person i (checker). The model assigns a multiplier of 0.05 to 1.0 as follows: i 1

1.0 Complete Dependency - the action is performed by one person with no verification, check, or supervision by another crew member 0.5 High Dependency - the action is performed by one person, with intermittenf checks or supervision by another crew member 0.14 Moderate Dependency - the action is performed by one person, with constant, over-the-shoulder verification by another crew member 0.05 Low Dependency - the action is performed by one person, with another crew member either taking part in performing the action, re-performing the l

14 j 1

t

1 action, or constantly monitoring compelling signals during performance of the action.

This simplified model to quantify the potential for inter person dependency, has I some characteristics similar to the THERP/ASEP models for inter-person j dependency. ' Like the THERP model, it is based on judgment' of the analyst and  ;

is fundamentally a " notional" model. That is, the model is logical and provides a  ;

consistent means for quantification of an observed phenomenon, but is not firmly  !

based in theory or empirical data and therefore is arguable. An assumption made in this particular model that seems highly questionable in our viewpoint is that the j above factors are considered individually and independently for four different categories of checkers: (1) the Senior Reactor Operator (SRO), (2) another RO, (3)  :

the shift technical advisor (STA), and (4) "Other Personnel".. In theory, the l overall multiplier, or " Personnel Factor" could be as low as 0.05 x 0.05 x 0.05 x ,

0.05 = 6.25E-06. This assumption of independent checking by four different types  ;

of personnel seems unrealistic and could result in overly optimistic results. The  !

licensee states that in practice, the ANO-2 analysis assumed complete dependency l (multiplier of 1) for all but six cases. A brief summary of the rationale for those i six cases was provided in a response to an NRC RAI. In those cases, the  ;

dependency was assumed to be complete for all but the fourth category "other  !

personnel". The dependency level for other personnel was assumed to be ,

l~  !'

j moderate (0.14) because another technician (maintenance or I&C) acts as an

! independent checker. The overall " personnel factor", therefore was calculated as l.0 x 1.0 x 1.0 x 0.14 = 0.14. This error recovery factor is not inconsistent with i 4 other models and with assumptions used in other PRAs. Thus, it appears that the '

ANO-2 implementation of the model for pre-initiator actions was reasonable (not

{' overly optimistic).  :

i j Any number (typically less than 2) of performance shaping factors were applied,  ;

(4)

! based on the THERP Handbook or analyst's judgment. The licensee states that in .

! practice PSF values of I were ust.d for all pre-initiator human errors modeled in the IPE. That is, no PSFs were accounted for.

Overall, we find the licensee's approach to quantification of pre-initiator human actions to be

) relatively crude and mechanistic. Use of such " generic" screening approaches limits the j ability of the licensee to obtain insights about the potential impact of this type of human error on plant risk and certainly about the underlying factors influencing human error. Those i i

insights could lead to simple, cost-effective ways to improve risk. However, the approach in  :

l general is not inconsistent with treatment of pre-initiator human errors in other PRAs that .

have been accepted. It assumes a basic human error probability consistent with accepted methodology (THERP), modifies that BHEP in some cases by factors intended to account for  ;

l F legitimate error recovery mechanisms, and arrives at a nominal HEP that may be a reasonable  :

estimate for use in the PRA to quantify that error probability. The lack of in-depth plant-specific analysis is a weakness that has beer. observed in some other PRAs/IPEs. Some IPEs l

}

. i 1

15  ;

l 8 1 _ _ _

1 have included a more rigorous assessment, and pre-initiator actions have been found to be a significant contributor to plant risk.

2.3 Post-Initiator Human Actions Human errors in responding to an accident initiator, e.g., by not recogrdzing and diagnosing the situation properly or failing to perform required activities as directed by procedures, can have a significant effect on plant risk, and in some cases have been shown to be dominant contributors to core damage frequency (CDF). These errors are referred to as post-initiator i human errors. The NRC staff review determines the types of post-initiator errors considered by the licensee, and evaluates the processes used to identify and select, screen, and quantify post-initiator errors, including issues such as the means for evaluating timing, dependency among human actions, and other plant-specific performance shaping factors.

2.3.1 Tvoes of Post-Initiator Human Actions Considered.

There are two important types of post-initir.or actions considered in most PRAs: resoonse-tvoe actions, which include those human actions performed in response to the first level directives of the emergency operating procedures / instructions (EOPs, or EOls); and, recoverv-tvoe actions, which include those performed to recover a specific failure or fault (primarily equipment failure / fault) such as recovery of offsite power or recovery of a front-line safety system that was unavailable on demand earlier in the event. The ANO-2 HRA addressed both response-type and recovery-type actions.

The ANO-2 HRA categorizes post-initiator human actions as one of two broad categories: (1) human failure events (HFEs), and (2) recovery events. HFEs are failures in " manual actuations"; recovery events involve " actions of a corrective or recovery nature." These definitions are not totally consistent with the terminology used by NRC (e.g., in the paragraph above). Recovery actions in the SAIC model may be proceduralized or non-proceduralized actions. Mcwever, qualitative screening in the ANO-2 analysis eliminated non-proceduralized procedures. Recovery actions in the ANO-2 analysis include some that are " global", such as recovery of offsite power, and some that are more specific, such as proceduralized steps to align HPSI for hot leg injection, or to crosstie specific valves per procedure. In fact, the majority of actions quantified, per the listing in Table 3.4-1 of the submittal, appear to be recovery actions, in particular, ex-control room recovery actions, per the definition used by the licensee.

Three types of post initiator HFEs and recovery actions were included in the ANO-2 model:

(1) Failure to respond in time, for either (a) in-control room actions, or (b) ex-control room actions (2) Ex-control room slips 16 l

t (3) In-control room slips Consistent with most PRAs to date, human-error-induced initiators were not considered l

directly because they are assumed to be already accounted for in the initiator frequency  :

estimates. Mistakes by the Technical Support Center (TSC) staff or other Emergency i Operations Facility (EOF) personnel were not modeled. .

t The submittal also notes that mistakes, in or out of the control room, were not quantified, and  ;

that this includes errors of commission. The submittal states that these errors are not treated  ;

because they are felt to be highly unlikely because of symptom-based procedures and because modeling of these kinds of errors would require " breaking new ground," which was not ,

intended for the IPE program: These statements may require some interpretation. It is true  !

that cognitive errors leading to errors of commission, e.g., misdiagnosis leading to incorrect assumptions and therefore inappropriate actions on the part of the crew, are in general not i modeled explicitly in PRAs. For example, an error of commission which would significantly l alter the course of the accident sequence, say to the point where different EOPs should be  ;

applied, typically is not modeled. However, the time reliability correlations used in the .

ANO-2 analysis and in other PRAs are not necessarily restricted to errors of omission. The  !

correlations focus on the distribution of crew response time, which is the time until correct  :

action is taken. The correlations do not deal with the nature of the error or the reason for delay. They simply provide a probability that the correct action will be taken, or not taken,  !

within a specified time. This failure to realistically model fully the dynamic nature of the  :

accident sequence and the interaction of the crew and plant systems is a weakness of the time reliability correlations, and in fact all HRA techniques that have been employed in IPEs.  :

Thus, explicit modeling of mistakes, as indicated by the licensee, typically is not pe-formed in l PRA. But simplified techniques such as the time reliability correlations can be said to model both errors of omission and errors of commission, though with a limited degree of i

sophistication. With regard to the claim that mistakes or errors of commission are highly unlikely due to the use of symptom-based procedures, it is true that one of the basic intents of i the symptom-based procedures was to reduce the " cognitive" demands on operators in responding to accident events, and we agree that it is reasonable to assume that they have ,

been effective in reducing the likelihood of mistakes, i.e., errors in diagnosis, decision I making, etc. However, we do not agree that evidence exists to support the assumption that i I

such errors are no longer significant.

1 I

A significant difference betwgen HFEs and recovery actions is the manner in which the two are incorporated into the IPE model. HFEs are included in fault trees in the same manner as equipment failures. However, sequence-specific influences on the HEP were accounted for by using different HEPs for the same action in different sequences. (See Section 2.3.4.4 below.)

. Recovery events were added to individual cutsets during the cutset review process after ' initial quantification and therefore are cutset-specific. The submittal notes that a maximum of one recovery event was added to a cutset. When more than one recovery event was possible, the j one most likely to be attempted, based on review of procedures and operator interviews, was selected.

17

% i j

2.3.2 Process for Identification and Selection of Post-Initiator Human Actions.

The primary thrust of our review related to this question is to assure that the process used by the licensee to identify and select post-initiator actions is systematic and thorough enough to provide reasonable assurance that important actions were not inappropriately precluded from examination. Key issues are whether: (1) the process included review of plant procedures associated with the accident sequences delineated and the systems modeled; and, (2) discussions were held with appropriate plant personnel (e.g., operators, shift supervisors, training, operations) on the interpretation and implementation of plant procedures to identify and understand the specific actions and the specific components manipulated when responding to the accident sequences modeled.

The submittal does not provide much discussion of a systematic process for identification of human errors to be included in the IPE model. However, there are general statements in a number of discussions in the submittal indicating that procedures were reviewed and that operations and training personnel were appropriately involved in identification and review of operator actions. The discussion of accident sequence delineation and the sequence descriptions in Section 3.1 of the submittal identify specific operator actions as key events in the sequence. These discussions suggest that a reasonably systematic approach was used to identify important operator actions as the accident sequences were defined and that substantial emphasis was placed on identifying imp ~ tant operator actions.

Much of the initial basis for accident sequence delineation (including identification of important human actions) was previous industry PRA studies. Safety functions were identified from the immediate actions section of the EOPs and from a CE publication describing critical safety functions. Identification of initiating events was based on EPRI publications, plant-specific history, and review of previous PRAs of similar plants, including ANO-1, Calvert Cliffs, SONGS-2&3, and Oconee. The event trees from the ANO-1, Waterford Steam Electric Station-3, and Calvert Cliffs PRAs provided the initial basis for functional event tree development. In fact, the process for event tree development involved primarily review and adjustment of these existing trees to the ANO-2 design. It appears that identification of important operator response actions was an integral part of the sequence anaysis (event tree development) and of the development of the top logic fault tree models in the systems analysis. General statements in the submittal cite the involvement, at least by way of review, of plant personnel with the appropriate expertise in operations and systems design.

As indicated above, recovery actions were identified from review of important sequences (generally defined as cutsets with a CDF of IE-08 or greater) after initial quantification, and were not hicluded in fault trees. Selection of recovery actions to be included was based on a

" qualitative screening" described in Section 2.3.3 below. The submittal states that appropriate plant personnel in operations, engineering and maintenance were involved in this process via interviews and independent review.

18 t

J

l A high-level comparison of post-initiator human actions selected for incorporation into the IPE model with human actions typically included in other PWR PRAs indicates that, in  ;

general, the important actions appropriate for ANO-2 were included.

Based on the above findings, it appears that the licensee employed a reasonably systematic process to identify and select potential post-initiator actions, which included review of procedures and discussions with plant personnel, and that the process provided reasonable assurance that important actions were not overlooked. '

2.3.3 Screeninn Process for Post-Initiator Response Actions. ,

The submittal states that all post-initiator HFEs were screened at an initial value of 0.4. This  ;

numerical value is typical of values that have been found to be effective screening values in other PRAs. The licensee notes in its response to an NRC RAI that all post-initiator ,

response-type human actions remaining in the model after the truncation of cutsets below the 1.0E-08/yr cutoff were requantified using one of the more detailed modeling approaches.

Recovery actions were screened by a qualitative process involving the following three criteria, all of which had to be met in order for the recovery action to be included in the model:

(1) Is a recovery physically possible (i.e., does equipment that could mitigate the ,

current failure exist)?

(2) Is there time available to accomplish the required actions?

(3) Is the recovery addressed in procedures, taught in training, or otherwise obvious to operators?

As indicated earlier in this TER, the submittal notes that at least for some ex-control room recovery actions walkdowns were performed and interviews were conducted with plant personnel as part of the evaluation.

2.3.4 Ouantification of Posalnitiator Human Actions.

For HFEs determined to be important enough to require calculation of nominal HEP values or recovery events surviving the qualitative screening, there were two types of quantification techniques employed: (1) a time independent technique, and (2) a time-dependent technique.

2.3.4.1 Time-Indeoendent Ouantification Techniaue. The time-independent technique was applied to post initiator slips. It is the same simplified model that was used for pre-initiator human actions. Two post initiator slips were evaluated:

19 1 t

4 (1) FHF20FWCSD - Operaters fail to properly control main feedwater (MFW) when switching to manual control during the shutdown process following a trip; HEP = 2.lE-05.

(2) OPER Operator fails to maintain RCS pressure less than MSSV setpoint; HEP

= epsilon (i.e., negligible).

The assumed BHEP in both cases was 0.003. The default value of I was used for the multiple component beta factor. All performance shaping factors assumed the default value of

1. The multiplying factor responsible for the final value is the " personnel factor," which includes multiplying dependency factors for the four different types of crew members. In case 1 above, the SRO dependency level was assumed to be moderate because the SRO provides over-the-shoulder verification of steps in the Emergency Operating Procedures (EOPs). Dependency of the "Other RO" category was assumed to be low because of the "close interaction between the Turbine Operator and Reactor Control Board Operators during the shutdown process following a trip. Dependency was assumed to be complete for the other two types of personnel (STA and "Other"). Thus the personnel factor was calculated as 0.14 x 0.05 x 1 x 1 = 0.007; and the HEP value is 0.003 x 0.007 = 2.lE-05.

In the second case above, the dependency for the SRO, Other RO, and STA was assumed to be low (a multiplier of 0.05), and the personnel factor was 0.000125, which results in the very low HEP value.

)

As indicated earlier, we believe this modeling approach of assuming four independent  !

possibilities for recovery to be unrealistic. It easily could produce overly optimistic results.

Without a detailed assessment of our own, it is not possible to judge whether the numerical l values in this case are overly optimistic. The actions quantified using this simplified approach are very i'ew, and the impact on the overall IPE results probably are not significant.

However, this overly simplified, mechanistic, and probably optimistic approach is considered a weakness of the licensee's HRA.

2.3.4.2 Time-Decendent Ouantification Techniaue. The time-dependent quantification technique is used for the majority of the HEPs quantified. There are two variants of the technique; one for in-control room actions (Ref. 3), and one for ex-control room actions (Ref.

4). Both are part of the TRC approach developed by SAIC and, as discussed earlier, are intended by the licensee to treat failures to perform the required action within the required time. The licensee does not i~ntend these models to apply to mistakes, including errors of commission. Other PRAs typically have used time reliability correlations to represent the

" cognitive" aspects of operator response to an accident, including " diagnosis", " detection", and

" decision making". While the licensee categorizes errors as either slips or mistakes, the licensee's use of time reliability correlations quantitatively is essentially the same as in other PRAs. One difference from some other PRAs is that the licensee apparently makes no attempt to treat both " cognitive" and " manipulative" aspects of the same action and then combine them as is done in some approaches.

20

i l

l Since there are two types of human errors (HFEs and recovery actions), and two locations (m I out of the control room), there are four different treatments using the time-dependent i techniques (TRCs):

l (1) In-control-room HFEs, treated by using the in-control-room time-dependent technique; included in the fault tree models; ,

(2) Ex-control-room HFEs, treated by using the ex-control-room time-dependent j technique; included in fault tree models; )

I (3) In-control-room recovery actions, treated by using the in-control-room l time-dependent technique; added to the cutset after model quantification; or (4) Ex-control-room recovery actions, treated by using the ex-control-room time-dependent technique; added to the cutset after model quantification.

The majority of the HEPs identified in Table 3.4-1 of the submittal are of the last type above, '

ex-control-room recovery actions.

l Time-Devendent In-Control Room Techniaue. The TRCs are lognormal distributions based on simulator data for measure time required until completion of correct action. Input parameters  !

to the in-control-room technique are:

l (1) Net time available - the difference between total time available and other times as human factors considerations require, such as a cue that occurs at a time later than time zero, or an action that takes more than a minute or so to carry out.

(2) A type factor: "0.25" for verification actions; "0.5" for rule-based actions; "1" for others. The type factor adjusts the input of the time available to complete the operator response to the " base TRC". The base TRC is fitted to a curve which is assumed to apply to responses dominated by diagnosis, not aided by rules, and r.ot dominated by burden or some other source of hesitancy. Nominal responses (those that were judged to fit this type) were assigned a type factor of 1.0. For actions which are highly proceduralized, or " rule-based", a type factor of 0.5 was used. A type factor less than one reduces the HEP estimate substantially. For example, using a 0.5 instead of I reduced the HEP one HEP from 6.0E-02 to 7.2E 03.

(3) A success likelihood f ctor to reflect various performance shaping factors. The SLI can range from 0 to 1, with 0.5 being " nominal". Nine performance shaping factors are potentially considered in the model: 1) procedures,2) training, 3) indications, 4) team structure and dynamics,5) difficulty of task,6) preceding actions context,7) experience, 8) communication, and 9) perceived consequences. Using a value of 0.5 has the effect of not adjusting the input, or in effect, using the base TRC. Using a value less than 0.5 or greater than 0.5 effectively changes the ratio of the time 21

1 available to complete the operator action, and therefore modifies the HEP. In all cases for the ANO-2 assessment, the default value of 0.5 was used, which effectively is not using the SLI parameter. The licensee states that, "Most of these factors are unknown for all situations in which a recovery could be applied or are considered in other factors (e.g., the presence of burden considers both difficulty and perceived i consequences)."

(4) ' A burden factor: "1" when no burden is assumed; "2" otherwise. The burden factor was used in the evaluation of the error factor associateo with the estimate of the mean  !

HEP. The presence of burden will increase the calculated mean,95th percentile, and i Sth percentile. A burden was assumed to exist if the operator faced the choice between a perceived severe consequence of action and more severe consequence of inaction. The licensee provided an example of the operator action to start once-through cooling upon loss of feedwater (OPER-2 for non-LOCA conditions). This action is considered to be burdensome, because of operators' reluctance to breach the reactor coolant system, even though the action is proceduralized in the EOPs. The calculated value with the burden factor of 2 was 4.5E-03; without burden, the calculated value would have been 3.8E-05.

(5) A model uncertainty, or error, factor, which is fixed at "1.68", indicating that the model uncertainty is distributed lognormally about the mean.

Time-Devendent Ex-Control-Room Techniaue. The parameters for the ex-control-room technique are as follows:

(1) Net time available 1

1 (2) An estimate from operations personnel's judgment or walkdowns of the expected time  ;

to locate, access, and manipulate the equipment, (3) Additional time to reflect the potential delaying effects of specific types of l ex-control-room hazards (e.g., contamination, steam),

(4) A " hazard factor", which is intended to account for specific factors which could impede operator response. The hazard factor actually is implemented in one of two i

ways: a) by increasing.the response time, or b) an adjustment to the error factor. An increase in the response time increases the HEP. The error factor increases the mean j

HEP and upper and lower bounds. The licensee provided for review " rules of thumb"  ;

I that 'were used for additions to the response time and adjustments to the error factor.

l Examples include adding one minute to the response time for reduced lighting or 10 minutes for no lighting, adding 15 minutes to the response time if protective clothing are required add I to the error factor (nominally 4.3905 for the ex-control room model) if non-operator support (such as shift maintenance) is required, and adding 1 to the adjustment factor for actions that are complex or not included in training. These 22 i l 2

3 9

adjustments provide a mechanism for adjusting the final HEP based on judgment of the analyst.

(5) A model uncertainty, or error, factor. The error factor for the ex-control-room model is greater thm the value used for the in-control-room model (4.3905 vs. 3.2). This is intended to adjust the correlation to account for coordination of ex-control-room activities from the control room. The error factor can be adjusted to account for influences such as security access, noise, availability of tools, or presence of radiation, which may be significant factors for ex-control-room actions.

Use of Model Parameters it should be recognized that the manipulation of the above parameters is simply a mechanistic means for implementing the analysts judgment, in particular to modify the basic data from the published correlations. The numerical results resulting from the analysts judgment should not be viewed necessarily as more " correct" because the mechanism is more involved or complex than simply adjusting the HEP.

However, the use of these parameters does provide a useful guide for the analyst to structure his/her thinking about factors that influence the human performance, for adjusting the  ;

simulator-based time reliability correlation results based on plant-specific assessment and for providing a reproducible record of the judgments made. From the information presented by ,

the licensee, it is apparent that there was some plant-specific assessment performed to support the judgment and the selection of some of the parameter values. It is difficult to judge from  :

the summary information in the IPE submittal the degree of rigor and depth in this plant- I specific evaluation. Note for example, that the SLI parameter, which is most directly  ;

associated with the usual performance shaping factors considered in HRA techniques, was not really used. Overall, the assessment performed by the licensee appears to be " typical" of other IPEs reviewed; i.e., some others have been more extensive in their plant-specific assessment (e.g. by performing comprehensive simulator runs or detailed evaluation of PSFs),

and some have used more generic, less rigorous evaluations.

2.3.4.3 Basis for Time Estimates. While the other parameters in the model can be used to adjust HEP values obtained from the basic time reliability correlations, the most important parameter is the relative time available, i.e., time required for the action vs. time available for completion of the action. Therefore, the licensee's basis for determining timing estimates is important. The licensee states that, "The available time was based on thermal-hydraulic calculations (when available, adjusted as appropriate), simulator data, informal quantitative estimates (when simplicity permitted), or conservatively bounding engineering judgment. The basis for each time available used is documented in the HRA work packages." While specific details are limited, this appears to be a reasonable approach typical of other PRAs for obtaining the time available.

' For most in-control-room actions, the estimated time required for operator actions was determined from operator interviews and " inspection of operating procedures." A concern in  ;

using operator estimates is that operator judgment on time required, particularly when obtained from unstructured interviews, often is optirnistic. Recommended practice is to 23 f

l multiply operator time estimates, e.g., by a factor of 2, to assure appropriate conservatism. In response to an NRC RAI, the licensee indicated that operator response time estimates were not adjusted. The licensee states that the human reliability analyst for ANO-2 is intimately familiar with the plant systems / components and their location and has a good knowledge of plant operations. The analyst verified (using his judgment) estimated response times for each action and found them to be, in his view, consistently conservative (longer than expected).

The licensee notes that this result was at least partially due to pre interview preparation and explanation to the operators on the intended use of the data.

In some cases, time required for in-control-room action was determined from simulator data.

The licensee notes an example case in which simulator data was used to obtain both operator response time and total available time by performing separate simulator runs with and without operator action. In this case, it appears that the operator response time estimate was based on a single run (per accident event), which provides a very limited statistical basis. Nonetheless, use of the simulator in this fashion to support HRA is a positive contribution to the IPE process, and while the licensee did not comment on this facet, probably helped to involve operations and training staff in the PRA process.

Operator judgment was also the primary basis for response-time estimates for ex-control-room actions. Those estimates attempted to account for extra time required to complete actions due to degraded plant conditions associated with the severe accident situation, e.g., degraded lighting during loss of power events.

While direct data sources - plant data, simulator data, timed walkthroughs, etc. - are preferred methods for assessing expected response times, operator judgment has often been used in PRA studies as a primary source. In the case of ANO-2 the operator estimates appear to have been guided by an interview process with some structure, and were reviewed by an HRA analyst very familiar with the plant systems / operations. In general, our review of sample HEP record sheets provided by the licensee suggests that response time estimates were reasonable  ;

(appropriately conservative),

i A general question of interest regarding the use of the published time reliability correlations is the consideration given by the licensee to the applicability of the simulator data to the specific I

plant being analyzed. In the SAIC models described above, there are more than the usual number of parameters which allow the analyst to effect his/her judgment based on a plant- l i

specific assessment and comppison to the baseline data. Additional basis for assuming applicability of the data for ANO-2 might be provided by demonstrating that the SAIC time I reliability correlations employed provide results consistent with other accepted models/ studies. l In response to an NRC RAI that there had been some attempt to " calibrate" the SAIC time reliability correlations used for the ANO-2 IPE to at least two other sources: I) the THERP nominal diagnosis curve, and 2) the ANO-1 IREP study (Ref. 5). The THERP comparison  ;

was made for the in-control-room model. The SAIC model is " calibrated" to a 60-minute rule-based acticn. An HEP value of IE-06 is obtained from the SAIC model using a success likelihood index (SLI) of 0.7, which matches the THERP value. The ex-control-room SAIC 24 ,

I l

t

l 1

i I

)

model matches the IREP value of 0.03 for an available time of 60 minutes and a mean l

response time of 10 minutes. Obviously, these " calibrations" are very limited, but they do provide some additional basis for, or at least indicate an awareness of the need to consider,  :

the applicability of general time reliability correlations to the specific plant ofinterest. l 2.3.4.4 Treatment of Decendencies in Post-Initiator Actions. An important concern in HRA is the treatment of dependencies. Human performance is dependent on sequence-specific response of the system and of the humans involved. The likelihood of success on a given action is influenced by success or failure on a preceding action, performance of other team members in parallel or related actions, assumptions about the expected level of performance of other team members based on past experience, etc. Thus two human actions that are essentially the same but occur in a different sequence / context could have very different HEPs.

Failure to account for these differences can lead to missed or inaccurate insights regarding the impact of human performance. When there are more than one significant human action required in an accident sequence, it is important to account for the impact that failure on one action has on the subsequent action. Error probability estimates for HRA are conditional probabilities. If dependencies are not specifically accounted for, and HEPs are treated as independent, the probabilistic combination of HEPs can lead to an unrealistically low estimate of human performance overall (i.e., of the joint human error probability), and to a significant ,

underestimate of risk, i The submittal discussion of the treatment of dependencies for post-initiator actions consists of the following sentences (page 3.4-5):

Interpersonal dependencies were modeled explicitly in the ANO-2 HRA for slips. The crew in the control room is modeled as a unitfor untimely responses. Ex-control room actions post-initiator are assumed performed by a l single person. For a pre-initiator the interdependency model is used. j This discussion deals with the issue of inter-person dependency. The first sentence about interpersonal dependencies modeled explicitly for slips apparently is a reference to item (3) in i I

the list of parameters discussed in Section 2.3.4.1 above for the time-independent model, in which the HEP was multiplied by a factor (default = 1) to account for recovery by another person. The second sentence regarding treating the control room crew as a unit rather than l explicitly modeling inter-person dependencies is consistent with the proper use of the time l reliability correlations. The third statement indicates that inter-person dependencies were not addressed for ex-control room actions, which is not unreasonable. The " interdependency" model for pre-initiators was discussed earlier in Section 2.2.4. l l

The above discussion does not address the issue of within-person (or within-crew) l dependency, i.e., the impact that failure of one task has on a subsequent task. For example, the top logic for the steam generator tube rupture includes two operator actions (OPER-5A, failure to depressurize the RCS, and OPER 12, failure to actuate MSIV closure) which l 25 l

l 1

j

apparently are treated as having zero dependence. In response to an NRC RAl, the licensee indicates that such dependencies were not explicitly treated in the ANO-2 model. The licensee states that during the screening process, the cutsets were reviewed, and the combinations of modeled human actions which occurred were evaluated. (While the licensee doesn't state this directly, we assume that combinations of actions that were completely illogical either did not occur or were removed from the model.) The licensee determined from this review that dependencies in each case could be considered negligible because the actions were separated in time, involved completely different systems, or were performed by different individuals. These are logical reasons for assuming at least a lower level of dependency. Without the details of the case-by-case assessment, we are not able to evaluate the licensee's judgment. In general, human performance is highly context and situation dependent. The assumption of zero dependency on preceding actions should be viewed skeptically and applied only when there strong indication of relative independence.

The above discussion also does not address sequence-specific influences on human actions.

Based on the discussion of the analysis and treatment of recoverv actions, which were identified and quantified in the context of specific sequences (cutsets), such dependencies were implicitly accounted for in the analysis of human recovery actions. For the HFEs, which were included in system fault trees, it is more difficult to account for such dependencies. In response to an NRC RAI, the licensee indicated that sequence-specific impacts were considered for the HFEs by using sequence specific basic events in the fault tree logic. That is, an action that is essentially the same in two different sequences could have two different basic event identifiers and two different HEP values. An example provided by the licensee is operator actions OPER-13 and OPER-14. Both actions are " Operators fail to energize 2Al/2A2 from SU#2." OPER 13 is used for this action in transients and has an available time of 55 minutes. OPER-14 is used in the small break loss of coolant accident and steam generator tube rupture sequences where the available time is 30 minutes. This s

example and the licensee discussions do not provide an indication of what other parameters besides available time were considered in such cases, but the modeling approach certainly permits the analyst to account for sequence-specific influences on the other parameters in the time reliability correlation.

l 2.3.4.5 Eauiement Failures Associated With Recoverv Actions. As noted previously, recovery actions were appropriately treated on a sequence-specific basis, and human error l recovery probabilities were added to cutsets after model quantification. To address the issue of .

random failures of associated, equipment, the licensee made an arbitrary decision that for cases j in which the HEP for the human recovery action is IE-03 or greater, that the human failure )

dominates, and the equipment failure probability can be ignored. For cases in which the HEP l is below IE-03, the HEP was adjusted (increased) to account for the potential failure of l associated equipment. The technique for determining the appropriate equipment failure (and therefore the amount of adjustment) depended on the particular equipment involved.

Examples provided in the submittal were: 1) in cases in which the recovery event involved the use of entire system and that system was modeled in the IPE, the failure probability of a gate in the fault tree might be used; and,2) if the recovery event involved manually opening two 26

)

series valves (both required for success) the failure probability for that valve was multiplied by two. Regardless of how the equipment failure probability was determined, the probability was simply added to the human error probability to arrive at an overall recovery probability, which was then added to the cutset. We consider this approach to be reasonable. The case by-case assessment of the equipment associated with the human recovery action is an added potential for gaining insights. And, while the IE-03.value is arbitrary, it is unlikely that significant error is introduced by ignoring equipment failure probabilities when the HEP is greater than that value. ,

2.3.4.6 Ouantification of Human Actions in the Flooding Analysis. A fairly lengthy -

discussion of the internal flooding analysis is presented in Section 3.6 of the submittal. The analysis consisted of a series of successive screening analyses with progressively "more realistic" or "less conservative" assumptions. Ultimately, all identified flooding sequences but one were screened out (on the basis of qualitative criteria or an estimated CDF contribution below IE-06/yr). In the initial quantitative screening analysis, all ex-control-room human actions were assumed to fail, and all in-control-room actions associated directly with initiation or propagation of flooding were assumed to fail. In addition, no cutset recovery factors were applied. In a refined quantification, a general human error probability of IE-02 was assumed I

for human recovery action to identify the source and isolate flooding within 20 minutes.

Also, ex-control room actions not affected by the flood were credited, and additional flood-specific recovery was applied where deemed appropriate. The final quantification eliminated all but one sequence.

I The licensee states that "The [ HEP] value of IE-02 was developed based on judgment which factored in the plant design features, including system annunciators, building sump level i

indication, plant personnel, etc." No other details are provided. This value has been used '

without detailed justification in other PRAs for operator action to identify and isolate floods.

In the case of ANO-2, the licensee notes that value was used only for cases oflarge floods, which should be relatively simple to detect within the assumed 20 minute time period because of the rapid system response. In a few cases, recovery actions were credited in the flooding analysis using HEP values calculated from the time reliability correlations for other sequences.

The licensee noted in its response to an NRC RAI that this credit was taken only in cases in which the flooding conditions would not affect the operators' ability to accomplish those recovery actions.

In general, the HRA associatesi with the flooding analysis used a highly simplified approach, i essentially assigning an HEP value for credible recovery actions based on analyst judgment. l l

This approach, and indeed the HEP value of IE-02, has been used in other PRA flooding analysis. While a more detailed assessment would be desirable, we have no firm basis in this document-only review for judging whether the particular values selected are or are not conservative.

27

2.3.4.7 Human Actions in the Level 2 Analvsis.

l The submittal states that, " Generally, no credit was taken for operator recovery beyond core damage." There is no indication that operator actions were considered directly in the Level 2 l analysis. Some general reference was made to the potential for human action contribution for two areas: 1) human action to recover containment sprays after recovery of AC power could ,

introduce a potential containment failure mode by de-inerting the containment and thus l increasing the potential for large hydrogen burns, and 2) operator action may be considered to  !

initiate depressurization for some sequences. These were discussed as potential actions and I were not quantified in the Level 2 analysis.

2.3.5 GSI/USI and CPI Recommendations. I Review of the submittal discussions of Generic Safety Issues (GSIs) and Unresolved Safety Issues (USIs) is primarily the focus of the front-end reviewer. Review of submittal discussions of any licensee actions in response to Containment Performance Improvement (CPI) recommendations is performed primarily by the back-end (Level 2) reviewer. If the licensee's discussion of these issues has particular significance to the HRA or human performance issues, those points are included in this review.

The front-end reviewers discussed the licensee's treatment of decay heat removal (DHR), USI A-45. The licensee employed a classification scheme based on mean CDF due to DHR failure. Based on the IPE results indicating that sequences with failure of DHR sequences have a CDF contribution of 3E-05/yr, the licensee concludes that the ANO-2 DHR vulnerability falls in the category 2 of 3, i.e., in between category 1 of " acceptably small" level and category 3 requiring prompt action. However, a sensitivity study performed to estimate the impact of plant improvements indicated that the improvements reduce the CDF to 1.5E-05/yr, which is category 1, and therefore that no unique DHR vulnerability exists for ANO-2. The licensee considered diverse means of decay heat removal, including: use of the power conversion system (PCS), feed and-bleed cooling, auxiliary feedwater, and emergency core cooling system. The licensee also addressed GSI-23 " Reactor Coolant Pump Seal Failures" and GSI-105 " Interfacing Systems LOCA at LWRs" in the IPE. Both types of events were determined by the licensee to have small contributions to CDF for ANO-2.

The original submittal did not include a response to CPI recommendations. However a separate contractor report evaluated containment and equipment vulnerabilities to local and global hydrogen combustion. . The Level 2 reviewer concluded that the licensee's contractor report was responsive to the CPI recommendation for PWR lag, dry containments regarding these vulnerabilities.

2.4 Vulnerabilities, insights and Enhancements 28 t

l

2.4.1 Vulnerabilities.

Vulnerability screening is discussed in Section 3.7.2 of the submittal. The licensee defines a vulnerability as a sequence group with a mean core damage frequency (CDF) greater than IE-04/yr or a containment event tree (CET) endstate group with a mean CDF with containment failure / bypass greater than IE-05/yr. This definition is based on, and partially adopts, NUMARC 91-04 (Ref. 6) guidance. Specific exceptions to the NUMARC guidance are identified, and a rationale is for those exceptions is provided. Conclusion regarding vulnerabilities that are listed in the submittal imply additional considerations by the licensee in screening for vulnerabilities: (1) the overall core damage frequency is within the range of values typical of other published PRAs for PWRs, and (2) no individual cutset contributes .  ;

i more than 25% of the total core damage frequency. In general, the licensee's approach to defining and screening for vulnerabilities appears to be reasonable. No vulnerabilities were identified. ,

2.4.2 Insiahts Related to Human Performance.  :

2.4.2.1 Imoortant Ooerator Actions. The importance of operator action to the estimated core ,

. damage frequency is cited in the summary sections of the submittal (Sections 1.4 and 7.0) as ,

one of the major findings of the IPE. Two actions, both dealing with a potential for a l complete loss of heat removal capability, are cited as particularly important in these summary i discussions:

(1) Manual transfer of offsite oower from the auxiliary transformer to the start-uo transformer; there are five different HEPs for this action, depending on the specific {

sequence involved; one (MANOSPREC, HEP = 1.3E-01) is listed (Table 3.5.4-3) as the most important basic element,' including human errors and equipment failures, with a Fussel-Vesely importance of 0.471. Ultimately, each sequence involves a loss of normal (MFW), emergency (EFW), and alternate (AFW) feedwater supply and the failure to initiate once-through cooling (bleed and feed).

I (2) Realionment of AC or DC oower from a faulted train to an available train: upon loss of a 4160V AC safety-related bus, the operator must realign the affected battery to the swing battery charger or realign the affected 480V AC bus to the unaffected j l

4160V bus; the actions ACREALIGN (HEP = 2.6E-01) and ACREALIGND (HEP

= 9.5E-02, used after battery discharge) are listed in Table 3.5.4-3 among the ten i most important basEc events; with Fussel-Vesely importance of 0.141 and 0.139, )

l respectively. Sequences involving loss of DC bus 2D01 or 2D02 with failure of the unaffected EFW train (TBF-1 sequences) comprise the largest percentage of core  !

damage cutsets and provide the single largest cutset of the ANO-2 model. These  !

sequences ultimately lead to total loss of heat removal as with the sequences noted  !

above. Operator actions to realign 2D01 or 2D02 to the swing battery charger l 29 l

1 t

1 2D34 (OPER-15/16, HEP = 7.9E-02) are listed as the fourth and twelfth most important basic elements.I

! Operator action to trip the RCP pumps following a loss of service water (OPER-1) is treated 4

as necessary to prevent a seal LOCA. This is the second most important basic element overall, with a Fussel-Vesely importance of 0.399. The front-end reviewers noted that the  ;

assumption that a seal LOCA will not occur if the pumps are tripped within 30 minutes is

unusual in PWR PRAs. On the other hand, the licensee notes that assuming failure of HPSI ,

due to the seal LOCA is conservative because HPSI pumps do not require lube oil or room cooling during the initial injection phase. (HPSI is required for inventory control if a seal i LOCA occurs.) The HEP for this in-control-room action is 6.0E-03.

i Operator actions related to control emergency feedwater (EFW) pump trains are important to

establishing shutdown cooling after failure of DC or AC buses or in the case of station blackout. Operator action P7AMANREC, " operator fail

, to manually control EFW pump" (HEP = 2.0E-01) is among the ten most important operator actions. Overall, six of the ten  ;

4 most important basic events, and nine of the top sixteen, are human errors. Table 2-2 below

[ lists the human errors with Fussel-Vesely importance 0.01 or greater.2

~

2.4.2.2 Sensitivity Studies. Section 3.5.4 of the submittal summarizes a number of sensitivity  :

j studies performed by the licensee and insights noted by the licensee based on those studies.

These studies, most of which have direct or indirect relevance to human performance, are j summarized below.

J 2.4.2.2.1 Use of Plant-Soecific vs. Generic Data. As with most PRAs, the ANO-2 IPE used a combination of plant-specific and generic data. The licensee notes that the differences for ,

component failure rates are most notable for the turbine driven pump failure to run and for air l l operated valves failure to operate. The licensee performed two model quantifications, one j j using plant-specific data and the other using generic data. The submittal presents the top 100 .

l i

Table 2-2 I Two other actions, DCPIACGN and DCREALIGND, are listed in Table 3.41, Human Failure Event Data, with the same description, but do not appear in the listing of basic event importance in Table 3.5.4-4.

2 The submittal includes two tables of imponance rankings. Data was obtained for some components on a '

l plant-specific basis and on a generic basis. Table 3.5.4-3 is for calculations using plant-specific data. Table 3.5.4-4 is for calculations using the generic data. The information in Table 2-2 and the comments in the text regarding relative importance rankings are all taken from Taole 3.5.4-3 using plant-specific data.

30 l

l

r Human Actions With F-V Importance 2 0.01 l I

DESIGNATOR DESCRIPTION TYPE

• HE F-V RANKb MANOSPREC Operators fail to align offsite power to 2Al/2A2 EX 1.3E-01 4.71E-01 1 OPER1 Operator fails to trip RCPs in 30 minutes IN 6.0E-03 3.99E-01 2  :

OPER 15 Operator fails to realign 2D01 or 2D02 to 2D34 EX 7.9E-02 2.55E-01 4 j MANOSPRECD Operators fail to align offsite power to 2Al/2A2 EX 5.7E-02 1.74E-01 6 ACREALIGN Operators fail to realign 2B5 from 2A3 to 2A4 EX 2.6E-01 1.41E-01 7 l i

ACREALIGND Operators fail to realign 2E5 from 2A3 to 2A4 EX 7.9E 02 1.39E-01 8 OPER 16 Operator fails to realign 2D01 or 2002 to 2D34 EX 7.9E-02 9.49E-02 12 l P7AMANREC Operators fail to manually control EFW pump EX 2.0E-01 4.93E-02 15  :

QCSTKXFER Operators fail to align EFW suction to alternate ANO-1 condensate source EX 4.6E-02 3.61E-02 16 l T7 REC Operator fails to start standby service water pump IN 2.3E-02 2.31E-02 21 l

OPER-2 Operator fails to trart once-through cooling within  :

one hour (small break LOCA) IN 4.5E-03 1.82E-02 24 l ACXTIE Operator fails to reduce loads & crosstie 2A3/2A4 EX 2.4E 01 1.63E 02 26 P7AMANRECD Operators faii to manually control EFW pump '

speed and discharge valves; after batt. disch. EX 6.7E-02 1.50E-02 30 SWECPREC Operators fail to trnsfr to ECP on loss SW pump EX 1.7E-Oi 1.49E-02 31  :

FOPERFWCSD Operators fail to prevent SG overfill EX 7.2E-03 1.26E-02 34 1

QHF2 REFILL Operators fail to align EFW suction to alternate condensate source EX 1.2E-02 1.lE 02 38 l

(a) ' Type" refers to the model used to quantify the HEP: EX = ex-control room TRC technique;IN =

in-control-room TRC technique.

(b) " Rank" refers to the relative ranking of.a.l]. basic events, including human error and equipment failure. ,

cutsets for and basic element importance values for both calculations. The CDF estimate of 3.28E-05/yr using plant-specific data is reduced to 2.01E-05/yr with the generic data set. The '

licensee asserts that while this is a significant difference in CDF estimate, use of either data set does not affect conclusions of the IPE. From our review and comparison of the dominant  :

cutsets and importance values (Table 3.5.4-1 A vs. 3.5.4-1B, and Table 3.5.4-3 vs. 3.5.4-4) it appears that this general conclusion by the licensee is accurate. That is, the order of dominant cutsets and the relative rankin'g of specific basic events changes, but not dramatically. The dominant sequence group by far is still the TBF sequences, which accounts for the majority of the CDF estimate, and the top 30 to 50 basic elements are not significantly different.

2.4.2.2.2 Test and Maintenance Unavailabilitv. The sensitivity of the overall core melt frequency to the contribution from estimated unavailability due to test and maintenance activities was estimated by quantifying the model with these data increa&d by a factor of ten ,

and with these data reduced by a factor of ten. A f actor of ten increase resulted in a 58%

31 f

+ ,- .- -, - -

increase in CDF to 5.17E-05/yr; a factor of ten reduction resulted in a 6% decrease in CDF to 3.09E-05/yr. The licensee's conclusion is that, while the test and maintenance activities have a significant impact on CDF, they do not dominate or drive the estimate. The scope of our review does not include review of these data values for reasonableness. The results of the sensitivity study suggest that it probably it would be prudent to assure that the estimates are reasonably conservative. Decreasing the values has little effect. However, if the "true" values are higher, the impact on CDF is not significant.

2.4.2.2.3 Common Cause Data. The uncertainties associated with common cause data and the overall impact of common cause failures on the estimated CDF was examined by a similar set of calculations increasing and decreasing common cause beta factors by an order of magnitude. The quantitative impact was very nearly the same as for the test and maintenance unavailability. Increasing the beta factors by a factor of ten resulted in a 54% increase in CDF (to 5.06E-05/yr); decreasing them by a factor of ten resulted in a 6% decrease in CDF (to 3.09E-05/yr). The licensee again concluded that while these results indicate common cause failures are significant, they should not dominate or drive the CDF results.

2.4.2.2.4 Loss of Offsite Power Recoverv. To examine the sensitivity of CDF estimates to loss of offsite power recovery, the non-recovery probabilities were decreased by a factor of ten (i.e., the potential for recovery was increased) and the non-recovery probabilities were increased to match the values used in the NUREG-ll50 Sequoyah study. Increasing the non-recovery probabilities to the Sequoyah values (i.e., reducing the probability of recovery) resulted in a 41% increase in CDF (to 4.64E-05/yr). Increasing the probability of recovery by a factor of ten resulted in a 5% decrease in CDF (to 3.13E-05/yr). The licensee notes that loss of offsite power recovery has a significant ef'ect on, but does not dominate, CDF.

2.4.2.2.5 HEP Screenine Values. As indicated previously, a number human actions were determined to be relatively unimportant and were retained at the screening value. The submittal reports a sensitivity study in which these HEPs were increased by a factor of ten, which resulted in only a 1% increase in CDF. The submittal states that this sensitivity study was performed to assure that these events do not overwhelm the model. It is not clear to us how this study accomplishes that goal.

2.4.2.2.6 Use of ECCS Vent Valves. The ANO-2 plant does not have Power Operated Relief Valves (PORVs), but bleed-and-feed operations can be accomplished by using the ECCS vent valves, which are manually actuated. To examine the impact of these operator actions and the use of the vent valves on the CDF, a calculation was performed increasing the HEPs for the operator actions to 1.0. The operator actions involved and the nominal HEP values are as follows:

OPER 2 Operator fails to initiate once-through cooling, non-small-break-LOCA (4.5E-03)

OPER-2CHG Operator fails to initiate once-through cooling (2.lE-03)

OPER 3 Operator fails to initiate once-through cooling (2.2E-02)

OPER-8 (not identified in listing of HIPS) 32

Different timing requirements and other sequence-specific factors are associated with these different HEPs. The time available for OPER-2, OPER2-CHG, and OPER-3 is 45,20, and 120 minutes, respectively. There is no information provided on OPER-8. Increasing all of these HEPs to a value of 1.0 results in a 536% increase in CDF, to 2.09E-04/yr. Clearly these manual actions and use of the ECCS vent valves to provide once through cooling (bleed and feed) are important factors in the estimated CDF for ANO-2. This is not obvious from the listing of basic element importance values. OPER-2 ranks 24th in the list, and OPER-2CHG and OPER-3 are 74th and 75th, respectively. OPER-8 is not listed.

2.4.2.2.7 Imoact of Planned Alternate AC Power Source. An alternate AC power murce is planned for the future that will be capable of supplying power individually to any of the ANO-1 or ANO-2 4160V AC safety buses. This new power source will be air cooled and therefore completely independent of service water, and will have separate batteries so that it will not be dependent on DC power. It will be able to be aligned and started from the control room. To estimate the impact of this alternate power source, the recovery probabilities associated with restoration of offsite power, including human and equipment probabilities, were decreased by a factor of ten. The CDF was reduced 47% to 1.74E-05/yr. Interestingly, a final sensitivity study which combined the impact of the new alternate power source with the impact of reducing the offsite recovery probabilities to match NUREG-1150 (paragraph 2.4.2.2.4 above) resulted in a 43% decrease in CDF to 1.87E-05/yr, even though the two cases individually appeared to have essentially counterbalancing impacts. The licensee concludes that the addition of the altemate power source dominates any uncertainties associated with the recovery probability data.

2.4.2.3 Credit for Operator Recovery Actions. The submittal provides a summary listing (Table 3.5.4-5) of the top 50 cutsets with operator recovery actions not credited. (Note that in this context, we are using the licensee's terminology in which " recovery" action includes many planned response-type actions in the EOPs as well as alternative actions as we defined the term.) Without recovery actions, all but one of the top 50 sequences are TBF sequences.

With recovery actions, the majority of the top 50 cutsets are TBF sequences, but a number of I other sequence types occur - TQX, TBX, MX, SU, SX, AU. The overall CDF without  ;

recovery actions is 1.5E-02/yr compared to the nominal estimate of 3.4E-05/yr. This l sensitivity study affirms the importance of action in response to accident sequences in the ANO-2 study. Recall that the ANO-2 definition of " recovery action" is broader than usual and includes some actions that would be considered response-type actions in other PRAs.

Nonetheless, this estimate by,,the licensee is another indicator of the importance of credit for operator action in the ANO-2 analysis.

i 2.4.3 Human-Performance-Related Enhancements.

The submittal (Section 6.2) identified a number of plant improvements, including procedures enhancements, hardware modifications, and candidate severe accident management guidance.

The submittal states that none of the improvements are considered essential from an overall risk perspective. All are considered by the licensee to be notential improvements which will 33

be evaluated and dispositioned. No commitments are identified in the submittal. The procedures improvements and accident management candidates, which are more directly related to human performance, are summarized briefly below:

(1) Loss of Service Water (SW) Procedure - Loss of service water events are important contributors to CDF and release frequency. Without cooling ECCS components eventually overheat and fail. The poter,ial exists for service water recovery, but overheating of containment spray (CS) ud LPSI pumps may occur before SW recovery.

Potential improvements to the loss of SW procedure were identified which would secure one train of CS and LPSI eliminating unnecessary use and overheating and providing a

" backup" if one train fails.

(2) Shutdown Cooline System Procedure - An additional verification check is proposed to be added to the shutdown cooling system procedure to verify suction line isolation valves are closed once the system has been secured after startup. This check is intended to reduce the potential for an interfacing system loss of coolant accident (ISLOCA) through this path.

(3) Station Blackout Procedure - A potential exists that a small unisolated leak path can be established through the containment atmosphere system (CAMS) test connection during containment leak testing. This leak path could become significant if a station blackout (SBO) event occurred with the test unit in service. A change to the SBO procedure is proposed to close a manual valve which would eliminate this potential containment leak path.

l (4) Emercencv Feedwater Flow Control - As noted previously in Section 3.2.1, control of emergency feedwater flow (EFW) is an important operator action for  !

establishing shutdown cooling. Procedure changes implemented are intended to increase i assurance that the EFW discharge valves are manually controlled prior to a loss of DC )

power. This is to prevent interruption in the cooling of the RCS in the event that batteries are unable to supply power to open the DC discharge valves when they are i closed during modulating control. The specific change implemented required manual control of the EFW system for protracted station blackout events.

(5) Decraded Power Procedure - The potential exists during a degraded power condition for an unisolated leak path to develop from the containment through the 2" containment vent header line to the waste gas surge tank through two valves. The valves are normally open and are signaled to close by the containment isolation signal. In some degraded power events, the valves may remain open. An improvement to the degraded power procedure is proposed to manually close these valves or associated valves in series to help assure containment isolation.

(6) Fuel Transfer Tube Seal Protectiqn - The potential for a high temperature induced l failure of the Fuel Transfer Tube flange seals was identified as a concern for severe l l

34 t i I

t I l

accidents involving high pressure melt ejection events. Both hardware and procedure  !

modifications are being' considered as part of accident management strategy. One  ;

possibility is to flood the tube in the event of a core damage event. This would help cool-l the tube flange and seals, and in addition, should the seals fail, it would help scrub fission i products escaping through the failure.  !

i i

I 6

e I

i l

I l

l

\

j i

)

l l

4 i

i l

4 n

l 9

35 -

1 s

0

i i .

} 3. CONTRACTOR OBSERVATIONS AND CONCLUSIONS i The intent of the IPE is summarized in four specific objectives for the licensee identified in Generic Letter 88-20 and NUREG-1335:

1  !

(1) Develop an appreciation of severe accident behavior.

(2) Understand the most likely severe accident sequences that could occur at its plant.

4 l (3) Gain a more quantitative understanding of the ove all probability of core damage l and radioactive material releases.

(4) If necessary, reduce the overall probability of core damage and radioactive material  ;

l i release by appropriate modifications to procedures and hardware that would prevent or mitigate severe accidents. j i

j With specific regard to the HRA, these objectives could be restated as follows:

l

(1) Develop an overall appreciation of human performance in severe accidents; how i human actions can impact positively or negatively the course of severe accidents,

and what factors influence human performance.

} (2) Identify and understand the operator actions important to the most likely accident ,

! sequences and the impact of operator action in those sequences; understand how human actions affect or help determine which sequences are important. j i I (3) Gain a more quantitative understanding of the quantitative impact of human '

performance on the overall probability of core damage and radioactive material release.

(4) Identify potential vulnerabilities and enhancements, and if necessary/ appropriate, ,

implement reasonable human-performance related enhancements.

l The following observations and conclusions are pertinerit to NRC staff's determination of whether the licensee's submittal meets the intent of Generic Letter 88-20: .

I

1) The submittal and suphrting documentation indicates that utility personnel were  ;

involved in the HRA, and that the walkdowns and documentation reviews constituted a l viable process for confirming that the HRA portions of the IPE represent the as-built,

. as-operated plant. ,

I

2) The licensee performed an in-house peer review that provides some assurance that the i HRA techniques have been correctly applied and that documentation is accurate. l 36

3) The licensee's analysis of pre-initiator human actions was appropriate in scope in that it considered both calibration and restoration errors, and used a reasonable process to identify and select pre initiator errors to be included in the model.

4) The numerical screening process used for pre-initiator actions was, in our opinion, overly optimistic and therefore could have eliminated some important cutsets. The licensee reviewed their screening approach in response to an NRC RAI and concluded that it was appropriately conservative. It is not possible to determine from this document-only review whether the potential non-conservatism had a significant impact on overall results.

5) The quantification process used for the pre-initiator HRA was essentially a " generic" assessment with very limited plant-specific evaluation. This generic approach limits the opportunity for the licensee to identify and understand factors influencing human performance in pre-initiator events.

6) The post-initiator HRA addressed both response-type and recovery-type actions. The process for identification and selection of post initiator human actions included review of procedures and discussion with plant operations and training staff. A numerical screening value and sequence cutoff value were employed which were unlikely to have eliminated important actions / sequences. Based on these findings, it is our judgment that the post-initiator HRA employed a process for identification, selection, qualitative screening, quantitative screening that provided reasonable assurance that the important post-initiator actions were identified and included in the IPE model.

7) Quantification of post-initiator response-type actions followed employed primarily a methodology developed by the contractor, SAIC, that uses time reliability correlations based on published simulator data. The SAIC model has not been extensively reviewed and used by the HRA community. Information provided by the licensee in was provided a reasonable explanation of the methodology and its use in the ANO 2 analysis. The SAIC methodology includes more parameters than most time reliability correlations (TRCs), which can be used by the analyst to alter the estimated HEP based on judgment. These parametric inputs may also be useful for guiding the analyst's subjective plant specific evaluation of performance shaping factors. The primary factor determining the HEP is the relative time available for operator action, which, in the case of ANO-2 appears to have been determined in a reasonable, albeit subjective manner (i.e:, operator judgment). Evaluation of other performance shaping factors appears to have included some, but not especially detailed, plant-specific and compreheni.ve, and in general used appropriately conservative assumptions.

8) An exception to the statement above on use of " conservative" assumptions is the credit for independent error recovery by multiple other types of crew members in the " time-independent" model used for post-initiator slips. Credit for multiple, independent 37

' 7

recovery by four different crew members is in our view unrealistic. Because the number of HEPs affected is very small, the overall quantitative impact on IPE results probably is not significant.

i

9) Some post-initiator response type actions (HFEs) were included in fault trees, and l dependencies among multiple human actions in a cutset were not treated quantitatively. I In all cases, multiple human actions in a cutset were qualitatively assessed to have negligible dependence. This assumption may lead to some optimistic results. The licensee's assessment is that the impact is negligible. Independent assessment is not feasible within the scope of this document-only review.

l

10) Sequence-specific influences (primarily differences in available time) were accounted for on a case-by-case basis by designating a separate basic event for the same human j action appearing in different sequences. An example provided by the licensee indicates that this approach could, with some difficulty, be effectively used to account for sequence-specific variations. The information provided in the licensee's submittal materials is not detailed enough to determine the extent to which this approach actually accounted for all of the significant sequence-specific variations.

I1) The licensee identified operator actions important to risk using importance calculation (Fussel-Vesely) and sensitivity studies. Human actions appear to be generally of somewhat greater importance in the ANO-2 analysis than " typical" for PWR studies.

Six of ten most important basic events are human actions, and the overall contribution of credit for operator action is responsible for a reduction of approximately three orders of magnitude in CDF (compared to what the CDF would be if no credit were taken for operator action).

12) The licensee employed a systematic process to screen for vulnerabilities and identify potential enhancements. The process identified a number of human-performance-related (procedure) enhancements expected to reduce the likelihood of human error, and consequently reduce the estimated CDF.

4 38 l

q ,

/ 4. DATA

SUMMARY

SHEETS p Important Operator Actions / Errors:

The most important human actions identified (Fussel-Vesely importance value greater than or  ;

equal to 0.01) are:

DESIGNATOR - DESCRIPTION T.H E' liEP F-V RANKb MANOSPREC Opera: Ors fail to align offsite power to 2Al/2A2 EX l.3E-01 4.71E-01 1 OPER-1 Operator fails to trip RCPs in 30 minutes IN 6.0E 03 3.99E-01 2 OPER-15 Operator fails to realign 2D01 or 2D02 to 2D34 EX 7.9E-02 2.55E-01 4 ,

MANOSPRECD Operators fail to align offsite power to 2Al/2A2 EX 5.7E-02 1.74E-01 6 l ACREALIGN Operators fail to realign 2B5 from 2A3 to 2A4 EX 2.6E-01 1.41E-01 7 ACREALIGND Operators fail to realign 2B5 from 2A3 to 2A4 EX 7.9E-02 1.39E-01 8 OPER 16 Operator fails to realign 2D01 or 2D02 to 2D34 EX 7.9E-02 9.49E 02 12  :

P7AMANREC Operators fail to manually control EFW pump EX 2.0E-01 4.93E 02 15 QCSTKXFER Operators fail to align EFW suction to attemate -

ANO-1 condensate source EX 4.6E-02 3.61E-02 16 T7 REC Operator fails to stan standby service water pump IN 2.3E 02 2.31E 21 OPER-2 Operator fails to start once-through cooling within  ;

one hour (small break LOCA) IN 4.5E 03 1.82E-02 24 ACXTIE Operator fails to reduce loads & crosstie 2A3/2A4 EX 2.4E-01 1.63E-02 26 P7AMANRECD Operators fail to manually control EFW pump speed -

and discharge valves; after battery discharge EX 6.7E-02 1.50E-02 30 SWECPREC Operators fail to transfer to ECP on loss SW pump EX 1.7E-01 1.49E 02 31 ,

FOPERFWCSD Operators fail to prevent SG overfill EX 7.2E-03 1.26E 02 34  ;

QHF2 REFILL Operators fail to align EFW suction to attemate 1 condensate source EX 1.2E-02 1.lE-02 38 (a) " Type" refers to the model used to quantify the HEP: EX = ex-control-room TRC technique;IN =

in-control-room TRC technique.

(b) " Rank" refers to the relative ranking of a!]. basic events, including human error and equipment failure.

r Human-Performance Related Enhancements: 1 The following enhancements were identified from the IPE study:

39 l

l

6 i

(1) Loss of Service Water (SM Procedure - Potential improvements to the loss of SW procedure were identified which would secure one train of CS and LPSI eliminating unnecessary use and overheating and providing a " backup" if one train fails.

Shutdown Coolina System Procedure - An additional verification check is (2) proposed to be added to the shutdown cooling system procedure to verify suction line isolation valves are closed once the system has been secured after startup. This check is intended to reduce the potential for an interfacing system loss of coolant accident (ISLOCA) through this path.'

Station Blackout Procedure - A change to the SBO procedure is proposed to close (3) a manual valve which would eliminate one potential contamment leak path.

(4) Emergency Feedwater Flow Control - Procedure changes implemented are intended to increase assurance that the EFW discharge valves are manually controlled prior to a loss of DC power, which improves response to extended loss of power events.

Degraded Power Procedure - An improvement to the degraded v swer procedure is

.(5) proposed to manually close containment vent header valves or associateu valves in series to help assure containment isolation.

p 40

REFERENCES

1. A.D. Swain, " Accident Sequence Evaluation Program Human Reliability Analysis Procedure," NUREG/CR-4772, February,1987. -

1

2. A.D. Swain and Guttmann, H.E., " Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications, Final Report," NUREG/CR-1278F, August,1983.

3. E.M. Dougherty, Jr., and Fragola, J. R., Human Reliability Analysis: A Systems Engineering Approach With Nuclear Power Plant Applications, John Wiley & Sons, New l

York, NY,1988.

4. E.M. Dougherty, Jr., "An Ex-Control Room Human Reliability Model," Transactions of the 1989 Winter Meeting of the American Nuclear Society, TANSAO 601-792, November 28,189. (Referenced in the Submittal.)

5. NUREG/CR-2787, Vol. 2 (Cited by the licensee in response to the NRC RAI.)

6. NUMARC Document 91-04, " Severe Accident Issue Closure Guidelines," January,1992.

i 6

e f

41 1

5 b

l 1

• l l

l l

l l

APPENDIX C ARKANSAS NUCLEAR ONE, UNIT 2 TECHNICAL EVALUATION REPORT (BACK-END) i l

I

" I l

1 l

1

)

i i

l l

k I