ML20128F469

TER on IPE Submittal Human Reliability Analysis,Final Rept

ML20128F469
Person / Time
Site:	Beaver Valley
Issue date:	08/23/1995
From:	Swanson P CONCORD ASSOCIATES, INC.
To:	NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES)
Shared Package
ML20128F445	List: ML20128F441 ML20128F469 ML20129H177 ML20129H300
References
CON-NRC-04-91-069, CON-NRC-4-91-69 CA-TR-94-019-32, CA-TR-94-19-32, NUDOCS 9610080085
Download: ML20128F469 (37)

Text

.-

l l

CONCORD ASSOCIATES.INC. CAITR 94-019-32 Systems Penformance Engineers l BEAVER VALLEY POWER STATION, UNIT 1 l TECIINICAL EVALUATION REPORT ON THE l IPE SUBMITTAL l

IIUMAN RELIABILITY ANALYSIS 1 ,

l FINAL REPORT i

l l By l PJ. Swanson j

l Prepared for:

l U.S. Nuclear Regulatory Commission l OfTice of Nuclear Regulatory Research

! Division of Systems Technology l

Draft Repon November,1994 Final Repon August 23,1995

\

11915 Cheviot Drive 725 Pellissippi Parkway 6201 Picketts Lake Drive Herndon,VA 22070 Knoxville,TN 37932 Acwonh,GA 30101 (703)318-9262 (615) 675-0930 (404) 917-0690 9610000085 960930 PDR ADOCK 05000334 P PDR

l l

CA/TR-94-019-32 BEAVER VALLEY POWER STATION, UNIT 1 TECIINICAL EVALUATION REPORT ON THE '

IPE SUBMITTAL IIUMAN RELIABU.ITY ANALYSIS FINAL REPORT By:

P. J. Swanson Prepared for: .

1 U.S. Nuclear Regulatory Conunission Office of Nuclear Regulatory Research  ;

Division of Systems Technology ,

Draft Report November,1994 Final Repon August 23,1995 CONCORD ASSOCIATES. INC.

Systems Performance Engineers 725 Pellissippi Parkway Knoxville, TN 37932 Contract No. NRC-04-91-069 Task Order No. 32 l

l i

i TABLE OF CONTENTS 1

E. EXEC UTI V E S U M M A RY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 E.1 Plant Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . l 1

E.2 Licensee IPE Process . . . . . . . . . . . . ...................

2 E.3 Human Reliability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 E.3.1 Pre-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . .

2  :

E.3.2 Post-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . .

4 Generic Issues and CPI . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

E.4 Vulnerabilities and Plant Improvements . . . . . . . . . . . . . . . . . . . . . 4  !

E.5 5 l

- E.6 Observations .....................................

7 .

1. INTRODU CTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 l 1.1 HRA Review Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7 1.2 Plant Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

2. TECHNIC AL REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9 2.1 Licensee lPE Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . >

2.1.1 Completeness and Methodology ..................... 9 2.1.2 Multi-Unit Effects and As-Built, As-Operated Status . . . . . . . . 10 2.1.3 Licensee Participation and Peer Review . . . . . . . . . . . . . . . . 10 2.2 Pre-Initiator Human Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . I1 11 2.2.1 Pre-Initiator Human Actions Considered ................

2.2.2 Process for Identification and Selection of Pre-Initiator Human 12 A c d on s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2.3 Screening Process for Pre-Initiator Human Actions . . . . . . . . . . 13 2.2.4 Quantification of Pre-Initiator Human Actions . . . . . . . . . . . . . 13 13 2.3 Post-Initiator Human Actions ........................... 14 2.3.1 - Types of Post-Inidator Human Actions Considered . . . . . . . . . .

2.3.2 Process for Identificction and Selection of Post-Initiator Human 15 Acdons ....................................

2.3.3 Screening Process for Post-Initiator Response Actions . . . . . . . . 15 15 2.3.4 Quantification of Pst-Initiator Human Actions . . . . . . . . . . . .

2.3.4.1 Consideration of Plant-Specific Factors for Dynamic 18 (Response) Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.3.4.2 Consideration of Timing .................... 19 l 2.3.4.3 Consideration of Dependencies for Dynamic (Response) '

Acdons .................................... 20 2.3.4.4 Quantification of Electric Power Recovery Actions. . . . . 21 2.3.4.5 Treatment of Operator Actions in the Internal Flooding  !

22 Analy si s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.3.4.6 Sequences Screened Out Due to Credit for Recovery 22 Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.3.4.7 Treatment of Operator Actions in the Level 2 Analysis . . 22 2.3.4.8 GSI/USI and CPI Recommendations . . . . . . . . . . . . . 23 1

i 1

i.

Table of Contents (continued) 2.4 Vulnerabilities, Insights and Enhancements . . . . . . . . . . . . . . . . . . . 23  ;

2.4.1 Vulnerabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 -!

l 2.4.2 IPE Insights Related to Human Performance . . . . . . . . . . . . . . 23  !

2.4.3 Human-Related Enhancements ...................... 26 j

3. CONTRACTOR OBSERVATIONS AND CONCLUSIONS . . . . . . . . . . . . . . . . 27 l

4. DATA SUMM ARY SHEETS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 .

i i-REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31  !

i e

i i

I i

)

1 1

l i

t

(

I i i i i

]

i l 1

! . l j

I i

E. EXECUTIVE

SUMMARY

This Tecnnid Evaluation Report (TER) is a summary of the documentation-only review of the human reliability analysis (HRA) presented as part of the Beaver Valley Power Station Unit 1 (BVPSI) Individual Plant Examination (IPE) submittal from Duquesne Light Company (DLC) to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to assist NRC staff in their evaluation of the IPE and conclusions regard c whether the submittal meets the intent of Generic Letter 88-20.

E.1 Plant Characterization Beaver Valley is a two unit site; this review considers only one of two IPE's submitted, that for Unit 1. Unit 1 is a 3-loop Westinghouse pressurized water reactor (PWR) rated at 2660 MWT, with a pre-stressed concrete, steel lined, subatmospheric containment. No systems are shared between Units 1 and 2. Commercial operation began in 1976.

Similar units in operation are North Anna and Surry. Important operator actions associated with distinctive design features cited by the front-end reviewer include: (1) No need for operator actions to switchover ECCS from injection to recirculation, (2) Operator actions to use outside recirculation spray for long term core cooling, (3) Operator action to restore offsite power to IE buses if fast transfer to system station service transformers fail after plant trip, and (4) Operator actions to provide compensatory cooling for the emergency switchgear rooms.

E.2 Licensee IPE Process i

The BVPSI IPE was a Level 2 PRA and considered operator actions in the Level 1 l analysis only. The HRA process considered both pre-initiator (performed during test, maintenance and surveillance) and post-initiator operator actions (performed as part of the response to an accident). A limited set of pre-initiator actions (both miscalibration errors and restoration errors following test or maintenance) were examined and screened out from further consideration in the formal HRA. Post-initiator actions include both response (referred to in the Submittal as " dynamic" actions) and recovery type actions.

Utility personnel appear to have been appropriately involved in the development and I

application of PRA/HRA techniques to their facility, and associated walkdowns and documentation reviews constituted a viable process for confirming that the IPE represents f the as-built and as-operated plant. The licensee performed an in-house peer review that provided some assurance that the IPE analytic techniques had been correctly applied and documentation is accurate.

Operator actions were selected for quantification by reviewing the plant event sequence diagrams (ESDs) and event trees to identify operator actions that impact plant risk, and that this process generally followed the methodology outlined in steps 1 and 2 of the Systematic Human Action Reliability Procedure (SHARP). The primary methodology used to evaluate post-initiator actions was the PLG adaptation of the " Success Likelihood Index Methodology (SLIM)" developed by Brookhaven National Laboratory for the NRC.

1 J

Electrical recoveij actions were treated separately using the PLG STADIC4 code. The post-initiator actions quantified and included in the IPE model were compared with and found to be generally similar to the response actions addressed in the NUREG-1150 Surry study and other PWR IPEs reviewed previously. All of the actions identified by the NRC front-end reviewers as important were addressed by the licensee.

E.3 Human Reliability Analysis E.3.1 Pre-Initiator Human Actions.

The licensee states that they did not systematically address pre-initiator human errors in the PRA models or component failure data. However, both miscalibration and restoration type pre-imitators were given consideration during the process of pre-screening for selection. The licensee's approach regarding miscalibration errors was to compare BVPSI plant experience with those miscalibration errors judged by the HRA Analyst to be most important in the NUREG-1150 study for Surry. In this process the licensee evaluated only one miscalibration error, that being RWST level channels. The licensee states that because of BVPSl's staggered instrument calibrations, and independent verification and restoration checks performed after maintenance, test, and calibration procedures, it was decided that the likelihood of any such errors would be small when compared to the total failure of a single train. Subsequently, no miscalibration errors were included in the models. Restoration type pre-initiator errors were also excluded based on similar rationale. Errors which could impact multiple trains were not considered because they were already treated under common cause failures. l Although the arguments posed by the licensee for not including pre-initiator errors in PRA models are not uncommon, and not without some merit, the absence of a systematic structure in their assessment gives rise to a possibility that important human errors could be overlooked. The HRA could have been strengthened by inclusion of a reasonably rigorous process to identify potential pre-initiator human error contribution to system unavailability. We feel the limited nature of the licensee's treatment of pre-initiator human actions is a significant limitation and may have deprived the licensee of valuable insights on the contribution which maintenance, test, and surveillance can have on CDF.

E.3.2 Post-Initiator Human Actions.

The HRA addressed both response and recovery actions. Operator actions were selected for quantification by reviewing the plant event sequence diagrams (ESDs) and event trees to identify operator actions that impact plant risk. The actions quantified and included in the IPE model were compared to actions addressed in the NUREG-1150 Surry analysis and in other PWR IPEs reviewed previously. This comparison indicated that BVPSI is consistent with response actions addressed in other PWR PRAs. Also, each action i:'entified as important by the NRC front-end reviewers were addressed in the Submittal.

Bar a on our review, it appears the licensee had a reasonable process for identifying post-iniottor human actions which are important to risk. No numerical screening of 2

post-initiator human actions was performed. Actions were selected for quantification from qualitative analysis using the PLG SLIM-based process. In the SLIM method, groups of " expert judges" perform the assessment for human actions identified. Although multiple groups are recommended under SLIM guidelines, the exact number and composition is left up to the user and BVPS1 used only one group. The uncertainties for the rating estimates are to some degree dependent on the number of rating groups participating and the expertise within those groups. We believe the HRA would be stronger if the licensee used more than one group of experts and finalized on values representing a consensus of the rating groups. The relatively high number of operator actions, about one third, were assigned values by the IPE/HRA analyst based upon being similar to those rated by the group, or similar actions to those analyzed in the Unit 2 PRA. Both BVPS Units 1 and 2 have EOPs based on Westinghouse Owners Group Emergency Response Guidelines, and have similar actions between the Units in response to an accident.

The method for treating the plant-specific effects of performance shaping factors follows the SLIM methodology and appears to have been implemented properly. The seven PSFs used in BVPSl's HRA are reasonable and consistent with PSFs found treated in other accepted PSAs. The SLIM method emphasizes the assessment of performance shaping facters in the context in which the action takes place in the plant. The operator action descriptions provided as input to the expert raters provides the information available from the systems analysis such as that regarding the time window available. BVPSl's basis for detail:6 scenario information was obtained from a combination of thermal hydraulic analysis, plant data, limited simulator exercises, judgment, and previous PRA experience.

Although the licensee did not perform specific plant walk-downs to assess adequacy of time to accomplish task, some simulator exercises were conducted. Plant-specific and event / sequence-specific factors are inherently addressed by the raters. The performance shaping factor, "Significant Preceding and Concurrent Actions" addresses directly concerns about the dependency of failure of one task on preceding and concurrent tasks.

Sequence-specific analysis was performed where tree top events were subject to change and the HEPs were adjusted accordingly. Two operator actions were identified by the licensee as being most significant, the operators setup of portable fans to cool Emergency Switchgear and restoration of electric power, given SBO with AFW, no LOCA.

Recovery actions credited included two types, those dealing with process system failures I

and electric power system failures. Recovery from system failures (not electric power) were quantified using the same methodology as response actions. The STADIC4 l computer code (PLG proprietary) was used to develop the calculational model to analyze the electric power system hardware and the operators' actions to restore AC power following a loss of offsi'.e power initiating event. The STADIC4 Monte Carlo code runs  !

a time-integrated model for failures and recovery actions which assesses the effect of l diverse failure causes and models corresponding responses started at different times after event initiation that require different amounts of time to accomplish. Power recovery considers a combination of recovery of diesel generator power and recovery of offsite i

3

power. The licensee's treatment of these recoveries is fairly straight forward and appears consistent with excepted practices used in other IPEs.

BVPSI reran the base case plant models with all human actions evaluated to tre base case error rate or 0.1, whichever is higher, to identify those human actions which were below the 1.0E-07 cutoff because of the adjustments made to these errors in the quantification process. The total number of new sequences falling above the 1.0E-07 cutoff after rerunning the analysis were 130. The licensee stated that the key lesson learned from the sensitivity case evaluation was that the new sequences primarily involve failures of two or more actions.

E.4 Generic Issues and CPI The licensee's consideration of generic safety issues (GSIs) and unresolved safety issues i

(USIs) and of containment performance improvements (CPI) recommendations are the subject of the front-end review, and back-end review, respectively. The Submittal does not propose to resolve any issues besides Decay Heat Removal (DHR) directly with the IPE. The Submittal concludes that there are no vulnerabilities in the associated systems 1 and options used for DHR. Operator actions are associated with notable features that l directly impact the availability to provide DHR are, (1) ability to use outside recirculation j spray pumps for long term core cooling, (2) requirement to provide ventilation to the I

emergency switchgear rooms, and (3) requirement to transfer offsite power from the unit station service transformers to the system station service transformers following plant trip.

No notable CPI related issues rated to HRA were identified by the back-end reviewer.

E.5 Vulnerabilities and Plant Improvements The submittal did not contain a specific definition of " vulnerability", nor did it contain a l description of a systematic process or criteria for identifying vulnerabilities. However, j the licensee did list several insights derived from the IPE and noted eight potential l enhancements for further assessment. Four of these enhancements relate to human performance and potential human-related plant improvements. Improvements related to human actions include:

1) Loss of Emergency Switchgear Room HVAC - alarm response procedures that inform the operators to investigate the cause of trouble are being reviewed to determine if they can provide more explicit guidance on how to establish sufficient alternate cooling in the event that both emergency switchgear ventilation fan trains fail.

2) Fast 4,160 V Bus Transfer - explicit procedures and training on how to promptly repair or change out the failed breakers is being pursued.

3) Battery Capacity for Steam Generator 1.evel Instruments - consideration is being given to improving the limit on recovery time by providing more explicit guidance 4

- . - - -. - . - . - - . - - - - - _ - - - . - . _ - . - ~ - ..- - -

I

_ on battery load shedding or by providing for some means of charging a battery l: during a loss of all AC power. ,

i

4) Reactor Trip Breaker Failure - should both reactor trip breakers mechanically bind, consideration is being given to removing power from the bus with l

l procedural guidance.

L E.6 Observations ,

The following observations from our document-only review are pertinent to NRC's l determination of whether the licensee's submittal meets the intent of Generic letter 88-20: .i

1) Utility personnel were appropriately involved in the development and application of PRA/HRA techniques to.their facility, and associated walkdowns and documentation reviews constituted a viable process for confirming that the IPE represents the as-built and as-operated plant. The licensee performed an in-house peer review that provided some assurance that the IPE analytic techniques had been correctly applied and documentation is accurate. The HRA may have been strengthened if participation included other areas of plant support, particularly maintenance depanment personnel. j l

2)' The IPE/HRA process involved very limited consideration of pre-initiator human i action. The lack of a systematic process for identification of pre-initiator human actions for inclusion in PRA models gives rise to a possibility that important human errors could be overlooked. The HRA could have been strengthened by the licensee's i use of a reasonably rigorous proc:ess to identify potential pre-initiator human error contribution to system unavailability. The limited nature of the licensee's treatment of pre-initiator human actions may have deprived the licensee of valuable insights on the contribution which maintenance, test, and surveillance can have on CDF.

3) The licensee's HRA considered human events that are needed to prevent an accident as well to mitigate the consequences of an accident. Both response type actions and recovery type actions were addressed. The process used by the licensee to identify and select the post-initiator human events included, (a) review of plant procedures associated with the accident sequences delineated and review of systems modeled in the IPE, and (b) discussions with appropriate plant personnel on the interpretation and implementation of plant procedures. No numerical screening pre ess was employed.

All operator actions identified as significant were quantified with Mominal" values.

The licensee's treatment of time considered both the time available end the time required for the human action. - The licensee's departure from recc,inmended SLIM practice to use multiple groups of raters in the evaluation of plant-specific factors i

! influencing human performance in response actions creates a potential for biased or skewed results.

4) The licensee did not define vulnerability. However, a number of meaningful  !

enhancements to address " vulnerabilities" cited by the licensee were identified for i

! 5 1

l

t l

l '

implemenation or further evaluation. All HRA pertinent enhancements involve l procedure modification and/or improved training.

1 5

9 J

l 1

l 4

l I

6

1

1. INTRODUCTION This Technical Evaluation Report (TER) is a sommary of the documentation-only review of the human reliability analysis (HRA) presented as part of the Beaver Valley Power Station Unit 1 (BVPSI) Individual Plant Examination (IPE) submittal from Duquesne Light Company (DLC) to the U.S. Nuclear Regulatory Commission (NRC). The review was performed to  ;

. assist NRC staff in their evaluation of the IPE and conclusions regarding whether the submittal meets the intent of Generic 12tter 88-20.

1.1 IIRA Review Process The HRA review was a " document-only" process which consisted of essentially four steps:

(1) Comprehensive review of the IPE submittal focusing on all information pertinent to HRA.

(2) Preparation of a draft TER summarizing preliminary findings and conclusions, noting specific issues for which additional information was needed from the licensee, and formulating requests to the licensee for the necessary additional information.

(3) Review of preliminary findings, conclusions and proposed requests for additional information (RAls) with NRC staff and with " front-end" and "back-end" reviewers.

1 (4) Review of licensee responses to the NRC requests for additional information, and preparation of this final TER modifying the draft to incorporate results of the additional information provided by the licensee.

Findings and conclusions are limited to those that could be supported by the document-only review. No visit to the site was conducted. No review of detailed " Tier 2" information was performed, except for selected details provided by the licensee in direct response to NRC's request for additional information (RAls). In general it was not possible, and it was not the intent of the review, to reproduce results or verify in detail the licensee's HRA quantification process.

1.2 Plant Characterization Beaver Valley is a two unit site; this review considers only one of two IPE's submitted, that for Unit 1. Unit 1 is a 3-loop Westinghouse pressurized water reactor (PWR) with a pre-stressed concrete, steel lined, subatmospheric containment. The unit is rated at 2660 MWt and 835 MWe (net). Commercial operation began in 1976. Similar units in operation are North Anna and Surry. Table 1.2-1 provides a listing of the distinctive design features cited by the front-end reviewer as having impact to CDF. Also included in the listing are important operator actions.

7

Table 1.2-1 Design Features and Associated Operator Actions Considered Important to CDF FEATURE OPERATOR ACTION IMPACT ON CDF Dedicated feed water pump powered N/A Reduction off the emergency response facihty DG and an Appendix R backup for auxiliary feedwater Automatic switchover of ECCS from No need for operator actions Reduction injection to recirculation Ability to use outside recirculation Operator actions to use Reduction spray pt mps for long term core cooling outside recirculation spray for long term core cooling Operation with 2 of 3 PORV block valves N/A Increase closed l

Inability to cross tie IE power between N/A Increase units I and 2 Note: IPE enhancement has established this action, but it was not credited in the I analysis and no HRA performed I

Requirement to transfer offsite power from Operator action to restore Increase the unit station service transformers to offsite power to IE buses if the system station service transformers fast transfer to system station I following plant trip service transformers fails after plant trip Requirement to provide ventilation to the Operator actions to provide Increase l emergency switchgear rooms compensatory cooling for the  !

emergency switchgear rooms I l

Design to stay on-line following load N/A Increase )

rejection results in increased chance of PORV opening and failing to reclose. 1 Two operator actions identified by the licensee as being most significant include the operators setup of portable fans to cool Emergency Switchgear and restoration of electric power, given SBO with AFW, no LOCA. These are consistent with the findings from the front-end review. Details on these actions and others found to be important in HRA may be found in Section 2.4.2 of this report.

8

2. TECHNICAL REVIEW 2.1 Licensee IPE Process The BVPS HRA process addressed both pre-initiator (performed during test, maintenance and surveillance) and post-initiator operator actions (performed as part of the response to an accident). A limited set of pre-initiator actions (miscalibration errors and restoration errors following test or maintenance) were examined and screened out from further consideration in the formal HRA. Post-initiator actions include both response (referred to as " dynamic" actions) and recovery type actions. Operator actions were selected for quantification by reviewing the plant event sequence diagrams (ESDs) and event trees to identify operator actions that impact plant risk, and that this process generally followed the methodology outlined in etcps 1 and 2 of the Systematic Human Action Reliability Procedure (SHARP)

(Ref.1). The primary methodology used to evaluate post-initiator actions was the " Success Likel hood index Methodology (SLIM)" developed by Brookhaven National Laboratory for the NRC (Ref. 2). The post-initiator actions quantified and included in the IPE model were compared with and found to be generally similar to the response actions addressed in the NUREG-1150 Surry study and other PWR IPEs reviewed previously. All of the actions identified by the NRC front-end reviewers as imponant were addressed by the licensee.

2.1.1 Completeness and Methodoloev.

The SLIM methodo!ogy used for the HRA was an adaptation developed by Pickard, Lowe and Garrick, Inc. (PLG). Development of the HRA was performed by DLC, Engineering and Analysis and Assurance (EAA) group with limited support from PLG. Technical 4 I

expenise of the DLC group was gained during their involvement in the development of the IPE for Beaver Valley Unit 2. For the most part, the BVPSI HRA is a reasonably thorough and comprehensive analysis. One notable limitation in the licensee's approach is that the IPE submittal did not systematically address pre-initiator human errors in the PRA models or component failure data. We view this as a significant limitation in the licensee's HRA.

PLG's SLIM-based methodology was used to assess both response and recovery post-initiator  ;

i human errors. Electrical recovery actions were treated separately using the PLG STADIC4 code. Operator actions accounted for in the flooding analysis were treated using the same SLIM-based process as response actions.

The " baseline plant configuration" date for the IPE is identified as early 1988. The submittal notes that the BVPS1 configuration management program provides assurance that the PRA ,

represents the as-built, as-operated plant. Additionally, plant modifications and procedure changes were reviewed for incorporation in the IPE model. The submittal provides a listing of plant-specific IPE information sources that includes sources typical of previous IPEs and other PSAs. Included in the HRA were procedures for surveillance, and normal, abnormal and emergency operations. BVPS1 made use of the Beaver Valley training simulator to support the HRA process through the assessment of a limited number of operator actions.

Also, selected results from the PRA performed for Unit 2 were used on Unit 1. The PRA team reviewed the Surry, NUREG-1150, PRA as part of the information assembly. Surry is 9

very similar in design to Beaver Valley. Additionally the PRAs for Zion, Indian Point and Millstone were reviewed for insights.

2.1.2 Multi-Unit Effects and As-Built. As-Operated Status Beaver Valley is a two unit site, but due to the considerable time frame between completion of Unit I and Unit t no systems are shared. Independent IPEs were performed for each unit.

The Unit 1 IPE benefitted from the initial IPE performed on Unit 2 in that the utility personnel involved with the analysis on Unit I benefitted from the experience gained on Unit 2.

The DLC PRA team was located at the plant and participated in plant walk-throughs and inspection on a regular basis. Three plant walk-throughs are summarized in the submittal: (1) a plant familiarization tour; (2) an internal flood analysis walk-through; and, (3) a containment walk-through for the back-end analysis. The DLC PRA team appears to have been adequately represented along with the supporting contractor in the plant walk-throughs.

System walk-throughs to assess operator consideration such as equipment accessibility, typical of that found in other IPEs, were not discussed. i i

Finally, the subjective SLIM-based process used for evaluation of " dynamic" human actions l (i.e., actions in response to an initiating event) included significant direct involvement of  !

operators and others knowledgeable of plant operations in the evaluation and quantification of human error for these types of actions. These individuals served as " subject matter experts" providing the ratings from which the human error probabilities included in the IPE model were derived.

Overall, the submittal documentation indicates that the licensee took steps to provide reasonable assurance that the HRA-related aspects of the IPE model represented the as-built, as-operated plant at the time of the cutoff date of December 31,1989.

2.1.3 Licensee Particination and Peer Review. i I

The NRC review of the submittal attempts to determine whether the utility personnel were involved in the development and application of PRA techniques to their facility, and that the associated walkdowns and documentation reviews constituted a viable process for l confirming that the IPE represents the as-built and as-operated plant.

The submittal discussion of the organization of the IPE program indicates significant involvement of utility personnel - the DLC Nuclear Engineering Depanment, and to a lesser degree, site personnel - in the development of the IPE. The DLC Director, Engineering Analysis and Assurance (EAA) Group had overall responsibility for the PRA program. Six individuals from various departments at DLC were assigned to the EAA group and given responsibility for supplying engineering and operational support in the development and review of the PRA (IPE). The EAA group was primarily involved with thermal hydraulic (transient and LOCA) analysis, fuel analysis, and 10CFR50.59 safety 10

i l evaluations. Two members of the group had been through the operator training program at Beaver Valley.

The submittal discusses considerable operations and training staff involvement in the development of the HRA. However, as would be expected with the limited treatment of pre-initiator human error, no mention is made of any maintenance department personnel .

involvement in HRA development. l l

l The submittal (Section 5.2) indicates that the guidance provided in NUREG-1335 regarding an independent in-house review was addressed by assembling an review team of personnel from departments that would particularly provide knowledge of plant design, system configuration and operating procedures. It appears that the independent review included an appropriate level of involvement by company management, engineering and operation. Specifically mentioned are personnel from the STA group, engineering, licensing, operations, t:aining, testing and plant performance, radiological engineering and ISEG. The HRA portion of the submittal received independent review by licensing, operations, training and ISEG.

The licensee states that the main accomplishments of the DLC independent review were a verification that the models reflect the actual plant design and operation and a development of a PRA awareness and knowledge throughout the DLC organization. It is stated that particular attention was given to system descriptions, system analysis and system dependencies. The submittal provides only limited information on findings, comments and changes which were incorporated as a result of the independent review.

One example, is the improved and more accurate modeling of the DC power systems.

In our opinion, the reviews appear to constitute a reasonable process for an "in-house" peer review that provides some assurance that the IPE analytic techniques were correctly l applied and that documentation is accurate.

2.2 Pre-Initiator Human Actions Errors in performance of pre-initiator human actions (i.e., actions performed during maintenance, testing, etc.) may cause components, trains, or entire systems to be unavailable on demand during an accident, and thus may significantly impact plant risk.

Our review of the HRA portion of the IPE examines the licensee's HRA process to determine what consideration was given to pre-initiator human actions, how potential actions were identified, the effectiveness of quantitative and/or qualitative screening process (es) employed, and the processes for accounting for plant-specific performance shaping factors, recovery factors, and dependencies among multiple actions.

2.2.1 Pre-Initiator Human Actions Considered.

l In general, we feel the licensee's treatment of pre-initiators, and/or lack thereof, is a l

! significant limitation in the BVPSI IPE. Reference is made to misalignment (restoration) l 11

errors in Section 3.3.3.1 of the IPE, but the assessment is performed under systems analysis and HRA details are very limited. There is no mention of specific consideration which may have been given miscalibration errors.

The submittal states that pre-initiator errors involving the restoration of a component or flow path to normal after the completion of testing, inspection, or maintenance activities were considered. These actions are described in the submittal as being system-specific activities, routinely performed by one or more individuals as part of his normal workday duties. The Submittal states that "'1hese routine testing, maintenance, and surveillance actions are evaluated separately and incorporated into each system analysis as specific causes for equipment inorperability." To better understand how the licensee addressed these actions additional information was requested. The licensee states in their response that they did not systematically address pre-initiator human errors in the PRA models or component failure data. However, both miscalibration and restoration type pre-imitators were given consideration during the process of pre-screening for selection. j 2.2.2 Process for Identi6 cation and Selection of Pre-Initiator Human Actions.

The key concerns of the NRC staff review regarding the process for identification and selection of pre-initiator human events are: (a) whether maintenance, tesi and calibration procedures for the systems and components modeled were reviewed by the systems analyst (s), and (b) whether discussions were held with appropria,te plant personnel (e.g.,

maintenance, training, operations) on the interpretation and implernentation of the plant's test, maintenance and calibration procedures to identify and understand the specific actions and the specific components manipulated when performing the maintenance, test, or calibration tasks.

The licensee's approach regarding miscalibration errors was to compare BVPSI plant experience with those miscalibration errors judged by the HRA Analyst to be most important in the NUREG-1150 study for Surry. In this process the licensee evaluated only one miscalibration error, that being RWST level channels. The licensee states that because of BVPS1's staggered instrument calibrations, and independent verification and restoration checks performed after maintenance, test, and calibration procedures, it was decided that the likelihood of any such errors would be small when compared to the total failure of a single train. Subsequently, no miscalibration errors were included in the models. Restoration type pre-initiator errors were also excluded based on similar rationale. Errors which could impact multiple trains were not considered because they were already treated under common cause failures.

Although the arguments posed by the licensee for not including pre-initiator errors in PRA models are not uncommon, and not without some merit, the absence of a systematic structure in their assessment gives rise to a possibility that important human errors could be overlooked. The HRA could have been strengthened by inclusion of a reasonably rigorous process to identify potential pre-initiator human error contribution to system unavailability. We feel the limited nature of the licensee's treatment of pre-initiator 12

human actions may have deprived the licensee of valuable insights on the contribution which maintenance, test, and surveillance can have on CDF.

For example, review of the BVPS1 baseline data, specifically the historical data for Unit I trip events, Table 3.3.1.6 (submittal pages 3.3.25 - 28), suggests that approximately 25% of the underlying causes can be attributed to testing / maintenance practices, some examples are provided in Table 2.2-1. It is recognized that plant trip data is accounted for in appropriate initiating event frequencies for transients. However, in light of the plant's experience with testing and maintenance errors contributing to significant number of unit trips, it is surprising that a more systematic and probing process for evaluating these types of errors was not performed. The licensee's decision not to include pre-initiator actions in the PRA models is viewed as a significant limitation in the BVPSI IPE.

Table 2.2-1, Excerpts from Beaver Valley Unit 1 Trip Reports Event No. Descriotion 1

Inadvertent safety injection signals during SSPS maintenance l 1

Rx trip due to spike from source range detector while fuse was being  !

8 '

replaced 29 Low level S-G IC - troubleshooting caused bypass FW reg. valve to close Turbine trip du-ing pedestal check (

44 Lcw pressurizer pressure trip due to error during testing l 49 67 Human error during surveillance caused Rx trip breaker to open 74 Low lever in S-G 1 A during startup due to FWI signal not being reset 2.2.3 Screenine Process for Pre-Initiator Human Actions.

There were no pre-initiator human actions include in the BVPSI PRA models.

2.2.4 Ouantification of Pre-Initiator Human Actions.

There were no pre-initiator human actions quantified for inclusion in the PRA models.

2.3 Post-Initiator Human Actions Human errors in responding to an accident initiator, e.g., by not recognizing and diagnosing the situation properly, or failure to perform required activities as directed by procedures, can have a significant effect on plant risk. These errors are referred to as post-initiator human errors. Our review assesses the types of post-initiator errors considered by the licensee, and evaluates the processes used to identify and select, screen, and quantify post-initiator errors, including issues such as the means for evaluating 13

timing, dependesy among human actions, and other plant-specific performance shaping factors.

2.3.1 Tvres of Post 'nitiator Human Actions Considered.

There are two important types of post-initiator actions considered in most nuclear plant PRAs: (1) response actions, which are performed in response to the first level directives of the emergency operating procedures / instructions (EOPs, or EOIs); and, (2) recovery actions, which are performed to recover a specific failure or fault, e.g., recovery of offsite power or recovery of a front-line safety system that was unavailable on demand earlier in the event.

The Beaver Valley HRA addressed both response and recovery actions. The submittal uses the term " dynamic", rather than " response" actions. Dynamic actions are described as scenario-specific, mission-directed activities which are an integral part of the plant response to an initiating event. The operators must accomplish well-defined tasks for manual initiation, control and alignment of plant emergency equipment or selected backup systems. These tasks are generally guided by the plant emergency response procedures.

Recovery actions, as defined in the submittal, also can be classified as mission-directed activity, but they involve recovery from unexpected failures that completely or partially disable automatic system response during a plant transient. The submittal notes that actions classified as recovery actions in the analysis may not be as well documented in the plant emergency response procedures (emergency or abnormal procedures). In explaining this statement, the submittal notes that the actions are proceduralized, though not necessarily in the EOPs. Recovery from electrical system failur.:s were treated differently from other system-type failures and are discussed separately in Section 2.3.4.4 of this report.

The submittal notes that another reason for categorizing an action as a recovery action j may be simply that it is a dynamic action that is added to a " risk sensitive" sequence after quantification of the model. This permits " detailed analyses" for specific recovery actions in "very specialized plant response event sequences" without undue expenditure of time or  ;

resources (presumably, compared to the effort that would be required if this detailed and specialized analysis were performed for all dynamic actions). However, the analysis of recovery actions in most PSAs, including Beaver Valley, typically is not more " detailed and specialized" than the analysis performed for the dynamic actions. In fact, many PSAs use relatively high HEPs typical of screening values because of the greater uncertainty associated with estimating the likelihood of successful recovery actions.

With regard to incorporation of recovery actions in the BVPSI IPE model, the submittal indicates that some dynamic and recovery actions were incorporated into the system model (fault trees), and some were quantified separately and became part of the decision points c; split fractions in the plant response tree model.

14

l l

2.3.2 Process for Identification and Selection of Post-Initiator Human Actions.

The primary thrust of our review related to this question is to assure that the process used by the licensee to identify and select post-initiator actions is systematic and thorough enough to provide reasonable assurance that important actions were not inappropriately precluded from examination. Key issues are whether: (1) the process included review of plant procedures (e.g., emergency / abnormal operating procedures or system instructions) associated with the accident sequences delineated and the systems modeled; and, (2) discussions were held with appropriate, plant personnel (e.g., operators or training staff) on the interpretation and implementation of plant procedures to identify and understand the specific actions and the specific components manipulated when responding to the accident sequences modeled.

The submittal notes that operator actions were selected for quantification by reviewing the plant event sequence diagrams (ESDs) and event trees to iden'ify operator actions that impact plant risk, and that this process generally followed the methodology outlined in steps 1 and 2 of the Systematic Human Action Reliability Procedure (SHARP) (Ref. 3).

Those two steps in the SHARP process are " Definition" and " Screening", though it does not appear that any numerical screening was performed. The ESDs are pictorial representations (flow charts) of the sequence derived by the PRA analysts from review of procedures, plant documentation and discussion with personnel knowledgeable of plant operations. They are an interim product to, and they support development of, the event trees. To some extent, then, there is some screening of potential operator actions by the PRA analysts as the ESDs are developed. However, we assume that review and feedback from the HRA process contributed to and modified the ESDs.

The dynamic actions quantified and included in the IPE model were compared to actions addressed in the NUREG-1150 Surry analysis and in other PWR IPEs reviewed previously. This comparison indicated that the dynamic actions quantified in the Beaver Valley IPE are generally consistent with response actions addressed in other PSAs. Also, each of the actions identified as important by the NRC front-end reviewers were addressed in the submittal. Based on our review, it appears the licensee had a reasonable process for identifying post-initiator human actions which are important to risk.

2.3.3 Screenine Process for Post-Initiator Resoonse Actions.

l No numerical screening of post-initiator human actions was performed. Actions were selected for quantification from qualitative analysis using the PLG SLIM-based process.

2.3.4 Ouantification of Post-Initiator Human Actions.

Two members of the Beaver Valley Unit 1 plant operation staff (one SRO and one RO) and two PRA/HRA analyst provided ratings. In the SLIM method, groups of " expert judges" perform the assessment for human actions identified. Although multiple groups are recommended under SLIM guidelines, the exact number and composition is left up to 15

the user and BVPS1 used only one group. Because of scheduling conflicts and time limitations, only those actions deemed most important by the analyst were assigned for review by the expert group. The expert group evaluated 41 out of the 77 human actions identified, approximately 68%. The remaining 32% were rated by the HRA analysts using similar previously evaluated actions by the group as a guide, or using the Beaver Valley Unit 2 HRA as a guide for similar actions, or both. Submittal Table 3.3.3-6 identifies which of the actions were reviewed by the expert group and which were assigned values by the analyst (s). We had difficulty identifying those actions classified as "similar" actions to those ranked by the BVPS1 expert group. The majority of actions assigned values by the analyst are believed to have been based upon BVPS2's PRA. Both BVPS Units 1 and 2 have EOPs based on Westinghouse Owners Group Emergency Response Guidelines, and have similar actions between the Units in response to an accident. The group provided ratings for PSFs for each action against two criteria: (1) the degree to which the PSF helps or hinders the operator in the performance of the action (scale of 0 'e 10); (2) the relative importance (weight) of each PSF on the likelihood of success of the action (rated high, medium, or low). The submittal provides blank samples of the sheets given to the evaluators that define each performance shaping factor (PSF) and provide guidance for scaling. Sample forms used to record results from l each rater, for each action were also provided in the submittal. The uncertainties for the l rating estimates are to some degree dependent on the number of rating groups participating and the expertise within those groups. We believe the HRA would be stronger if the licensee used more than one group of experts and finalized on values ,

representing a consensus of the raung groups. 1 In order to provide a manageable set of numerical variables that reasonably represent the variability in human performance, analysts using SLIM often group actions having similar PSF weights. In the Beaver Valley analysis, numerical weights of 10, 5, and 0 were assigned to high, medium and low ratings, respectively, and these weights were then normalized to sum to 1 for each evaluated human action. The normalized PSF weights are used in the computation of the success likelihood index, or in the PLG methodology, the failure likelihood index (FLI). Eleven action groups were selected for the BVPS1 analysis and the following examples were provided:

• Actions for which training and plant indications dominate, such as manual control of plant parameters.

• Actions for which time and preceding actions are most important

• Actions for which stress, procedures, and preceding actions dominate.

• Recovery actions for which the training and experience of the operators to diagnose the problem and choose an acceptable course of action; the complexity of the action and time available are important.

An important consideration in the quantification of human error using SLIM is the selection of calibration tasks. The submittal notes that the calibration tasks must be influenced by PSFs with the same relative weights as the group of actions and have known or accepted values of the human error rate. If PSF ratings are available for the calibration task, they 16

l l

l J

should be used in the quantification for each of the evaluator groups. If PSF ratings are not available for the uJibration task but the calibration task is found to be equivalent to one of the actions being evaluated, the PSF ratings given to the evaluated action by each evaluator group should be used for the calibration task in quantifying that evaluator group's actions. A '

calibration task must either have its own PSF ratings or be equivalent to an evaluated action.

The notion of identifying a task that is "similar" and then applying the PSF ratings from another task is somewhat problematic. In behavioral terms, what makes one task "similar" to another is, to a large degree, the various PSFs that are significant for both tasks. The PSFs, in effect, are the dimensions along which similarity is measured. If two tasks could be determined to be behaviorally similar/different by some other set of factors, then those factors should be considered as the basis for defining the degree of " similarity", and the basis for grouping tasks together; i.e., they should be included as PSPs. What is important to identify for HRA is the set of task characteristics that significantly affect the probability of success / failure. If the PSFs rated do that, then they define what will be a similar or different task and what will be a similar HEP.

The submittal provides a listing (Table 3.3.3-8) of the calibration tasks used for the BVPS1 analysis. They were selected from six different previous PSAs and EPRI/RSSMAP. The submittal does not state whether PSF ratings were available for the calibration tasks taken from the previous PLG reports, or whether they were simply judged by the analyst (s) to be "similar" and the Beaver Valley groups' ratings were then applied. We were not able to examine the five referenced PRAs performed by PLG to assess the basis (i.e., the calibration tasks) used in those studies. Since there is no comprehensive data base of HEPs from actual experience, it is still necessary to base HEP estimates essentially on " judgment." It is i important that this judgment represent, as best as possible, the cumulative judgment and consensus of the community and results from different HRA techniques and different HRA l analysts. Thus the selection of source data for calibration tasks is important. A concern is to l l

assure that initial subjective estimates by PLG analysts in early PRA studies' are not simply

" propagated" and gain inappropriate credibility by virtue of repeated use alone. While it is  !

beyond the scope of this document-only review, detailed information on the ultimate sources of data should be available for examination.

A significant feature of the SLIM process, which is emphasized in the submittal, is the treatment of operator actions in the context of the scenario in which they are embedded. The submittal emphasizes the importance of the qualitative evaluation performed prior to quantification to fully understand and document the scenario and sequence-specific factors that influence operator performance, likelihood of success, and the importance of the human action within the scenario. The important consequence of this emphasis is that the HRA team reviewed the scenarios in depth from the perspective of human performance issues, and did not simply quantify specific human actions identified by systems analysts. The qualitative analysis characterized the general scenario and plant, including success criteria for operator response. In some cases individual sequences with common features from the perspective of operator decisions and actions were grouped together. Factors affecting operator response time were identified. Significant preceding actions that may have dependency effects on the action of interest were identified. And a qualitative assessment of important factors relating 17

to procec : .s and to operator traimc3 and experience were noted. The discussion of PLG's SLIM metnodology states that where possible, operator actions should be observed in simulator training sessions, and that these observations provided the IPE analysts with an orientation and framework within which to evaluate plant-specific human actions by giving them a better sense of the timing and complexity associated with the selected human actions.

The licensee cited several simulator observations which were performed in support of the analysis.

Documentation of this qualitative analysis is prepared in the form of a human action

, description form that is used to provide a common basis for evaluation during the quantification by expert raters. A sample of one form is provided in submittal Table 3.3.3-4.

The submittal notes that these action descriptions were prepared by the IPE analyst, reviewed by the human action analyst, and reviewed by personnel with operation experience to ensure consistency with plant practices and nomenclature. Seventy-seven dynamic operator actions were considered for quantification. While much of the submittal discussion of this qualitative analysis is written as though the list of actions to be quantified was already determined and then the analysis was performed, the nature of the interaction between the HRA and the other IPE analysis is iterative. We assume that this analysis significantly influenced the selection of operator actions to be quantified.

2.3.4.1 Consideration of Plant-Soeci6c Factors for Dynamic (Response) Actions. Seven performance shaping factors were considered in the SLIM-based assessment. The basis for selection of these seven and elimination of other possible factors was not discussed in the submittal. Other HRAs performed by PLG as part of IPEs have used a similar set of PSFs with some variations. The seven included in the BVPSI assessment were:

1) Man-Machine Interface and Indications of CondilignL- relates the impact of the man-machine interface on the likelihood of success; measures degree to which the control room or the local conditions, at the time when the action must be accomplished, assist [or impede] the operator in performing the action.

2) Significant Preceding and Concurrent Actions -impact of preceding and concurrent actions in the same accident sequence set the stage for the modeled action and make it necessary and obvious to the operators; they can also divert operators' attention from this action or event cause failure. Lack of preceding actions may create a surprise effect that should be accounted for in the PSF.

3) Task Comolexity - rates the effect of multiple requirements on task success; may include such factors as coordination, multiple locations, remote operations, variety of tasks, communication requirements, and availability of resources.

4) Procedural Guidance - accounts for the extent to which plant procedures enhance the operators' ability to perform the action; e.g., procedures available, clear, definite, vague, misleading.

18

. t j

t

! 5) Trainine and Exoerigsg Jmeasure the effect of familiarity and confidence the j operators have about their actions. i l

, 6) Adeauacy of Time to Accomolish Action - measure of the time required to act compared with the time available to recognize, diagnose, and accomplish the action; i

judgment of the evaluators based on input provided in task descriptions. l

. i

7) Stress - accounts for situations that may endanger the operator, damage or l l contaminate either the plant or the environment, or result in a long plant outage; i depending on its level, stress can serve as an incentive to accomplish the action, produce a reluctance to act, or provide a diversion of attention that increases the l

likelihood of failure.

I j The methodology for treating the plant-specific effects of performance shaping factors follows

' the SLIM methodology, which is well documented and generally accepted by the HRA 4 community. In general, the method appears to have been implemented properly. Additional

, information on the basis for selection of the seven PSFs addressed probably should have been l provided in the submittal, but those seven are reasonable and consistent with PSFs found' j treated in other accepted PSAs. It appears that fairly general descriptive information was I prepared, and that the results therefore are highly dependent on the raters' knowledge of the j details of performing that action. Simulator exercises for a limited number of specific

operator actions provided some of the basis for operator judgment of timing.

! 2.3.4.2 Consideration of Timing. In some post-initiator operator actions, timing - time available vs. time required by the operators - is a critical determinant of likelihood of success. It is important to assure that the licensee's process for estimating both time available j and the time necessary for operators to complete the required actions takes into account

plant-specific conditions and provides realistic estimates. Plant-specific phenomenological i analysis (accident analysis computer codes) should be used to determine the available time.

Actual measures using currently licensed operators in realistic walk-throughs or control room j simulator exercises is a preferred approach for estimating expected /necessary operator response time. Especially for local actions outside of the control room, it is important to

assess time to get to the equipment, accessibility, possible impacts on timing of special

clothing or environmental factors, etc. Guidance in ASEP and THERP is that estimates L based on operator judgment alone should be multiplied by a factor of 2.

i

} Timing of operator actions is specifically addressed in the qualitative and quantitative analysis

, . performed in the BVPSI evaluation of post-initiator actions. The submittal notes that there is -

l a "relatively well-defined time window available for successful operator response." It also notes that timing determines important factors that influence the operators' ability to diagnose the problem, decide what actions are appropriate, and complete those actions within the  ;

4 required time window. ,

i

{ The operator action descriptions provided as input to the expert raters provides the l information available from the systems analysis regarding the time window available. The 19 t

i

estimated time available for each operator action is listed in the submittal (Table 3.3.3-5).

The basis for detailed scenario information was obtained frora a combination of thermal hydraulic analysis, plant data, limited simulator exercises, judgment, and previous PRA experience.

The expected time of operator response is not estimated directly. The likelihood of operators performing the required action within the available time is one of the performance shaping factors rated by the panel of experts. Thus, the impact of timing is accounted for in the SLIM process essentially by subjective evaluation, and is therefore subject to the biases and uncertainties (and approaches to eliminate or account for those biases and uncertainties) inherent in the subjective process employed. Some simulator exercises were performed to obtain data or to help " anchor" the subjective process.

2.3.4.3 Consideration of Devendencies for Dynamic (Response) Actions. An important concern in HRA is the determination of how the probability of success or failure on one task ,

l may be related to success or failure on another. Human behavior typically is highly dependent on the context in which the task is performed - saccess or failure on a preceding task, performance of other team members in parallel or related tasks, assumptions about the i expected level of performance of other team members based on past experience, and many other factors. The human error probability estimates for HRA are conditional probabilities.

If dependencies are not specifically accounted for, and HEPs are treated as independent, the probablistic combination of HEPs can lead to an unrealistically low estimate of human performance overall (i.e., of the joint human error probability), and to a significant underestimate of risk.

One of the advantages cited by some for the SLIM methodology is that it emphasizes the assessment of performance shaping factors in the context in which the action takes place in the plant. Thus plant-specific and event / sequence-specific factors are inherently addressed by the raters. The performance shaping factor, "Significant Preceding and Concurrent Actions" addresses directly concerns about the dependency of failure of one task on preceding and concurrent tasks. The sample action description sheet included in the submittal (Table 3.3.3-

4) is used by the rater in assessing dependency for preceding and concurrent actions. The associated scaling guidance does provide a set of verbal descriptors, or " anchors" for the knowledgeable operator who is thoroughly familiar with and consciously thinking of the '

context of the action to make a rating of the impact of previous and concurrent actions. The process applied by BVPSI appear reasonable for assuring that factors which needed to be considered to properly assess dependencies were addressed. Table 3.3.3-5 of the Submittal provides a listing of Top Events with multiple human action error rates reflecting event / sequence-specific factors influencing those Top Events.

The scaling guidance provided the raters (Table 3.3.3-2, Sheet 2 of 7) states that "If necessary, some strongly dependent failures may be accounted for by specific split fractions in the event trees." The licensee defines actions appearing in the same accident sequence as strongly dependent if they are directed at the same goal, guidance is provided by the same procedure, and time period in which the actions are to occur are roughly the same time 20

i frame. Actions directed at the same goal but separated by several hours in time, are not considered strongly dependent. The PRA analysts judgement served as the basis for determining if an action met the above criteria No credit was taken by the analyst for the second action in many cases where the impact of the dependency between actions on the human error rate for the second action may be more pronounced than can be realized through the linear equation for combining performance shaping factors. In these cases, the split factor assigned to the top event, which accounts for the second human action, was set to 1.0 for the sequence. Then during sequence quantification, the split fraction assigned to the top event, which accounts for the second human action, is set to 1.0 for that sequence.

One example given by the licensee deals with the actions to initiate recirculation from the sump following a small LOCA (Top Event OR) and the action to align for long-term makeup to the RWST (Top Event MU) given recirculation for the containment sump is unavailable. 1 In the split fraction assignment logic for Top Event MU, when Top Eve.nt OR fails earlier in l l

the sequence, no credit was taken for Top Event MU; i.e., effectively the operator error rate was set to 1.0 by assigning a split fraction with value of 1.0. l i

A second example provided, is that for initiation of manual control rod insertion (Top Event .

RI) and emergency boration (Top Event OA) during an ATWS following attempts by the l operators to manually initiate a reactor trip (Top Event OT). If OT fails, the error rate of 1.0 was used for Top Event RI. Also, if Top Event OT fails, no credit was taken for j emergency boration via Top Event OA. These dependencies between the three actions are accounted for during event tree quantification by the split fraction assignment logic.

BVPS1 treatment of dependencies appear reasonable and consistent with the HRA methodology applied.

2.3.4.4 Ouanti6 cation of Electrical Recovery Actions. Electrical recovery actions are discussed in Section 3.3.3.4 of the submittal. The STADIC4 computer code (PLG proprietary) was used to develop the calculational model to analyze the electric power system hardware and the operators' actions to restore AC power following a loss of offsite power i initiating event. The STADIC4 Monte Carlo code runs a time-integrated model for failures and recovery actions which assesses the effect of diverse failure causes and models corresponding responses started at different times after event initiation that require different amounts of time to accomplish. Power recovery considers a combination of recovery of diesel generator power and recovery of offsite power.

Scenarios analyzed are described in Table 3.3.3-10 and a summary of results provided in  ;

Table 3.3.3-11. The recovery scenarios' results are converted to split fractions for the RE top event by accounting for the proportion of the sequence frequency to which the recovery applies. The licensee's treatment of these recoveries is fairly straight forward and appears consistent with excepted practices used in other IPEs.

21

2.3.4.5 Treatment of Operator Actions in the Internal Floodine Analysis. The submittal includes HEP result listings for operator actions which were considered in the quantification of flooding scenarios and that human error probabilities were based on judgment, but essentially no further information is provided regarding consideration of human error in the flooding analysis. Flooding was determined to be only a minor contributor to CDF. The flooding analysis discussion (Section 3.3.8) does not address specific operator actions considered or credited. However, alarms and procedures for leak isolation and building inspection are credited for reducing risk in scenarios associated with River Water flooding in service, control, and auxiliary buildings. Three operator actions dealing with isolation of RW systems in the auxiliary building and one for control room HVAC room are accounted for in the HRA listing of dynamic human actions. Results for these actions are consistent with values seen for similar events in other IPEs reviewed.

2.3.4.6 Secuences Screened Out Due to Credit for Recoverv Actions. BVPS1 teran the base case plant models with all human actions evaluated to the base case error rate or 0.1, whichever is higher, to identify those human actions which were below the 1.0E-07 cutoff because of the adjustments made to these errors in the quantification process. The total number of new sequences above the 1.0E-07 value were 130. Of these the top fifteen are discussed in some detail in the submittal.

The licensee observations were as follows:

• Of the thirteen highest frequency scenarios - all involve failure of high head safety injection, and RCS cooldown and depressurization.

• Various initiators which initiate a SI signal, ECCS equipments supposedly start and 1

provide RCS heat removal however high head safety injection is lost (ZHEHH2) as a direct result of the operator prematurely securing SI (similar to the TMI accident).  ;

• Operator action ZHESEl, failure to trip the RCP on loss of seal injection effected  ;

sequences one, two, seven, eight and nine.

e Action ZHEODI, failed to depressurize RCS to allow RHR cooling in sequence five i

e Actions ZHECD3, ZHECD6 AND ZHECD7 involve a failure to cooldown and depressurize the steam generators on loss of HHSI in sequences five, eight and nine. l e In sequences fourteen and fifteen, steam generator tube rupture with an action ZHECD3 failed to depressurize using atmospheric steam dumps, second operator action ZHZWM1 involved a failure to supply borated makeup water to RWST, causing failure of high pressure makeup to RCS.

The licensee states that the key lesson learned from the sensitivity case evaluation was that {

the new sequences whose frequency of occurrence is greater that 8E-07 primarily involve failures of two or more actions.

2.3.4.7 Treatment of Overator Actions in the Level 2 Analysis. The BVPS1 back-end analysis did not include any operator actions. The only operator actions for which credit was taken during Level 2 analysis are treated in the Level 1 analysis.

22 i

l 2.3.4.8 GSI/USl and CPI Recommendations. The licensee's consideration of generic safety l l issues (GSIs) and unresolved safety issues (USIs) and of containment performance improvements (CPI) recommendations are the subject of the front-end review, and back-end review, respectively. The Submittal does not propose to resolve any issues besides Decay Heat Removal (DHR) directly with the IPE. The Submittal concludes that there are no l vulnerabilities in the associated systems and options used for DHR. Operator actions are  !

associated with notable features that directly impact the availability to provide DHR are, (1) ability to use outside recirculation spray pumps for long term core cooling, (2) requirement to provide ventilation to the emergency switchgear rooms, and (3) requirement to transfer offsite power from the unit station service transformers to the system station service transformers following plant trip. No notable CPI related issues rated to HRA were identified by the l I

back-end reviewer.

2.4 Vulnerabilities, Insights and Enhancements 2.4.1 Vulnerabilities.

The submittal did not contain a specific definition of " vulnerability", nor did it contain a description of a systematic process or criteria for identifying vulnerabilities. However, the I licensee did list several insights derived from the IPE and noted eight potential enhancements l for further assessment. Four of these embancements relate to human performance and i potential human-related plant improvements described in Section 2.4.3 of this report.

2.4.2 IPE Insichts Related to Human Performance.

l The core damage frequency (CDF) estimate Beaver Valley Unit 1 is 2.lE-04/yr. The initiating events dominating the CDF estimate are Loss of Offsite Power, Loss of Emergency AC power train, Loss of Main Feedwater, and Total Imss of River Water. These four events combined contribute more than 55% of the CDF. Table 2.4-1 lists nine initiating event contributing to 88% of CDF.  ;

Table 2.4-1, Contribution to CDF by Initiating Event l rnntrihtitinn Initiatine Event to CDF Imss of Offsite Power 23.9 %

Iess of Emergency AC Power Train 19.3 % i Partial Loss of Main Feedwater 12.3 %

Total Loss of River Water 11.2 %

Non-Isolable Small LOCA 5.6%

Excessive Feedwater 3.9 %

Loss of Emergency DC Power 3.5 %

l Steam Generator Tube Rupture 3.4 %

Isolable Small LOCA 2.8 %

l Loss of Emergency Switchgear Ventilation 2.2 %

i 23

Six sensitivity cases were included to examine the gross sensitivity to the treatment of human reliability and common-cause failures. These sensitivity case are:

e Sensitivity Case A (Increased Human Errors) - all operator errors were arbitrarily assigned failure frequencies 10 times higher than the calculated mean values. When the mean values were 1.0E-01 or higher, the frequency of failure was set to 1.0.

(electrical power recovery handled separately in Case B below.)

e Sensitivity Case B (Less Electric Power Recovery) - oase case split fractions are used for all systems and operator actions except for the offsite power nonrecovery split fractions, which are each increased by a factor of 10.

e Sensitivity Ctse C (No Common Cause) - all common cause beta factors in the model that cover nearly 300 components and 900 basic events are set to zero to see the effect of deleting common cause failures from the model.

• Sensitivity Case D (Common Cause at 95%) - common cause failure parameters are set to the 95th percentile value of their respective probability distributions to consider importance of the uncertainty in the common cause parameters relative to the mean values used in the base case.

e Sensitivity Case E (No Recovery of Ventilation) - no credit assumed for recovery from ,

emergency switchgear ventilation failures, given both fan trains lost.  !

l e Sensitivity Case F (Eliminate Ventilation Dependencies) - assumes that the temperature of the emergency switchgear rooms would not increase sufficiently to fail any plant equipment, even without considering operator recovery.

The dominant sequence model was used to quantify the impact of the above sensitivity cases I l

on CDF. The results of sensitivity analysis are presented in Table 2.4-2.

Table 2.4-2, Core Damage Frequency Sensitivity Cases

% CHANGE IN CDF I CASE CHANGES M ADE A Increased Human Error by 10 +795 B Increase Electric Power Non-recovery by 10 +228 C No Common Cause -14.9 D Common Cause at 95th Percentile +7 E No Recovery of Ventilation +6137 F Eliminate Ventilation Dependencies -14 %

24

As would be expected, the two human error cases A and B, have a large impact on the core f damage frequency. Of particular interest is the effects from electrical power nonrecovery I about which the submittal states, "the high sensitivity to electric power nonrecovery stems l from a significant contribution from SBO sequences and the fact that there is normally a high i probability of successful recovery." The results highlight the dramatic effect emergency l switchgear ventilation has on overall CDF.

Table 2.4-3 provides a rank ordered listing of significant operator actions that are included in the plant sequence models. As would be expected, from the results of the sensitivity case studies, human actions associated with emergency switchgear ventilation are dominant contributors.

In general, the process used by BVPS1 to assess importance and contribution to CDF for those human errors selected for analysis appears comprehensive and thorough. Insights which were identified by the licensee are reflected in the enhancements taken under consideration.

Table 2.4-3, Operator Action Importance to Core Damage EVENT ACTION DESCRIPTION ERROR R ATE PERCENT IDENTIFIER of CDF ZHEBV3 Operator setup portable fans to cool Emergency Switchgear 7.lE-02 12.4 REl* Restore electric power, given SBO with AFW, no LOCA 2.1E-02 10.7 ZHEBV1 Operator opens normal switchgear ventilation supply louvers 1.0 4.3 ZHERIl Operator manually inserts control rods, ATWS, OT=S 5.9E-04 3.3 ZHEHH2 Operator prematurely secures safety injection 7. lE-04 2.7 ZHEOR2 Operator aligns outside RS pump to LHS1 for high pressure recirculation 2.8E-03 2.0 ZHECD6 Operator depressurizes and cooldown secondary, SLOCA, HHSI failed 5.0E-02 1.8 RE2' Restore electric power given SBO with AFW, PORV LOCA 1.4E-01 1.5 RESA* Restore electric power, - 1 Train fast transfer recovery, given plant trip 1.2E-01 1.4 ZHEBV4 Operator starts Emergency Switchgear Ventilation exhaust fan; LOSP 7.0E-03 1.4 ZHECD3 Operator depressurizes using Atmospheric Steam Dump, SGTR 5.1E-03 0.9

• These are actually electric power nonrecovery factors that account for the time-dependent loss of emergency AC power leading to station blackout.

25

2.4.3 Enhancements and Commitments.

A total of eight enhancement are discussed in the Submittal, four of which penain to human actions. Of the four related to human actions, two were considered unnecessary i and two have been implemented. Those improvements identified as having HRA significance and their final disposition is as follows:

1) Loss of Emergency Switchgear Room HVAC - alarm response procedures that inform the operators to investigate the cause of trouble are being reviewed to determine if they can provide more explicit guidance on how to establish sufficient alternate cooling in the event that both emergency switchgear ventilation fan trains fail. More specific response procedures have been develeped.

2) Fast 4,160 V Bus Trar.sfer - explicit procedures and training on how to promptly repair or change out the failed breakers is being pursued. Procedure enhancements have been completed.

. 3) Battery Capacity for Steam Generator Level Instruments - consideration is being given to improving the limit on recovery time by providing more explicit guidance on battery load shedding or by providing for some means of charging a battery during a loss of all AC power. The installation of a 4160V station cross-tie has eliminated the need for load shedding gui' dance.

4) Reactor Trip Breaker Failure - should both reactor trip breakers mechanically bind, consideration is being given to removing power from the bus with procedural guidance. The changes made to the PORV ATWS model reduced CDF due to ATWS significantly and need for this actions was considered by the licensee to be unwarranted.

4 i

26

t 3. CONTRACTOR OBSERVATIONS AND CONCLUSIONS l i The purpose of our document-only review is to enhance the NRC staff's ability to determine with the licensee's IPE met the intent of Generic Letter 88-20. The Generic Letter had four specific objectives for the licensee:

l (1) Develop an appreciation of severe accident behavior.

(2) Understand the most likely severe accident sequences that could occur at its ,

l plant.

. (3) Gain a more quantitative understanding of the overall probability of core l

damage and radioactive material releases.

l (4) If necessary, reduce the ove all probability of core damage and radioactive .

material release by appropriate modifications to procedures and hardware that would prevent or mitigate severe accidents.

l 1

With specific regard to the HRA, these objectives might be restated as follows: 1 (1) Develop an overall appreciation of human performance in severe accidents; how )

human actions can impact positively or negatively the course of severe 1 accidents, and what factors influence human performance.

(2) Identify and understand the operator actions important to the most likely ,

accident sequences and the impact of operator action in those sequences; understand how human actions affect or help determine which sequences are l

important. l (3) Gain a more quantitative understanding of the quantitative impact of human performance on the overall probability of core damage and radioactive material release.

(4) Identify potential vulnerabilities and enhancements, and if necessary/ appropriate, implement reasonable human-performance-related enhancements.

The following observations from our document-only review are seen as pertinent to NRC's determination of the adequacy of the BVPS1 submittal:

1) Utility personnel were appropriately involved in the development and application of PRA/HRA techniques to their facility, and associated walkdowns and documentation reviews constituted a viable process for confirming that the l IPE represents the as-built and as-operated plant. The IPE may have been l strengthened if participation included other areas of plant support, particularly l' maintenance department personnel.

l 27  ;

, O_ t

2) The licensee performed an in-house peer review that provided some assurance that the IPE analytic techniques had been correctly applied and documentation is j

F '

E accurate.

1- ~

3) The IPE/HRA process involved very limited consideration of pre-initiator -  !

human action. The lack of a systematic process for identification of pre-  !

initiator human actions for inclusion in PRA models gives rise to a possibility that important human errors could be overlooked. The HRA could have been l t (

strengthened by the licensee's use of a reasonably rigorous process to identify potential pre-initiator human error contribution to system unavailability. The {

i limited nature of the licensee's treatment of pre-initiator human actions may i have deprived the licensee of valuable insights on the contribution which 1 4- maintenance, test, and surveillance can have on CDF.  ;

4) The licensee's HRA considered human events that are needed to prevent an l i

accident as well to mitigate the consequences of an accident. Both response type actions and recovery type actions were addressed. The process used by the  !

licensee to identify and select the post-initiator human events included,- (a) review of plant procedures associated with the accident sequences delineated and i

+

review of systems modeled in the IPE, and (b) discussions with appropriate plant personnel on the interpretation and implementation of plant procedures.  !

5) No numerical screening process was employed. All operator actions identified as significant were quantified with " nominal" values. The licensee's treatment of time considered both the time available and the time required for the hu' man action. The licensee addressed in a systematic manner a reasonable set of plant-specific factors influencing human performance in response actions.

Recovery actions only dealt with restoration of electrical power. Dependencies between human actions were generally accounted for through the subjective SLIM-based process for development of human error probabilities for response actions.

6) The licensee's use of only one group of expert judges to rate response actions is not consistent with the recommendations of SLIM and tends to increase the uncertainty of the analysis. Also noteworthy is that a relativ:ly high number of operator actions, approximately 32%, were assigned HEPs by the analysts and not subjected to expert group assessment.

7) The licensee did not define vulnerability. _ However, a number of meaningful enhancements to address." vulnerabilities" cited by the licensee were identified for implementation or further evaluation. All HRA pertinent enhancements involve procedure modification and/or improved training.

28

+

i i

f

4. DATA

SUMMARY

SHEETS Important Operator Actions / Errors:

The top events involving human error, their error probabilities, and their overall importance ranking are as follows  !

)

Operator Action Importance to Core Damage  !

EVENT ACTION DESCRIPTION ERROR RATE - PERCENT IDENTIFIER of CDF 1 l

I ZHEBV3 Operator setup portable fans to cool Emergency Switchgear- 7.1E-02 12.4 l

.RE1* Restore electric power, given SBO with AFW, no LOCA 2.lE-02 10.7 ,

ZHEBV1 Operator opens normal switchgear ventilation supply louvers 1.0 4.3 i

ZHERIl Operator manually inserts control

. rods, ATWS, OT=S 5.9E-04 3.3 l Operator prematurely secures safety l

- ZHEHH2  ;

injection 7.lE-04 2.7 ZHEOR2 Operator aligns outside RS pump to LHSI for high pressure recirculation 2.8E-03 2.0 l ZHECD6 Operator depressurizes and cooldown secondary, SLOCA, HHSI failed 5.0E-02 1.8 '

RE2* Restore electric power given SBO  !

with AFW, PORV LOCA 1.4E-01 1.5 RE5A* Restore electric power, - 1 Train fast transfer recovery, given plant 1.2E-01 1.4 trip ZHEBV4 Operator starts Emergency Switchgear Ventilation exhaust fan; LOSP 7.0E-03 1.4 ZHECD3 Operator depressurizes using Atmospheric Steam Dump, SGTR 5.lE-03 0.9

• These are actually electric power nonrecovery factors that account for the timeWat loss of emergency AC power leading to station blackout.

t a

29

I l

Human-Performance Related Enhancements:

Four significant human-performance-related enhancements were reponed as having l

resulted from the IPE/HRA analysis:

1)

Loss of Emergency Switchgear Room HVAC - alarm response procedures that  ;

r inform the operators to investigate the cause of trouble are being reviewed to determine if they can provide more explicit guidance on how to establish i

sufficient alternate cooling in the event that both emergency switchgear ventilation fan trains fail. l l

i

2) Fast 4,160 V Bus Transfer - explicit procedures and training on how to promptly repair or change out the failed breakers is being pursued. l l

3) Battery Capacity for Steam Generator hvel Instruments - consideration is being given to improving the limit on recovery time by providing more explicit l' guidance on battery load shedding or by providing for some means of chargmg a battery during a loss of all AC power.

l

4) Reactor Trip Breaker Failure - should both reactor trip breakers mechanically bind, consideration is being given to removing power from the bus with l procedural guidance. l l

30 l

I

l REFERENCES I t

1. Hannaman, G.W., and A.J. Spurgin, " Systematic Human Action Reliability l' Procedure (SHARP)," EPRI-NP-3583, Electric Power Research Institute,1984 l

- 2. Proprietary PLG Methodology, based on D.E. Embrey, et al. " SLIM-MAUD: An Approach To Assessing Human Error Probabilities Using Structured Expert Judgment," NUREG/CR-3518, July,1984. l i

l l

)

l 1

l I

l 1 l 31 i

j

-. .- - - ..