ML20071F885

Technical Evaluation Rept McGuire Nuclear Plant Individual Plant Exam Assessment of Human Reliability Analysis,Step 2 Review

ML20071F885
Person / Time
Site:	Mcguire, McGuire
Issue date:	08/31/1993
From:	Hass P CONCORD ASSOCIATES, INC.
To:	NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES)
Shared Package
ML20071F864	List: ML20071F885
References
CON-NRC-04-91-069, CON-NRC-4-91-69 CA-TR-93-019-17, CA-TR-93-19-17, NUDOCS 9407110235
Download: ML20071F885 (28)

Text

__ . _ _ _ ._ _ _ -. ..

CbNCORD ASSOCIATES,INC. cuTR 93-01917 Systems Performance Engineers I-... . .

TECHNICAL EVALUATION REPORT MCGUIRE NUCLEAR POWER PLANT INDIVIDUAL PLANT EXAMINATION ASSESSMENT OF '

HUMAN RELIABILITY ANALYSIS STEP 2 REVIEW P.M. Haas Prepared for U.S. Nuclear Regulatory Commisrion Office of Nuclear Regulatory Research Division of Safety Issue Resolution August,1993 1625 Autumnwood Dr. 725 Pellissippi Parkway 2676 Tammi Lane Reston, VA 22904 Knoxville,TN 37932 G tinesville, G A 30504 (703) 318-9262 (615) 675 0930 (404) 287-3367 DR DO K 00 69 o P PDR;

CAfrR.93 01917 TECilNICAL EVALUATION REPORT MCGUIRE NUCLEAR PLANT INDIVIDUAL PLANT EXAMINATION ASSESSMENT OF llUMAN RELIAllILITY ANALYSIS STEP 2 REVIEW Paul M. Haas Prepared for U.S. Nuclear Regulatory Commission Office of Nucl(ar Regulatory Research Division of Safety Issue Resolution August,1993

'~

CONCORD ASSOCIATES INC, Systems Performance Engineers 725 Pellissippi Parkway Knoxville, TN 37932 Contract No. NRC 04 91-069 Task Order No.17 ,

4 i

y e

I TAllLE OF CONTENTS 1.0 1NTRO D U(. TI O N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.1 The hicGuire IPE IIRA Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.2 Important insights Stemming from the hicGuire HRA . . . . . . . . . . . . . . . . . . 2 1.3 Overview of the Audit Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 Pre-Site-Visit Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 S i te Activi tie s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.6 Approach to Addressing Key Areas of Concern ...................... 5 2.0 A U DIT FINDING S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1 G e neral Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.1 Viable Process to Confinn that the Plant biodels Represent the A s Operated Plan t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.2 hiethodology Clearly Described and Justified for Selection; Capable of Identifying Important Human Actions . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Review of Sequences Involving Human Action . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Quantitative Process ........................................ 12 2.3.1 General Quantification Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3.2 Screening Values ..................................... 13 2.3.3 Fit 11 HEPs vs. Screening Values . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.4 Use of Generic vs. Plant Specific Data . . . . . . . . . . . . . . . . . . . . . . 14 2.3.5 Recovery hietnod and Credit for Recovery Actions . , . . . . . . . . . . . . 16 2.4 Vulnerability Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.5 Use of IPE Results by Other Licensee Organizations . . . . . . . . . . . . . . . . . . 18 3.0 CO N CLU S I O N S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 R EFEREN CES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 A P PEND IX A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 A P P EN D IX B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... 36 B.1 Introductory hiceting With hicGuire Staff . . . . . . . . . . . . . . . . . . . . . . . . . 37 B.2 Interviews and Discussions With IPE Team and Suppon Staff . . . . . . . . . . . 38 B.3 Information A udited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 B.4 Walkthroughs and Genem! Observation of Facilities , . . . . . . . . . . . . . . . . 39 B.5 De bri e fi n g . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..... ... 40 f

r ese

EXECUTIVE

SUMMARY

This technical evaluation report (TER) summarizes the process and activities, findings and conclusions from a site visit and " step 2" audit of the McGuire Nuclear Plant Individual Plant Examination (IPE) human reliability analysis (llRA). The step 1 and step 2 review focus on twelve specific issues in four general areas:

o The general approach employed by the licensee for the HRA o The sequences involving human action o The quantitative process (es) employed o Licensee evaluation of vulnerabilities.

The step 2 audit emphasized five concerns remaining from the previous step 1 " document only" review:

1) Effectiveness of the licensee's implementation of the overall llRA approach (SilARP) and the quantification techniques (e.g., the llCR model)

2) The technical basis and rigor of analysis supponing quantificaticn of human error

3) Adequacy of the documentation for the complete llRA process modeling assumptions and judgments, subjective input from operators, use of reference sources, etc.

4) The extent and nature ofinvolvement of personnel with appropriate qualifications and experience in plant operations and personnel with IIRA expenise

5) Followup and progress on implementaticn of human related enhancements identified in the submittal; and, general awareness and use of IPE/IIRA results by other plant organizations.

Findings and conclusions regarding each of the twelve issues identified by NRC as peninent to the llRA portion of the IPE are provided in the body of this repon. Conclusions regarding the five concems noted above are as follows:

1) The basic lira approach (es) selected and employed by the licensee are reasonable and appropriate approaches to meet the intent Generic Lette: 88 20.

, 2) The licensee performed a reasonably rigorous analysis in support of quantification of human error, plant specific factors were considered, and there was a consistent pattern of choosing j conservative values in dealing with uncertainties.

3) Documentation of the HRA effort is considered to be a significant weakness of the overall lira process for the McGuire 'PE.

4) There was sufficient input from qualified individuals with knowledge of McGuire operations.

Overall, the Duke team could be strengthened by addition of expenise, or at least additional traming, in human performance.

5) Duke management has been responsive to the proposed enhancements identified in the IPE submittal and has taken concrete action based on the results of the IPE to enhance plant safety.

Further, Duke management continues to be pro-active in its encouragement and support of the practical use of PRA/IPE methods and results to improve plant safety through specific programs

- and actions in a number of general areas, including procedures, training, maintenance, and i reduction of shutdown risk l

I Y

~

l 1 - - .

l.0 INTRODUCTION This technical evaluation repon (TER) summarizes the " step 2" review of the McGuire Nuclear Plant Individual Plant Examination (IPE) human reliability analysis (llRA). This repon complements and completes a previous TER prepared by Concord for the NRC which describes a " step 1" document only teview of the HRA (Ref.1). The step 2 review includes a site visit rad audit of " tier 2" documentation that is not included in the IPE submittal, it is intended to provide a better understanding of the licensee's lira process and address potential limitations identified from the step 1 review, Per guidance from NRC, this repon is organized into three sections. ' Itis introductery section includes the following: a brief overview of the McGuire IPE IIRA process; imponant insights stemming from the IPE/11RA: a brief overview of the step 2 audit process; documentation of pre site visit activities; and, documentation of the site activities. Section 2 discusses the findings from the audit, including any strengths or shoncomings identified and their signincance with respect to the overall IPE effon. Section 3 documents the audit's conclusions.

1.1 The McGuire IPE IIRA Process.

The McGuire IPE lira process followed, generally, the concepts and principles of the Systematic Iluman Action Reliability Procedure (SilARP) developed under EPRI contract (Ref. 2). Two basic types of human errors were quantified: (1) latent, or " pre initiator" errors, and (2) dynamic, or " post-initiator" errors. Latent errors are failures in human activities conducted during normal plant operations, e.g., realignment of equipment after maintenance or surveillance testing, that lead to inoperable equipment without causing an off normal condition in the plant. Dynamic errors occur during response to off normal conditions. Errors in two types of dynamic actions were quantified:

(1) " operational" human actions, and (2) " recoveries". Operational human actions are those required for successful operation of the system, given no hardware failures; they are usually directed by abnonnal or emergency procedure. Recoveries are actions taken, possibly without specific procedural direction, in response to equipment failures.

Most latent errors were quantified by assigning a value of 3.0E 03. This value was the upper bound of values calculated by a formula developed previously py an HRA consultant and used in the original (Rev 0) McGuire PRA. The formula modifies basic human error probabilities (HEPs) by several factors to arrive at estimated nominal HEPs ranging from 3.0E-05 to 3.0E 03. The 3.0E 03 value is essentially the basic HEP. In some cases, the analyst chose to use the modifying factors and calculate a lower number, or used actual plant data or referenced sources in lieu of the formula. No systematic numerical screening was performed. The audit investigation related to the process for evaluating latent error contribution focused on the underlying basis for the formula, the alternate data sources used, and any screening process employed. Findings are discussed in Section 2.

Most operational human errors were quantified using the Human Cognitive Reliability (HCR) model developed under EPRI sponsorship (Ref 3)in the early 1980s. Other sources such as referenced studies or " engineering judgment" were sometimes used in lieu of, or to modify, the HCR results.

Errors in recovery actions were estimated by various sources, including referenced studies, engineering judgment, and the HCR model. In general, relatively high values, felt by the analyst to be conservative, were applied to recoveries. The audit investigation in this area focused on the applicability and use of the HCR model and the basis for the alternate methods or data sources selected.

1 b

Quantified litent human errors and operational dynamic human errors were incorporated into the systems models as basic events in the fault trees, or in a few cases, in the " top logic" fault trees.

Recoveries were added to the cutsets after the initial solution.

1.2 Important insights Stemming from the McGuire ilRA important insights from the IIRA include findings that led to three potential enhancements that were identified in the IPE submittal:

1) SSF Operator Action - The standby reactor cooknt makeup pump, located in the Standby Shutdown Facility (SSF), is relied upon during a number of :equences to prevent a reactor coolant pump seal LOCA. An operator must be dispatched to the SSF to activate the SSF and start the pump. In sequences in which plant power is not available, the operator must start the SSF diesel generator. Failure to initiate the SSF within the required time has a significant impact on the overall PRA results. The HRA estimate of the time required for successful completion of the human actions is 15 minutes. Plant management has assigned a 10 minute criterion for success of this 1 uman action ond has implemented actions to help assure consistent operator response within 10 minutes.

2) Reactor Coolant Pump Restart Criteria - The existing plant emergency procedure for inadequate core cooling conditions directs the operator to restart the reactor coolant pumps, if the secondary side heat removal is unavailable, and the pressurizer PORVs are not open, the forced circulation of very hot gases fnm the core at high pressure could overstress the steam generator tubes, creating a containment bypass situation. A procedure change has been implemented to provide additional guidance to permit pump startup only when the steam generator tubes are covered.

3) Nucient Service Water Cross Connect - In the event of a totalloss of nuclear service water (RN) in one unit, the RN system from the other unit could be lined up to serve the critical loads of the-affected unit by opening the RN cross-connect valves. The valves are manually controlled and are normally locked closed. To enhance confidence that operators can successfully operate the valves in a timely manner, periodic exercising of the valves during refueling outages has been implemented.

~

These three potential enhancements were described very briefly in the submittal. The step 2 audit reviewed progress on implementation of these enhancements, Findings are reported in Section 2.4 of this TER.

1.3 Overview of the Audit Process j

The overall audit process consisted of three activities:

1) Review and identify Needed information - This activity included refamiliarization with the details of the HRA from the IPE submittal, a review of the step 1 findings associated with the HRA, review of questions sent to the licensee and the licensee responses, and development of a detailed site visit plan which included specific issues to be addressed, specific information needs from the licensee, and individuals to be interviewed, t 2

'I

. -_ _ -- - . _ _ - - _ - _ _ _ _= .

2) Perfonn the Site Visit A two and one half day site visit was conducted which included an audit of " tier 2" infonnation related to the IIRA, plant and simulator walkthroughs, and interviews with key licensee personnel.

3) Preparation of %Is TER - Detailed documentation obtained during the site visit was reviewed or reassessed; significant findings were identified; and conclusions and recommendations were forrned. The results were documented in this report.

All three of these steps in the overall process focus on the specific issues identified by NRC for their assessment of the llRA related portions of the IPE. These specific issues, listed in Table 1, are the technicalinues that are the focus of both step 1 and step 2 reviews. Findings from the step 2 review reported in Section 2 of this TER are organized to address each of these issues.

Table 1 Key Aspects of the Licensee's llRA Process Reviewed A. General Apprunch A.1 The IPE employed a viable process to confirm that the plant models represent the as-operated plant.

A.2 The employed HRA methodology is clearly described and justified for selection. The methodology (including human action taxonomy) employed is capable of identifying important hurnan acdons.

II. Sequences Invohing lluman Action it.1 The accident sequences appropriately considered human actions consistent with NUREO ll50 and other NRC accepted PSAs (e g., listed in NUREG 1335 Appendia B).

B.2 ne accident sequences screened out because of low human error (per NUREO 1335 Section 2.1.6.6) appear appropriate, based on the HRA techrdques employed.

h.3 For muld unit plant analyses- the IPE appropriately considered operational differences between units.

C, Quantitative l'rocess C1 In general, the licensee employed a reasonable process to understand and quantify human reliability, ne process led to a determiruuion of important human failure probabilities, and considered uncertainues in human response, either qualitative or quantitative.

C.2 The employed human error probability (HEP) screening values appear capable of screening ,1,n significant

. human errors. .

C3 ""he IPE developed human error probabilldes (HEPs) for significant hurnan actions, or provided rationale for using screening values.

C.4 Sources of generic human reliability data used in the IPE were documented. and the rationale for their use provided. Generic human error probability (HEP) data trere modified using plant specific Performance Shaping Factors (PSFs) as appropriate, and the radonale for selection of employed PEFs was provided.

C.5 The recovery method is clearly descritwi. and credit for recovery actions appears justified.

D. Vulnerability Evaluation 2

D.1 The IPE supports the licensee's definition of vulnerability with respect to human error. The licensee's dermidon provided a means by which the licensee could identify potential vulnerabilities (as so dermed) and plant modifications (or safety enhancements) to climinate or reduce the affect of vulnerabilities.

l ;, D.2 The identification of plant improvements include human-related plant modificctions (e.g., procedures and

, trainmg), and proposed ruoddications are reasonably expected to enhance hur:um reliability and plant safety.

. 3

There were five concerns remaining from the step 1 review that received special emphasis in the step 2 audit:

1) Effectiveness of the licerisee's implementation of the overall HRA approach (SHARP) and the quantification techniques (e.g., the HCR model)

2) The technical basis and rigor of analysis supporting qu:=t!Scation of human error

3) Adequacy of the documentation for the complete HRA process modeling assumptions and judgments, subjective input fmm operators, use of reference sources, etc.

4) The extent and nature of involvement of personnel with appropriate qualifications and experience in plant operations and personnel with HRA expertise

5) Followup and progress on implementation of human related enhancements identified in the submittal; and, general awareness and use of IPE/HRA results by other plant organizations.

1.4 Pre Site-Visit Activities The entire Step 1 TER and the HRA related portions of the IPE subadttal were reviewed. Strengths and weaknesses, items noted for improvement, items requiring funher information and specific questions proposed for NRC to ask the licensee were especially noted. As part of the re-examination of the IPE submittal, the calculation of every human error probability was reviewed (in as much detail as the submittal provided) to reassess the reasonableness of the assumptions and specific results. Other NRC accepted PSAs were rechecked for comparison to McGuire results. The HRA related questions to the licensee resulting from the step 1 review, and the licensee's response to those questions, were reviewed in detail. In some cases, the licensee's responses precipitated funher re-examination of the submittal or of reference documents, such as the HCR model or THERP

. references. All of this information was assessed and provided to the NRC Team Leader in the form of a draft detailed audit plan. Several iterations between NRC and Concord produced a detailed audit plan, which was transmitted to the licensee in advance. The audit plan emphasized the five concerns listed above and the twelve issues listed in Table 1. A copy of the audit plan is provided as Appendix A to this TER.

l.5 Site Activities The site visit was conducted over a two and one half day period July 28 30,1993. The NRC audit

'- team consisted of the NRC Team Leader and an HRA specialist from Concord Associates, Inc., the NRC contractor for review of the HRA portions of the IPE submittal. The proposed agenda for the site visit (see Appendix A) was followed with a few minor exceptions.

l

.~

N l

1.6 Appsbach to Addressing Key Areas of Concern Section 1.4 above identified five concems emphasized in the audit. Those five areas were addressed by the site visit activities as follows:

1) Remaining issues about the overall IIRA approach and quantitative techniques were addressed through the presentations by the licensee, the initial general discussions with the IPE/ lira team, and the subsequent audit of detailed tier 2 documentation.

2) The technical basis for quantification were addressed by the detailed discussion with the HRA team and tier 2 documentation review of specific human error quantificaicas, by tne plant walkthroughs, and by laterview with the operational staff pmviding expert judgment.

3) The adequacy of documentation was assessed directly by the tier 2 drumentation review, including materials prepared by Duke staff prior to the visit and the additional materials provided on demand throughout the site visit.

4) The involvement of appropriate qualified operations and lira staff was assessed by interview with essentially all of the staff directly involved in the HRA.

5) Followup of proposed plant enhancements was assessed by interview and direct verification of plant documentation such as " Procedure Process Records", "Pmblem investigation ProceAs Problem Investigation Forms", and " Resolution Completion Notifications". Use of IPE results by other plant organizations was addressed through interviews.

9 b ,

b T

s.

2.0 AUDIT FINDINGS Audit findings are summarized in the subsections below, organized to address each issue listed in Table 1. Items of concem relating to the various aspects of the licensee's process that remained from the step 1 review are noted, along with the pertinent findings from the audit. The summary finding is listed in bold with supporting information and rationale following.

2.1 General Approach There are two aspects related to the general appmach that are identified by NRC as key issues for the audit:

1) The IPE (HRA) employed a viable process to confirm that the plant models represent the as operated plant.

2) The employed HRA methodology is clearly described and justified for selection. The i methodology (including the human action taxonomy) employed is capable of identifying imponant human actions.

2.1.1 Viable Process to Confirm that the Plant Models Represent the As Operated plant.

The audit finding is that the lleensee did employ a viable process to confirm that the plant models represent the as operated plant. Weaknesses due to a lack of a formal structure appear to have been outweighed by the fact that the individuals directly involved in the llRA and the lndividuals supporting the analysis have extensive and up to date knowledge

- of plant operations.

While this issue was not identified as a significant weakness or uncertainty during the step 1

- review, there remained some concerns about the formality and documentation of walkdowns and interviews with operations staff, and about the involvement of HRA specialists in these IPE activities. The audit helped to confirm that the process used was reasonable, and it provideo i further clarification of the nature of the pmcess and its strengths and weaknesses. The effectiveness of the Duke IPE process with regard to this aspect is primarily a result of two important factors:

1) The individuals directly performing the HRA are engineers with a thomugh knowledge of Duke's plants - both design features, and operationst this includes the primary HRA analyst, his immediate supervisor, and the Severe Accident Gmup Leader, as well as contributors to earlier Duke PRAs who provided input to and review of the IPE/HRA.

Background and experience identified in the interviews, detailed discussions, and plant l

walkthroughs confirmed that the IPE/HRA team has a thorough familiarity with the

- McGuire plant and current operations.

l

2) Additional support from experienced operations staff with current knowledge of plant

- configuration, procedures, administrative controls, and operating practice was available throughout the IPF/HRA. Both of the primary sustained contributors have substantial operating experience at Duke and both are currently active in operations staff positions a 6 r"

which'provided them with up to date knowledge and familiarity with current operations practice, procedures, training, etc.

On the negative side, the Duke process in this area does lack formal stnicture, and the process could be strengthened in the future by formalir.ing and providing better documentation for cenain activities. For example, better documentation of walkdowns would climinate uncertainties for future analysts / reviewers. During the audit discussions and interviews with the IPE/HRA team, and during the walkthroughs,it was apparent that even the experienced analysts and operations staff on the team occasionally differed in their beliefs / understanding of current practice, status of procedure changes, etc., and/or in their recollection of what was assumed during the IPE development. And it was apparent that, as in any plant, there are essentially continuous small changes in plant status / operations. Several assumptions in the IPE, e.g., the absence of procedural guidance for a particular action, were shown to be outdated when those actions were walked through during the audit. Thorough documentation would at least provide a clear record as to what the status was at the time the IPE was performed, and what assumptions were made.

1 For complex actions, actions that are important contributors to risk, or actions which have greater uncertainty associated with them, a more structured walkdown process, perhaps involving preliminary task analysis, timed actions, and a structured list of human performance related factors, is warranted. A more formal process improves the quality and reliability of the infonnation, and provides more thorough documentation for improvement to reduce vulnerability or uncenainty.

Similarly, a more formal interview structure and better documentation of the interview inputs obtained from the SRO would strengthen the submittal and possibly improve the quality of the *

. infonnation extraction process. Structured guidance can help assure consistency and aid in systematic consideration of allimportant factors influencing perfonnance. And the documentation helps to make the rationale of the expert scrutable to others. The HRA data sheets provide some documentation of the results of the judgments made, but very little infonnation as to the basis for those judgments.

I 2.1.2 Methodology Clearly Described and Justified for Selection: Capable of identifying imponant Human Actions The audit findings helped to clarify precisely the methodology used for the llRA. The rationale for selection of the methodology was, essentially, the historic use of the approaches in previous Duke PRAs. The methodology placed heavy reliance on past Duke PRA results '

and experience and on the knowledge and experience of the IPE team. Ilowever, it -

appears to be capable of identifying important human actions. Documentation of the implementation of the methodology is weak.

. The lack of clarity in the description of the actual methodology employed, and the lack of clear i justification for the use of the various quantification approaches were issues remaining from the step 1 review, and there were several levels / aspects to these issues:

i 7

am

. . . . . . . - - , - - .- -. .x - - - . - -

1) There was an issue regarding the licensee's implementation of SIIARP as an overall framework for the HRA.

2) There were questions about the selection of quantification of techniques justification for use of the liCR model for dynamic errors, clarification of the basis for and the use of the

" factored TIIERP model" for latent errors, and use of "expen judgment", published references, or plant data on a selected basis in lieu of the primary approaches.

3) There were more detailed questions about application of the various quantification approaches for specific operator actions. The examination of these specific operator actions was used to better understand and assess the licensee's process.

The more detailed questions in item 3 about quantification of specific actions were addressed through the review of specific analysis, via tier 2 documentation review and plant walkthmughs.

Findings related to the quantification approaches are discussed later in section 2.3 of this TER.

Findings regarding the more general methodology issues 1 and 2 above are discussed in the following paragraphs.

2.1.2.1 Use of SliARP as a Framework for the lira. The submittalincluded a thorough description of how the SIIARP process should be applied, but never stated clearly if/how it was applied for the McGuire IPE. In response to step 1 questions, the licensee noted that the IPE IIRA was " consistent with the spirit" of SilARP but did not necessarily " rigorously follow" (the steps of the SilARP process discussed !a the submittal). A ten step process comparable to the published seven step SHARP process was described in the submittal. Our review of the limited documentation in the submittal suggested there may be weaknesses in the submittal information regarding Duke's step 4, " Characterize human interaction events", step 5, " Screen hur .an interaction", step 6, " Quantify humar "** action", and especially, step 10 " Document Analysis".

During the audit, the licensee clarified their basic approach as being consistent with SHARP and consisting of three basic steps:

' Identification and characterization of imponant iluman actions - performed primarily by

!)

systems analysts with input from the SRO

2) Quantification - perfomied primarily by the HRA analyst with input from the SRO and guided by past industry experience

3) Plant model update performed primarily by the system analysts with input from the SRO and from reviews by engirecting and by the remainder of the PRA team.

Discussions with the Duke team and review of the HRA data sheets indicated that there was a reasonable level of qualitative analysis and " characterization" of the dynamic human actions, though there was no formal descriptive process, task analysis format, task breakdown, or graphical representation as suggested by SHARP. The analysis was structured by the use of the data sheets to guide the analyst's assessment. The quantification methods selected did not require or support a great deal of qualitative assessment, e.g., evaluation of various performance shaping ;

factors or task decomposition. The discussion with the Duke team clarified that there was no 8

l

, 1 L

e 4 systematie numerical screening performed. Discussion with the team end audit of tier 2 documentation clarified remaining questions about quantification approtches actually used, and findings regarding quantification are discussed later in this TER.

Notable is the fact that documentation was not cited by the Duke team as one of the essential steps in the process. SilARP emphasizes the importance of thorough documentation of the inputs activities and outputs of each step to provide a traceable description of the process used to develop the quantitative assessments and the summary report for the main PRA (IPE) study document. "'Ihe analysis documentation should be organized to ensure that the information and data are scrutable; that the assumptions, data sources, mod:Is selected and criteria for elimination and retention of human interactions are recorded; and the human impact on fault trees, event trees, event trees, sensitivities, initiating events, etc., is stated." (Ref. 4) The findings from the site visit and audit confinned that documentation of the IIRA pmcess is a significant weakness overall. The primary tier 2 lira documentation supporting the limited summary material in the submittal is the llRA data sheets, and memos from team meetings in which comments were made peninent to the IIRA. 'Itese sources provide nominal documentation of the qualitative analysis performed and the basis for quantification. Ilowever, much of the infonnation on the basis of key decisions and assumptions lies within the individual and collective memory of the Duke PRA team, and is not available for others without in-depth discussion with those team members.

Obviously, it would be preferable to formalize the documentation process and provide explicit written infonnation, particularly in view of the desire and plans for a "living PRA" and the potential for loss of infonnation due to nonnal personnel turnover.

2.1.2.2 Justification for Selection of Ouantification Appmaches.

The primary justification given by the Duke PRA team for use of the liCR model for post accident actions was that it was the methodology used for the original McGuire PRA in 1984 and at that time was considered a " state of the art" methodology. Further, it was supported by EPRI, and Duke was participating in an effon to " validate" the methodology. Whether or not the llCR model was an appropriate choice in 1984, it is our view that at least the decision to use that methodology should have been reexamined when the McGuire IPE was performed, and alternative methods should have been considered. For example, we believe that a significant question exists as to the applicability of the llCR nodel for modeling post accident operator actions guided by current symptom based procedures.

On the other hand, the Duke PRA team appears to have recognized some of the pot ntial weaknessec of the IICR model and similar time reliability correlations. Simple subjective estimates were sometimes used in lieu of the HCR model when the licensee felt that the model was inappropriate. While the basis for those exceptions was not always well documented in the submittal, discussions with the Duke PRA team and review of the HRA data sh:ets suggests that a reasonable rationale typically existed. Moreover, usually the values for HEPs estimated by expert judgment were higher (more conservative) than would have been predicted by the llCR l model.

The licensee indicated that the " factored TIIERP" model was developed by a human reliability consultant (Daugherty) for previous Duke PRAs and was considered appropriate for latent actions. While it did not appear that the licensee analysts had a firm understanding of the 9

l l L

l underlying basis for the model, it did appear that the model was used appropriately. In actuality, ,

the fonnula was used for only a few of the latent errors. Instead, an arbitrary" value of 3.0E-3

• was assigned, which happens to be the upper bound of the values ht e .vould be calculated by 1 using the formula. Effectively, this approach amounts to using a screening value for the final 11EP, with excepdons taken for special cases. The exceptions wen: identined in the submittal.

Euh of those exceptions - us of the formula to generate numbers below 3.0E 3, use of plant specine data, etc. - was examined in detail during the audit. A reasonable rationale appears to exist in each case, though again documentation is sometimes weak.

2.1.2.3 Capability to Identify imponant liuman Mtions. The site visit and audit substantially confirmed the initial judgment from the step 1 review that the list of operator actions identined was reasonably comprehensive and consistent with other PS As. It also clarined funher the rather extensive historical basis from previous and ongoing Duke PRAs that contributed to the analysis, and it confirmed the familiarity and knowledge of plant operations included in the Duke PRA team staff. The appmach for idendfying important human actions relied heavily on previous Duke PSAs (and bmad industry experience) and did not use a highly stnictured process (such as detailed review of all procedural actions) flowever, it was apparent to us that the licensee team did not simply accept blindly the previous list of actions. The actions retained in the IPE models were identified through a process of multiple and essentially continuous reviews by individuals on the team who are quite familiar with plant design and operations, supported by direct review and input from the SRO and review by other Duke staff within and outside of the PRA group.

We believe the process, pnmarily because of the depth of historic experience and the plant knowledge embodied within the PRA team, to have been a reasonable one that identified the important human actions for incorporation into the IPE models.

2.2 Review of Sequences involving Iluman Action Three items are identified by NRC related to this aspect of the jcensee's IPE process:

(1) The accident sequences appmpriately considered human actions consistent with .

NUREG IISO and other NRC accepted PSAs.

l (2) The accident sequences screened out because of low human error appear appropriate, '

based on the HRA techniques employed.

(3) For multi. unit plant analyses, the IPE appropriately considered operational differences

^ '

between units.

The accident sequences appropriately considered human actions consistent with NUREG 1150 and other NRC accepted PSAs. The accident sequences screened out because of low human error (effectively none) appear appropriate, based on the llRA techniques employed. The IPE appropriately considered operational differences between the units.

. Comparison of sequences and human actions examined in the McGuire IPE to those in other PSAs revealed no major omissions or differences. The audit discussions conntmed the substantial depth of experience within the PRA team and management at Duke, both specific to

[, 10 b

J I

Duke PSAs and general experience and involvement in industry wide activities and programs.

The depth and breadth of experience canied forward from past PRAs, as well as within the current PRA team and management, helped to assure consistency with industry experience and comprehensiveness in the sequences and actions addressed. There were no issues related to potential unit differences that were identified by the review.

With regard to the accident sequences possibly screened out because of low human error, the step I review noted that the submittal hr.d not directly responded to this reporting guideline identified in NUREG-1335 (section 2.1.6.6), or at lesst had not provided a succinct listing. The Duke PRA team noted that no sequences were actually screened out (below the 1.0E-07 level). Their cutoff was 1.0E-08 for core melt frequency, and all sequences above this cutoff (1.0E 10 for containment bypass events) were reported. He licensee also noted that overall, the McGuire IPE l did not have " low" human error probabilities (in comparison to numbers typical of other PSAs).

Nonetheless, there are recovery actions for which significant credit is taken, and the uncenainty  ;

in human error probabilities warrants closer examination to assure that sequences were not lar lately climinated from consideration.

t a e of responding to this issue is to conduct sensitivity studies to determir.e the effect on th.. stitative results of substantially higher hEPs. The Duke PRA team noted that some sensitivity studies had been performed previously which included operator actions. Additional ~

sensitivity studies wem completed in response to NRC step 1 questions. He ten dynamic human actions with the most significant impact on core melt firquency were addressed. (These ten ,

acuons are the first ten in Table B.2, Appendix B of this TER.) HEPs for the ten actions were multiplied by a factor of 10 (except for one action in which the initial estimate was 3.3E-01; in 1 that case, the HEP was set to 1.0), Results indicated that increasing the HEP for these ten most important operator actions (initial values ranging from 1.0E 03 to 3.3E-01) an order of magnitude contributes from 3.9E-07 to 9.3E 05 to the core melt frequency, compared to the overall i estimated cort melt frequency of 7.4E-05.

t These sensi:ivity studies help to identify or confirm the importance of these opermor actions.

Technically, they still do not address the specific request of section 2.1.6.6 of NUREG-1335.

i Multiplying an HEP of, say 1.0E-03, by a factor of ten does not indicate the impact of the credit ,

taken for that action. To assess that, the HEP would have to be set to 1.0. In the case of a l "true" recovery action, in which the operator action was added to the cutset after the fault tree i solution, the quantitative impact of the HEP is straightforward and obvious. However, when the HEP is included in the fault tree solution, it is necessary to perform the solution in order to determine the impact on core melt frequency. The reporting guideline asks for reporting of any sequences which drop below the applicable cutoff frequency (e.g.,1.0E 07 for core melt) because thefrequency (not the HEP) was decreased by an order of magnitude or more due to credit taken for the human action. He sensitivity studies increasing the HEPs by_ an order of magnitude do not precisely address this guideline. The issue is further confused by reporting the s increase in overall core melt frequency due a change in a particular HEP, because in some cases '

the same HEP can appear in more than one sequence.

[

. While the licensee may not have complied precisely with the guidance in Section 2.1.6.6 of NUREG 1335, it appears that the intent of the guidance was met, it is unlikely that any important sequence was somehow overlooked or not reported due to inappropriately low values 9 11_

g

of HEPs for recovery actions. First, there were no recovery actions which appear to be " low" in comparison with typical numbers reponed in other PSAs. (Indeed "true" recovery actions typically had estimated HEPs of 1.0E 01 or greater.) Second, the cutoff was already an order of magnitude lower than " required". Third, sensitivity studies raising the HEPs, not necessarily to 1.0, but an order of magnitude, indicate a few cases of significant but not order of magnitude increase in estimated in core melt frequency. Founh, all of those human actions that have a significant impact were already ideDtified and discussed in the IPE.

2.3 Quantitative Process NRC identifies five items of focus for their assessment of the quantitative process:

(1) In general, the licensee employed a reasonable process to understand and quantify human reliability. He process led to a determination of important human failure probabilities, and considered uncertamties in human response, either qualitative or quantitttive.

(2) The employed human error probability (HEP) screening values appear capable of screening ja significant human errors.

4 (3) The IPE developed human error pmbabilities (HEPs) for significant human actiors, or provided rationale for using screening values.

(4) Sources of generic human reliability data used in the IPE were documented and the rationale for their use pmvided. Generic human error probability (HEP) data were modified using the plant specific Performance Shaping Factors (PSFs) as appropriate, and

, the rationale for selection of employed PSFs was provided.

T (5) The recovery method is clearly described and credit for recovery actions appears justified.

L 2.3.1 General Ouantification Process I The licensee,in general, used quantification techniques that are relatively coarse and simplifled but that have gained sene level of acceptance in the HRA community and/or are capable of providing reasonable quantitative estimates that are consistent with other results ,

• - of other techniques. The audit finding is that the licensee used these techniques appropriately to the end that the important human failure prchabilities were determined,

and uncertainties were addressed in an appropriately conservative manner.

The primary quantification techniques employed were the Human Cognitive ReL 0.y (HCR) l Model for the dynamic, or post initiator, human errors, and a " factored THERP" model for pre-initiator, or latent, human errors. While these techniques do not preclude thoughtful, in depth analysis that would provide en improved understanding of the underlying factors that g_

e influence human reliability, neither do they provide a strong framework that demands, guides, or documents such a thoughtful and detailed assessment. Thus much of the " quality" of the analysis-r underlying the quantitative result, and therefore, the level of understanding of the important

[ factors is left to the individual diligence, knowledge, and skills of the analyst, within the practical

. constraints of time, and availability of resources, e 12 L . _

Strengths of'the quantification process McGuire IPE ilRA are the cumulative history and experience base from the previous Duke PRAs, the plant oesign and operations knowledge of the llRA team, the operations input to the team, and the fairly extensive review of analysis by qualified systems and operations personnel within and extemal to the PRA group. The major weaknesses are the lack of a systematic framework for detailed assessmerit of underlying factors influencing human performance, heavy reliance on the judgment of a single SRO for the critient parameter (time to accomplish the task) in the !!CR model, and poor docume/itation of plant specific assessment that was accomplished.

While the focus of the audit and the entire NRC review of the IPEs is on pmcess and not simply numerical results, it is imponant to note that overall, the McGuire HRA numerical results, i.e.,

liEPs, are consistent with what are believed to be appropriately conservative estimates from other PSAs. There was no evidence that the analysts were taking advantage of the uncenainties in human error modeling or the coarseness of the quantitative techniques to selectively " pick" favorable numerical results. Indeed, there appeared to be a concerted effort to provide realisne but appropriately conservative results and to " error" in the directien of conservative values wcn uncenainties were greater.

We believe that a more rigorous and in depth analysis of underlying factors influencing human performance, panicularly with emphasis on potential plant specific factors, would add to the Duke staff's understanding of the contribution of human performance to risk, and could identify additional ways to enhance human performance and thereby further reduce risk and/or uncertainties. For example, an examination of surveillance and calibration practice and procedures in suppon of estimates of latent error HEPs may not only reduce uncenainties in those estimates but also identify potential systematic improvements that could be made.

Similarly, effort expended to systematically observe operator actions during simulator training using a controlled protocol could impmve confidence in estimates of dynamic error pmbabilities and help identify potential improvements in training and/or procedures. While we recognize that in depth " human factors" assessments may be beyond the scope of the IPE, we believe that the licensee in this case could have taken the " opportunity" afforded by the IPE process to perform a more effective assessment and thereby learn more about human performance and the underlying factors influencing that performance in the McGuire. Nonetheless, the general approaches used by the licensee and the results of the HRA are consistent with other accepted PSAs and appear to be reasonable to meet the intent of Generic Le:ter 88-20.

2.3.2 Screenine Values o

in general, the quantification approach employed by the licensee did not include numerical screening. Obviously, qualitative screen is employed in the selection of specific operator action events to quantify. But HEPs were developed for all human actions identified by the system analysts and HRA analyst for inclusion in the IPE models. No HEPs were excluded from the models simply on the basis of a numerical screening.

• - 2.3.3 Final HEPs vs. Screening Values As noted above, there was no systematic numerical screening performed.

3 13 7

L .

e 2.3.4 Use of Generic vs, Plant Specific Data In general, the licensee developed plant specific HEPs. Discussions with the Duke PRA team, examination of the IIRA data sheets and detailed review of specific HEP estimates revealed that for most cases, within the limitations of the quantification techniques selected, the licensee did perfonn plant specine evaluations to develop plant specific 11EPs. Specific cases in which generic data was used, or other appmaches different from the two basic techniques were used are identified. 'Iherefore, the Rndings of the audit are that sources of generic human reliability data used in the IPE were documented and the rationale for their use provided, Generic human error probability (HEP) data were modified using the plant specific Performance Shaping Factors (PSFs) as appropriate, and the rationale for selection of employed PSFs was provided.

2.3.4.1 1.atent Human Errors. HEPs for latent human events were said to be calculated primaiily from a " factored THERP" model, which uses the following equation to modify (generic) basic IIEPs from TIIERP tables:

Pu = P,

• f,

• f,

• N where P = basic slip occunence probability (from THERP Tables) f,= PSF for serveillance or for functional test f, = common cause (beta) factor for multiple trains N = number of components involved While the basic HEP from THERP is generic, the three factors offer a crude means for accounting for limited plant-specific differences in the task, e.g., by taking credit for surveillance or functional testing revealing latent errors prior to their causing a problem during an accident sequence, or by increasing the error probability to account for dependency effects when the task has to be performed on multiple components.

The audit of tier 2 irformation showed that there were 34 latent human errors (LHEs) included in i the IPE model. The value of P, was set to 0.003 for all cases, based on the rationale that various TIIERP tables pmvide values (depending on the circumstances) ranging from 0.001 to 0.01.

I Clearly, this is a generic value which does not use even the limited flexibility provided in by a THERP guidance on selection of the tables, Further, for all but seven of the 34 LHEs, the final quanti 0 cation value was set at this basic HEP value of 3.0E 03, which, of course, is the value calculated by the formula if all three multiplying factors are unity, i.e., no credit (or penalty) is taken for the three possible plant specific factors. In essence, this is an arbitrary selection of a generic HEP value, and no analysis of plant specific conditions is attempted. In fact, an HEP value of 3.0E-03 to 3.0E-(M is typical of HEPs used in PSAs for latent errors, and is not unreasonable. However, the advantages gained from qualitative assessment of underlying human factors in a more rigorous approach are lost.

There were several different case-specific reasons given for using values other than 3.0E-03 for the seven cases. (Actually a post-visit review of the detailed date indicates there were eight cases, plus an additional ninth one that was due s!mply to an input error that was corrected after the submittal v as prepared.) In three of those seven cases, there were plant specific data i

14

. t

k 4.

available which was considered to provide a better estimate. He backgrri nd data on which th estimate was based was provided and reviewed for two of those three cases, which were humar errors related to diesel generator failure. The HEP was estimated to be 2.25 E-03. The third case of plant specific data was a latent error which fails reactor coolant pump injection; the estimated value was 1.2E-02, considerably higher than the generic value.

In the other four cases, the formula was applied, and substantially lower HEPs (1.8E 05 to 5.0E 05) were used. Sample calculauons for those cases using the formula were reviewed and discussed dur'.ig the audit. In our judgment, the McGuire HRA analysts did not have a firm understandi .g of the underlying technical basis for the model, but they did apply the model appropriate 4y. The licensee verified modeling assumptions about plant systems or operational / maintenance practice. For example, if credit was taken for independent surveillance testing or functional testing, the existence of that testing in c..:rrent operationc.1 practice at McGuire was verified. Sample documentation of requirements for such testing were provided on demand during the audit. In response to a question from the audit team about sensitivity studies on these latent errors, the Duke staff indicated that there had been no broad sensitivity study performed on the LHEs for McGuire, but that sensitivity studies on similar actions for Catawba indicated that increasing these lower values by a factor of ten had little impact (less than 0.1%)

on overall core melt frequency. During the visit, the licensee per~ormed a sensitivity study on two of the McGuire LHEs in the IE 05 range, and those results were similar to the previous studies.

While we view the factored THERP model as simplistic, the model and the application of it by the licensee staff are not inconsistent with approaches used in other accepted PSAs; and numerical results appear to reasonable, appropriately conservative, and not inconsistent with accepted PSAs.

2.3.4.2 Dynamic Human Errors.

As indicated earlier, the Duke classification of human errors includes two kinds of dynamic human errors - operational human errors, and recoveries. Most of the opetational human errors i were quantified by using the HCR model. The HCR model is essentially a generic model, which can be manipulated to a degree to account for plant-specific differences. However, the model does not require a significant plant-specific evaluation in order to use it, and the level of rigor and qualitative investigation of potentially important factors influencing performance can be highly variable.

- Consideration of plant-specific conditions was structured to a degree by the HRA data sheets.

Factors considered include:

- Failure description

- Description of compelling signal

- Recognition (long or short time, straight forward or confusing, NEO required or from control room)

Degree of difficulty of action

. Amount of training or practice

- Skill / Rule / Knowledge or between 15 e

O e

. Source hf documentation (simulator / expert opinion, etc.)

+ Time required

. Time available (with supporting information, including type of analysis performed, e.g.,

MAAP, or other basis for the time estimate and the source of documentation for the estimate)

One weakness in the McGuire appmach in that the critical input parameter, namely the expected time required for the operator to act, was determined by the single SRO on the team (with, of course, review by the other team inembers). It is generally recognized that experienced operators, especially without the benefit of observation in context such as might be provided by dynamic simulator runs or realistic walkthroughs, will tend to overestimate performance, i.e.,

underestimate time required.

The audit reviewed with the Duke team a number of cases in which the HCR model was not used and examined the rationale and the source information for the alternative approach, i.e.,

plant-specific data, published studies, or " expert judgment". Sample documentation from referenced industry sources was pmvided and reviewed. The licensee IPE team provided evidence that before accepting data from published industry references they had reviewed referenced sources, assessed their applicability to McGuire, and considered plant specific factors.

Review of tier 2 information showed appmpriate use of historic data from McGuire.

Documentation of expert opinion or engineering judgment typically was weaker. Almost without exception, those HEPs estimated by subjective judgment were relatively high values, more conservative than might have been calculated by the HCR model. More germane to the issue of plant-specific vs. generic data, there was evidence in virtually every case in which the HCR method was not used, as well as when it was used, that some plant specific factors were taken into account.

2.3.5 Recovery Method and Credit for Recovery Actions, The audit findings confir;n that the recovery method is clearly described and credit for recovery actions appears justified.

Recovery actions were added to the IPE model cutsets after model solution. The HEP

' quantification techniques included essentially the same mix of approaches employed for the dynamic human actions discussed above, except that gre! ter reliance was placed on engineering judgment and plant-specific analysis. Data fmm the HRA data sheets (e.g., time required vs.

available, compelling signal, complexity, etc.) were used to guide subjective estimates.

There were 14 recovery actions included in the model, including the six related to restoration of offsite power. Those six were estimated using a published model with considera: ion of plant specific data. Failure of another important recovery action, recognizing and rect,vering failure of the automatic realignment of the Residual Heat Removal System to the containment emergency sump upon deletion of the FWST, was estimated using data from the Sequoyah

, NUREG-1150 study, with appropriate consideration of plant-specific factors.

16 s

4 4 In general, liEPs for recovery actions, especially those based on engineering judgment and -

plant specific analysis were relatively high (1.0E-02 to 3.0E-01), and appear to be appropriately conservative. Detailed review of the material papand by the Duke staff, and plant walkthroughs provided convincing evidence that the Duke staff considered critical factors affecting recovery actions e.g., existence and availability of procedures, amount of training and practice, presence of a compelling signal, and practical obstacles to ex-control room actions, such as sufficient staffing, accessibility to equipment, etc., (as well as timing). weakness is that the documentation of basis forjudgements is not thorough and mults may not be reproducible by future Duke analysts without the help of the current team.

2.4 Vulnerability Evaluation The issues related to vulnerabilities have to do with the process for identifying vulnerabilities and the apparent effectiveness of proposed enhancements, specifically whether:

(1) The IPE supports the licensee's definition of vulnerability with respect to human error.

The licensee's definition provided a means by which the licensee could identify potential vulnerabilities (as so defined) and plant modifications (or safety enhancements) to eliminate or reduce the affect of vulnerabilities.

(2) The identification of plant improvements include human-related plant modifications (e.g.,

procedures and training), and proposed modifi#.ons am reasonably expected to enhance human reliability and plant safety.

The audit findings are that significant human-performarce-related enhancements have been implemented and that they_can reasonably be expected to enhance human reliability and plant safety.

While the submittal did not formally identify "velnerabilities", there were a number of potential enhancements identified, including several that were human-performance related. The audi' obtained further information on the process used by Duke to assess whether enhancements shoeld i be implemented. And, it obtained updated information on the status of implementation of the human-performance related enhancements.

The McGuire approach to assessing the IPE results for potential plant enhancement included: (1) review of the results by the team to identify potential enhancements which could have an appreciable impact on core melt frequency and/or fission product release potential; (2) sensitivity

=

studies to quantify the potential impact; and, (3) a cost benefit study (using $1,000 per person rem whole body exposure). Candidates for enhancement were then reviewed by plant J management and engineering personnel to select the enhancements that would be made. Five such enhancements were identified for implementation, three of which were primarily

- human performance related and were identified through the HRA. These three were described in Section 1.2 of this TER.

The current status of each of these enhancements was reviewed during the audit. Backup documentation verifyiag imphnentation was provided and myiewed, and one detailed cut-benefit ar..aysis was provided and reviewed. Audit findings are:

m 17 Y

in

(1) SSF Operator Action - Because of the importance of this tetion, plant management has imposed a 10-minute success criterion, and has incorporated a requirement in the SSF operability test procedure and a specific job performance measure (JPM) in operator

, training requiring demonstration of the ability to activate the SSF within 10 minutes.

Observations by operations personnel supporting the IPE were that typical times in training axe less than 10 minutes. The SSF Operability Test procedure change dictating the 10 minute criteria and providing specifics about timing operators with a stopwatch was reviewed as documentation of implementation of the change. While no additional credit was taken in the IPE for the change, this enhancement should provide further confidence that this recovery action can be accomplished in a timely manner should the necd occur.

(2) Reactor Coolant Pump Restut Criteria - Additional procedural guidance to permit pump stanup only when the SG '.ubes are covered with a mixture level was recommended as a result of the IPE. This prr.cedure change has been submitted to the Westinghouse Owners Group for consideradon for incorporation into the WOG Emergency Response Guidelines.

! Documenta6on of the feedback to the WOG was provided by the Duke staff.

4 (3) Service Water Cross-Connect - An enhancement proposed from the IPE to provide further confidence that operators would perforTn this recovery action in a timely manner was to require periodic exercising of the valves during refueling outages. That operabi'ity test procedure has been implemented. A copy of the procedure and associated problem investigation process documentation implementing the change were revicwed by the audit team. Operability testing h required once every refueling outage. Current requirements are to manually manipulate each valve to the full or a position and then reclose it. An additional change is under consideration to require ctually establishing flow through the

- valves.

I In addition to these three, an enhancement related to the Refueling Water Storage Tank (FWST)

L level instrumentation, which indirectly involves operator action, was reviewed in detail, partly because of the risk significance of the undetected failure of the instrumentation, and partly

! because the licensee provided a complete cost-benefit assessment of this enhancement as an

' example of their process to determine whether or not an enhancement identified in the IPE should be implemented. The instrument sr n did not cover the full range of FWST level, and there was I a potential for undetected " failed-high" or " failed as-is" modes. Options considered were either to expand the instrument span to the full range or to increase the sensor test frequency. The cost-benefit study demonstrated that it would be more effective to implement an inen,ased test frequency, and that change has been implemented. The cost benefit study was of interest not only because it helped to establish in quantitative terms the reasonableness of the proposed enhancement, but it also helped to illustrate the use of PRA approaches and guidance (including NUMARC guidance on assessing vulnerabilities) to assess the efficacy of making potential enhancements identified in the IPE and to illustrate the benefits to management.

2.5 Use of IPE Results by Other 'ticensee Organintions An issue of concern for the overall IPE process is whether and how results from the IPE are used by other licensee organizations to enhance overall safety of plant operation. Audit findings are that the Duke PRA team identified a number of general activities and specific cases which s 18 L

~

demonstrate effective and increasing interfaces between the PRA Group and other Duke /McGuire organizations which is leading to both improved information to the PRA Group and practical use of the IPE/IIRA results.

The PRA Group has for some time reviewed and included in their " knowledge base" for the PRA/HRA, incident reports and similar information generated internally or in response to NRC requirements, as they identified an event of interest. More recently, they have been more formally involved in reviewmg such repons. They now have direct access to a computerized system incorporating all Pmblem Investigation Process (PIP) forms, which apparently includes pre LERs and vinually all reponable and non-reportable problems. The team reviews all reports on a routine, continuous basis, providing input to resolution as well as accumulating information for updating the IPE/HRA results. For Oconee and Catawba plants (presumably McGuire in the future) the PRA Group is beginning to provide the input for the " safety significance" section of the LER and include a paragraph on " core melt" safety significance.

PRA Group members are pan of the Engineering Group, which includes Maintenance Engineering, and PRA/HRA information is being used by Maintenance Engineering, e.g., for component reliability data and technical specification improvement. Coordination with Maintenance Engineering provides the PRA group continuous information on safety system unavailability which is routinely used for updating the PRA. Duke is currently planning a reliability centered maintenance program which will make heavy use of PRA models and results to prioritize systems and aid management of maintenance on a risk basis. The PRA group has already provided systems analysis and data for two RCM studies (manual feedwater and fesels).

The PRA Group also is providing input to the Duke shutdown risk program currently underway in response to NUMARC and INPO guidelines and NRC attention. The group and PRA techniques are used to assess outage schedules for opportunities for risk reduction. They have

~

panicipated in five outage scheduling effons to date, contributing to risk-based scheduling. An initial shutdown risk model (for Oconee) is under development.

In addition to these programmatic activities, the Duke team noted specific cases, and provided i supporting documentation, in which PRA results and/or PRA Group input has been used to modify training and procedures (including and in addition to the three human-related i enhancements noted in the IPE).

Overall, the findings fmm the audit are that there are significant and increasing routine programs and special initiatives in progress within the licensee's organization to encourage and enhance the use of PRA methods and results in practical ways to improve plant safety. This is consistent with the utility management's historic strong involvement with and suppon of PRA. One

, somewhat surprising inconsistency in this pattern is an apparent lack of strong and direct i

coordination of the PRA effons, in panicular the HRA efforts, with the corporate wide Human Performance Improvement effort. It is recognized that the program is a fairly high level one i providing broad coordination of Duke efforts to improve human performance of all personnel.

However, a direct interface with the PRA/IPE human reliability analysis effort would seem

. appropriate.

' 19 6

l I O

?

3.0 CONCLUSION

S Specific issues identified by NRC as pertinent to their nyiew of the licensee's HRA were listed-in Table 1. A succinct summary statement of audit findings related to each of the specific review items was provided in bold text for each major subsection of Section 2 of this TER. - For convenience, the summary statements are repeated below:  ;

o The licensee did employ a viable process to confirm that th; plant models -

represent the as. operated plant. Weaknesses due to a lack F a formal structure appear to have been outweighed by the fact that the individuals directly involved in the HRA and the individuals supporting the analysis have extensive and up to-date knowledge of plant operations, o The rationale for selection of the methodology was, essentially, the historic use of the approaches in previous Duke PRAs. The methodology used - placed heavy reliance on past Duke PRA results and experience and on the knowledge and experience of the IPE team. However,it appears to be capable of identifying important human actions. Documentation of the implementation of the methodology is weak. ,

o The accident sequences appropriately considered human actions consistent with NUREG 1150 and other NRC accepted PSAs. The accident sequences screened out because of low human error (effectively none) appear appropriate, based on the HRA techniques employed. The IPE appropriately-considered operational differences between the units.

o The licensee,in general, used quantification techniques that are relatively coarse and simplified but that have gained some leycl of acceptance in the 1- HRA community and/or are capable of providing reasonable quantitative estimates that are consistent with other results of other techniques. The audit i finding is that the licensee used thes techniques appropriately to the end that 1 the important human failure probabilities were determined, and uncertainties were addressed in an appropriately conservative manner.-

i '

" Lo Sources of generic human reliability data used in the IPE were documented.

Land the rationale for their use provided. Generic human error probability (HEP) data were modified using the plant specific Performance Shaping -

• Factors (PSFs) as appropriate, and the rationale for selection of employed -

PSFs was provided.

i o The recovery method is clearly described and credit for recovery actions

.. appears justified.

o Significant human-performance-related enhancements have been implemented r and that they can reasonably be expected to enhance human reliability and

. ;: plant safety.

_ L, 20 L _

4 ,

~

Five specific concems were identified for emphasis in the step 2 review. Conclusions regarding those five concems are as follows:

1) Implementation of the basic structure (SHARP) and the cuantification techniaues (e.g.. the HCR model) employed in the HRA. While the licensee did not follow the detailed SHARP process discussed in the submittal, the interviews and tier 2 documentation reviewed demonstrated that the licensee IPE staff understood and applied the essential basic guidance offered by SHARP to structure the HRA, with the exception of SHARP's emphasis on thorough documentation, which is discussed below. The selection of the two primary quantification tecimiques - HCR for dynamic errors and the " factored THERP model" for latent errors - apparently was made largely on the basis of historic use by Duke, with little thought given by the staff to altematives or more recent techniques. We believe these techniques to be simplified, relttively coarse appmaches that do not fully support (though they do not preclude) the kind of rigorous, qualitative and quantitative assessment that leads to a deeper understanding of human performance and aids identification of potential weaknesses / enhancements in the underlying factors that influence human performance.

However, these and similar techniques ' nave been used in other PSAs, and the HCR model, in panicular, did receive substantial review and use by the HRA technical community in the past. Funher, we recognize that there are many other types of NRC and licensee programs and requirements - procedures verification and validation, control room design review, operator training and licensing, etc. - that have in the past and/or continue to evaluate these

" underlying factors"; and we recognize the focus of the IPE on quantitative assessment of risk. Given all of these factors, we conclude that the basic HRA approach (es) selected and employed by the licensee are reasonable and appropriate approaches to meet the intent Generic Letter 88-20,

2) The technical basis and rigor of analysis supportine cuantification of human error, in general and specifically for a number of important human actions. Given (1) the limitations of the quantification techniques noted above, and (2) the limited documentation provided in the submittal, an import:mt focus of the step 2 audit was to evaluate the degree to which the numerical quantitative results summarized in the submittal were supported by rigorous, plant specific assessment. Based on results of the interviews, detailed review of tier 2 documentation, and plant walkthroughs reviewing plant-specific conditions and procedures related to the most risk-significant operator actions, we conclude that the licensee did perform a reasonably rigorous analysis in support of quantification of human error, that plant-specin'ic fhetors were considered, and that there was a consistent pattern of choosing conservative values in dealing with uncertainties. General strengths and weaknesses were noted in Section 2 of this TER. It is our belief that despite the general limitations of methodologies selected, and to some extent the state-of the-art in HRA, the licensee made a concerted effon to obtain realistic quantitative estimates of human error probabilities for its plant. Funher, where more rigorous analysis could not remove uncenainties, the license:s tended to use conservative values.

3) Adecuacy of the documentation for the complete HRA process - modeline assumptions and judgments. subiective input fmm operators, use of reference sources m es Our conclusion, which has been noted several times throughout this TER, is that documentation of the 21 e

I L -

IIRA e'ffort is considered to be a significaat weakness of the overall HRA process for the McGuire IPE. The entries on the HRA data sheets provide for reasonable documentation (within the constraints and demands of the methods employed) of the key parameters or factors used directly in make the quantitative estimate, though there. is inconsistency in the level of detail and completeness fmm case to case. Documentation of the basis for subjective estimates of operator tesponse times or other fanns of input from subject matter expens is weak. Walkdowns were not formally documented. Changes to assumptions, rationale, input data or results appear to be loosely controlled by the HRA analyst. Some documentation of technical review and of the basis for subsequent change is maintained by memo of group meetings and presentations. Other documentation is by informal notes on the handwritten data sheets, not all of which is being inen'porated into the computerized data sheets.

In general, much of the information about the underlying technical basis, rationale, and assumptions made in the qualitative and quantitative assessment of human seliability, and much of the knowledge gained about human performance in the McGuire plant and the factors that influence human performance, is accessible only through the memories of the individuals that have been involved with the McGuire PRA and IPE (and Oconee and Catawba PRAs, as they apply). While this individual and shared knowledge within the PRA group is a tremendous resource, and one of the notable suengths of the McGuire IPF/HRA, it is obvious that formal documentation is desirabh., especially in view of the intent by Duke to maintain a "living PRA" over an indefinite period. In our view there already has been a " deterioration" of corporate knowledge in that the current team was not always able to explain the underlying technical basis or rationale for certain inputs or decisions, referring instead to Duke staff or consultants who had previously been involved,

, perhaps on a previous IPE for Catawba or Oconee.

4) The personnel involved directiv and indirectly in the HRA related aspects of the IPE.

includine the IPE team and other plant personnel omvidine input to or usine results of the IPFJHRA. A centralissue remaining regarding personnel vras whether there was sufficient involvement of operations personnel to assure that the HRA represented the current I- as-operated plant. In panicular, because of the heavy reliance on " expert opinion" for basic data and information, there was a question as to whether there was sufficient in depth plant operations knowledge and experience available to the team. Other issues related to personnel were involvement of personnel with experience and qualifications in HRA, and whether there was appreciable interaction between the PRA Group to provide feedback of results from the IPE to appropriate operations staff. The issue of feedback of results to other organizations is addressed in item 5 below. The other two issues are discussed here.

The submittal had referred to judgments of a single SRO. As indicated in Section 2, the judgments about operator timing did rely heavily on a single SRO who was heavily involved in the McGuire IPE HRA. This is considered a weakness. Highly qualified i

personnel can be expected to overestimate capabilities, especially when estimates are not made in the context of dynamic exercises. Timed walkdowns, reviews in the context of i

. simulator exercises, review of historic experience, or multiple independent judgment can  !

l help reduce uncertainties in estimates. However, the knowledge available from the single SRO was supplemented by: (1) direct access to other operations, maintenance and e 22 I -

~~

enginee' ring staff for inpnt and review; (2) the per>onal knowledge of plant equipment and operations practice on the part of the engineers and analysts within the IPFJHRA team; (3) independent review by other Duke organizations; and, (4) the cumulative body of operations '

expertise that was accessible to the team from previous Duke PRAs (specific individuals were noted in the interviews). Consequently, it is our conclusion that there was sufficient input from qualified individuals with knowledge of McGuire operations.

There was substantial evidence of experience and knowledge in techniques for obtaining quantitative estimates of human error. It appeared that there was not always a complete understanding of the underlying technical basis for models and assumptions, nor an up-to-date familiarity with developments in HRA modeling since the mid 1980s. There appeared to be heavy reliance on modeling decisions and assumptions made in past PRAs.

On the one hand, this past history includes the collective knowledge and experience of many qualified individuals, and it would be foolish to ignore this substantial resource. On the other hand, it would appear that a " fresh look" by qualified personnel would be very useful in meeting the ultimate goals of the IPE process. A reasonable solution might have been to employ outside HRA expertise early in the process when methodologies were being selected, and perhaps in a review capacity periodically during the development of the HRA.

From the interviews and discussions during the short audit visit, we gained a great deal of respect for the knowledge of plant design and operations on the part of the HRA analyst (s) involved and on the part of the PRA Group technical leads and management, and we were greatly impressed by the diligence and energy displayed in developing the quantitative estimates. However, overall the Duke team could be strengthened by addition of expertise, or at least additional training,in human performance.

5) Followim and omgress on implementation of human-related enhancements identified in the submittal, and general awareness and use of IPF/HRA results by other plant organizations.

There was substantial evidence presented of implementation of the specific human performance-related enhancements identified in the submittal and of use of the IPF/HRA results by other plant organizations. We conclude that Duke management has been responsive to the proposed enhancements identified in the IPE submittal and has taken concrete action based on the results of the IPE to enhance plant safety. Further, we conclude that Duke management continues to be pro-active in its encouragement I and support of the practical use of PRA/IPE methods and results to improve plant

' safety through specific programs and actions in a number of general areas, including procedures, training, maintenance, and reduction of shutdown risk.

ht u b 23 l

E -1 REFERENCES

1. P.M. Haas and C.R. Bovell, ' Technical Evaluation Report, McGuire Nuclear Plant -

Individual Plant Examination Human Reliability Analysis, Step 1 Review," CA/I'R 92-019-01, Concord Associates, Inc., April,- 1992,

2. Electric Power Research Institute, " Systematic Human Action Reliability Procedure (SHARP)," EPRI NP-3583, June,1984.

3. G.W. Hannaman, AJ. Spurgin, Y.D. Lukic, "A Model for Assessing Human Cognitive Reliability Model for PRA Studies" IEEE Third Conference on Human Factors and Power Plants. Monterey, CA 1985.

4 See Reference 2, page 3-59.

4-t

_I _

.w r-w

{

J APPENDIX A AUDIT PLAN M

ett Omn Omv D'

l

.s. ... .

July 16, 1993 .

Docket Nos. 50-369 and 50-370 Mr. T. C. McMeekin Vice President, McGuire Site Duke Power Company 12700 Hagers Ferry Road Huntersville, North Carolina 28075-8935

Dear Mr. McMeekin:

SUB02CT: MCGUIRE INDIVIDUAL PLANT EXAMINATION (IPE) AUDIT With agreement of your staff, a two-person team will visit the' McGuire site on July 28, 1993, to perfor= a~two day IPE audit. The audit will focus specifically on the human reliability analysis that was part of the McGuire IPE.

The team leader is E. Chow, Office of Nuclear Regulatory Research. For your informatien, an agenda for this audit is enclosed.

Sincerely,

. .. CRIGINAL SIGNED BY:

Victor Nerses, Project Manager

' Project Directorate II-3 Division of Reactor Projects - I/II Office of Nuclear Reactor Regulation I

Enclesure: DISTRIBUTION Agenda Docket File NRC/Lccal PDRs E. Chow

{. P011-3 R/F- S. Varga

.cc w/ enclosure: G. Lainas D. Matthews See next page V. Nerses L. Berry OGC ACRS (10)

E. Merschoff, RII

. ,/

"E PDII-3/t.Ak PQtIkdPM $

8d[I3/kI N

L.BERRYh I.NERSES , D.MATTHEWS b D"E 7/h/93 7//6/93 l 7/8/93 ,

OFFICIAL RECORD COPY FILE NAME: G:\McGUIRE\IPEAUDIT et O '

-- ..- - . . ~ . -. _. _ -

, .: :.... z .

)

AGENDA FOR MCGUIRE HRA AUDIT July 29.1993 TI$1 ACTIVITY 8 a.m. Arrive at site Check in Site access training, etc.

10 a.m. Introductory meeting with licenses staff and management including presentation by NRC team leader on objectives and planning 10:30 a.m.

Licensee presents overview of HRA approach, data sources, and general results If applicable, identify any additional actions or information available since the submittal Discuss Catawba HPA Identify important differences contributing towards identificatien of different important human errors, and hence, different important accident sequences 12 Noon Lunch 1 p.m.

Initiate detailed discussion with licensee HRA staff and '

- review of tier 2 information

, 4 p.m. NRC team breaks to assess prc ress and icentify any

' additional or altered dat. . cads Schedule adjustments for July 29, 1993, etc., in time to

' inform the licensee staff prior to closure for day

- Continue discussions and assessment of tier.2 information

, until 5 p.m.

. 5 p.m. Adjourn

, JULY 29 19_93 "

M ACTIVITY 8 a.m.

Plant walkdown to presbected areas such as control and equipment rooms related to specific human actions cf i- importance to IPE (actions to be identified in advance with possible modification based on discussions on July 28,1993)

{ t

, - - 3.k . ,

' i 2-July 29.1993 (continued)

TIME' ACTIVITY 10 a.m. Meet with training personnel, preferably at-the plant simulator ,

Discuss traini.ng practice, control- room issues as applicable, and specific operating procedures 12 Moon Lunch 9

1 p.m. Meet with operations and maintenance personnel to obtain their insights regarding HRA at McGuire 3 p.m. NP.C team breaks to assess progress and identify any additional needs Schedule adjustments for July 30, 1993,.in time to inform licensee staff prior to closure for day Continue discussions and assessment of tier 2 information until 5 p.ni. '

JULY 30, 1993 TIME ACTIVITY 5 a.m. Reconvene with the licensen HRA staff-to resolve any remaining questions Collect copies of necessary documentation Identify follow-up informatien or action requirements 3 - 10 a.m. Debrief-licensee staff and management i 10:30'a.m. Departure g.--

che

" ~

p.

y . ., , -e . -,m.r ,-- -

+

..p-y,, -

McGuire IPE Human Reliability Analysis Step 2 Review I. REVIEW OF OVERAU. HRA PROCESS (TO BE ADDRESSED DAY.1, OURING LICENSEE PRESENTATI0N AND FOLLOWUP DISCUSSIONS)

We request that the licensee be preparsd to review in detail the mathodology employed for each major step of the HRA, and present' ,

complete sample Tier 2 documentation of the process and products of the analysis. It may be useful, but it certainly is not essential, to use the ten-step SHARP process identified in the submittal to guide this review. We ce not expect the licensee to expend a great deal of effort preparing -a formal presentation or general methodology rev.iew. The purpose for these discussieris is for NRC to assess the basic processes and methods actually used by the licensee to accomplish the essential functions of the HRA performed as part of the McGuire IPE:

• Identification of potentially important latent and dynamic human errors for the McGuire plant

• Assessment (screening) of the many possible human actions to identify a manageable set of the more important latent and dynamic errors warranting more in-depth analysis

• Detailed qualitative and quantitative analysis of the important human actions to understand the underlying factors-influencing

, human performance and to quantify the expected performance (HEPs)

• Integration of the HRA into the overall risk assessment

• Documentation of the inputs, activities, rationale, assumptions, and outputs of each step of the analysis

• Assessment of the overall results of the HRA/IPE to identify

, potential /necessary human-related vulacrabilities and, as i appropriate, enhancements.

The intent of this part of the review is to-examine and understand process, not to critique specific HE? values. However, in order to

.. bring specific focus to the review it is requested that the -licenses select one or more examples each of latent errors.and cf dynamic errors, and trace through the ecmplete HRA process using these examples.- -

2 Additional examples may be used to clarify differences in-techniques, data sources, etc. where necesstry. Thr. emphasis should be on plant-specific analysis, data, and assumptions, m

8 t;

L -

II. SPECIFIC ISSUES REMAINING FROM THE STEP 1 REVIEW (To BE ADORESSED DURING DETAILED DISCUSSI0HS ON DAY 1)

The questions belew are intended to provide further clarification of specific issues identified in the Step 1 review. Some of them may be clarified during the methodology review discussed in Secticn I above:

1. Our review has identified a variety of quantification " techniques" for obtaining final HEP values, including at least: the HCR model,

" Engineering Judgment", " historical data"; " operator intseviews",

NUREG/CR-4550 Seauoyah recovery model, the factored model in Eqn. 5.6-1, and arbitrarily 2etermined values, such as the 3.0E-03 value for soms latent errors. Please discuss the rationale for selection of each of the different quantification approaches used for different types of human actions and, in some cases. for different actions of the same type.

2. Why is the HCR model, which was derived using data from " event-based" procedures, and which focuses on diagnosis of events using event-based procedures, appropriate for McGuire operations with symptom-based procedures?

3. page 5.5-4 of the submittal states that in using the HCR model, "The effect of stress, control room equipment layout and other influences on the performance of control room crews can be taken into account by the modification of the median time of the crews." Please identify any instances in which such factors were taken into account in the McGuire HRA use of the HCR model, and explain the technical basis for the evaluation of these factors.

4. Both the submittal and the licensee's response to questions still seem i

to be self contradictory with regard to specifying the quantification approach used for latent errors. Statements in different places seem to say first that most latent errors were quantified using Equation 5.6.1

, (not 5.6.2) in the IPE, tnen later that most latent errors were quantified at the (arbitrarily assumed) value of 3.0E-03, which is at the high end of the range suggested by Equation S.6.1 and within an order of magnitude of the screening value of 0.01. Please describe what was the quantification process for latent errors, what, values were employed, and the basis for selection of the approach / values. Was a numerical screening using the value of 0.01 performed? If so, what ts the basis for selection of that screening value? What is the basis for the licensee's confidence (see page 14 of the response to Step 1 review)

. that, "The consistent use of this value (apparently 3.0E-03] led to the --

e identification of important latent errors."

. 5. The following dynamic actions were evaluated by " engineering judgment","

" engineering and operator judgment", " interviews with operators", etc.

In each case, explain the rationale f:r using these judgments vs. an t

5.

L .

. ....=.,

e established technique, the underlying technical basis for the judgment, the process used for eliciting the judgment, and the qualifications of the judges to estimate human error probabilities:

NNYSSFADHE and NNVESFBDHE - Actions to activate the Standby Shutdown Facility RNUNIT2 REC - Actions to cross-tie Unit 2 nuclear service water upon less of Unit I nuclear service water, HEP-0.1 FIREFLDREC - Actions to cross-connect electrical load to Unit 2 upon control room or cable room fire, HEP-0.05 SAGRCOLDHE - Actions to cool down to RHR conditions following small or medium LOCA and loss of high pressure injection, HEP-0.1 YAGRCOLDHE - Actions to cool down to RHR conditions following a SGTR with high pressure injection unavailable, HEP-0.1 ZWLM2210HE - Closing of motor-operated isolation valves manually following a loss of all ac power, HEP =0.01 .

6. The following dynamic actions were Ovaluated using " historical data".

Explain why this approach was selected. Identify the spacific data used and the analysis conducted to develop an HEP from the raw data.

AC**CG* REC - Actions for restoration of off-site power.

TCFOC01 REC and TCF0002 REC - Restoration of main feedwater.

7. Please previde similar information (specific data used and analysis performed to produce HEPs) for latent actions that were estimated using historical or " plant-specific" data, e.g., JDG001ALHE, JDG0013LHE, and NNVRCPSLHE. Also for " generic values' used for latent errors (e.g.,

ZCILVERLHE and ZCIUPERLHE) please describe the assessment that was made T

to determine that these generic values are appropriate for McGuire.

8. In Section 3.3.5.5 of the IPE, " Human Response for Flood Isclation in the Auxiliary Building," refers to KNOWLEDGE parameters in Equation 5.5-2 and states that this equation was used with various assumptions to calculate the non-response probabilities listed in Table'3.3-3. We find no Ecuation 5.6-2 in the submittal and no obvious alternat.tve that cculd

~

have been a simple typo. Please explain the rationale, assumptions and data sources used to estimate the probabilities in Table 3.3-3. -

i 9. Please identify the source of timing estimates, i.e., the " estimated L.

amount of time the average crew would take to perform the action', for any HEPs quantified using the HCR model.

b E

L

a

~.

l

10. Most analysts find the HCR model to be inapplicable if ths time available is very much greater than the time required. Please explain what considerations were given to applying the HCR model under these conditions (e.g., action FCAHOTVDHE assumes 10 min and 3.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> for the required and available times, respectively).

11. On page 5.5-5 of the submittal it is stated that, "A program is being put into place to collect data on the McGuire simulator as part of the

'living PRA' concept." Page 5.5-7 also refers to a program in which,

" Human reliability analysts from the PRA group will observe operator performance during requalification training." Are these t o references citing the same program? Please discuss the program (s) and any results obtained to date. -

12. Section 3 of the submittal discusses enhancaments identified as a result of the PRA findings. Included are procedure changes for activation of the SSF, and for starting the RCPs during a LOCA. Procedure changes were also under consideration for instructing the operators not to restart RCPs without steam generator tube submergence (to avoid a possible steam generator tube rupture). Please discuss the status of proposed changes and/or evaluations.

13. On Page 2-18 of the submittal, in the paragraph on no containment f ailure, there is a concern that the initiation of the containment sprays could have two adverse effects: (1) the containment atmosphere may no longer be inert following spray initiation, and (2) the spray may damage the containment cue to drawing a vacuum. Plecse comment on this issue raised by the back-end rev' ewers. Are there procedural cautions for the operators?

14. A second potential issue noted by the back-end reviewers is that without specific coerator intervention, there is a potential for an uncontrolled hydrogen burn upon restoration of power in the case where hydrogen has I accumulated in the containment and there was no power to the igniters.

Is this a valid concern? If so, what action has been taken to address j the concern?

w.

15. Please identify and discuss, per reporting guidance in NUREG-1335, any sequences that, but for low human error rates in recovery actions, would have been above the applicable core damage frequency screening criteria (i.e., any sequence that drops below the COF criteria because the frequency has_been reduced by more than an order of magnitudo credit taken for human recovery actions, t

L r

oa r]

III. PLANT WALXDOWN (MORNING OF DAY 2) ,

It is likely that the licensee presuntations and detailed discussions during Day I will-define, or redefine, a list of specific actions, issues, plant equipment operation, etc. to be addressed in the plant walkdowns. Some items may be clarified in the discussions without addressing them in the walkdowns, and some new items may arise. Control room manipulations would be better reviewed in the simulator, if access to the simulator and instructors is possible. At this time, candidate human error events to be addressed during the walkdowns (with detailed documentation examined separately) are:

Actions to activate the Standby Shutdown Facility and provide makeup using the SSF pump (NNYSSFADHE/NNVSSF3DHE)

• Failure to cross connect equipment from Unit 2 or to locally operate circuit breakers during a fire in che control room or .

cable room which disables redundant trains of equipment (FIREFLOREC)

• Actions related to isolation cf in interfacing systems LOCA ,

(ISLOCA10HE and ISLOCA2DHE)

Acticns to cross-famest Unit 2 auclear service water so Unit I during a LOCA (@ LOUT 2 REC or a transient (yldfT2 ten that involves complete toss of Unit I nuclear service water.

• Actions to prepare for feed and bleed (including RVIBACXDHE and RVIPORVDHE)

.' Actions to aggres-tvely cool down to RHR conditions following a small or median LOCA t,SAGTtCOLDR) or a SGTR (YAGRCOLDHE) when high

, pressure injection is not 'available. -

• Manual closurs cf MOV 1/L321 and ilL322 following a loss of all ac power (ZWLM2210HE).

t A.

'(

{ l

t

c. . ;_:.. _---- .-.

1 IV.  !

. MEETING WITH TRAINING / PROCEDURES PERSONNEL (MORNING, DAY 2)  ;

The meeting with training staff ideally should include examination of control room actions in the simulator.

those control room actions noted above andSpecific actions of interest are discussions with the HRA/IPE team as being/or identified during detailed significant.

V.

MEETING WITH' OPERATIONS AND MAINTENANCE PERSONNEL (AFTERNOO The meeting with operations and maintenance personnel is intended

• primarily to discuss involvement of these organizations in development of the IPE and their use or potential use of IPE results.

Supervisory / management staff are probably the appropriate level to address most issues. If there is an HFES Coordinator, or someone involved in similar human-related root cause analysis, it probabl be useful to include that individua1 in some of the discussions. y would 9

i 6

O L

i gs

~

e

o I

Mr. T. C. M:Meekin Duke Power Company McGuire Nuclear Station cc:

Mr. A. V. Carr, Esquire Mr. Dayne H. Brown, Director Duke Power Company Department of Environmental, 422 South Church Street Health and Natural- Resnurces Charlotte, North Carolina - 22242-0001 Division-of Radiation P"9:ection P.-O. Box 27687-County Manager of Mecklenberg County Raleigh, North _ Carolina 27611-7637 720 East Fourth Street Charlotte, North Carolina 2S202 Mr. Alan R. Herdt, Chief Project Branch #3 Mr. R. O. Sharpe U. 5. Nuclear Regulatory Commission-Compliance 101 Marietta 3treet, NW. Suite 2900

, Duke Power Company Atlanta, Georgia 30323 McGuire Nuclear Site 12700 Hagers Ferry Road Ms. Karen-E. Long Huntersville, NC 28078-1985 Assistant Attorney General North Carolina Departmert of J, Michael McGarry, III, Esquire- Justice Winston and Strawn P. 0. Box 629-1400 L Street, NW. Raleigh, North Carolina 27602 Washington, DC 20005

- Mr.' G. A. Coop Senior Resident Inspector Licensing - EC050

._ c/o V. S. Nuclear Regulatory

- Duke Pcwer Company Commission P. 0. Box 1005

12700 Hagers' Ferry Road Charlotte, North Carolina 28201-1005

- Hunte-sville, North Carolina 23078

" Regional Administrator, Region II.

Mr. T. Richard Puryear- U.S. Nuclear Regulatory Commission c Nuclear Technical Services Manager- 101 Marietta Street, NW. Suite 2900

, Carolinas District _

Atlanta, Georgia -30323 o

Westinghouse' Electric Corporation .

- P. O. Sox 32517 Charlotte, North Carolina 28232 -

i - Or, John M. Barry i

- Mecklenberg: County-

- Department:cf Environmental L~ ~

Protection

, -700 N. Tryon Street p  : Charlotte, North Carolina 28202 l

1 em i l w

I I .

• - - - - . _ _ - - - . _ _ _ _ _ _ _ _ _ _ _ _ _ .- y _ c , , - y-.

l APPENDIX B

SUMMARY

OF SITE VISIT ACTIVITIES 1

l l

l l

l l

e l

l L.

APPENDIX B

SUMMARY

OF SITE VISIT ACTIVITIES B.1 Intmductory Meetinc With McGuire Staff After checking in at the site, badging, whole body count, and a brief visit with the NRC Resident Inspector, an introductory meeting was held with McGuire staff and management. Attendees at the meeting are listed in Table B. I. The NRC Team i rader summarized the objectives, scope and intent of site visit. INncan Brewer, Senior Engineer, Nuclear Generation, who led the front Table B.1 List of Attendees for Opening Meeting NAME ORGANI7ATION P. Haas Concord Associates, Inc. (NRC Team)

L. Firehaugh Nuclear Services / OPS J. Painter McGuire/ OPS L. Azzarello Oconee Engineering K. Canady Nuclear Engineering R. Cross McGnire Regulatory Compliance P.Vu McGuin ? gulatory Compliance D. McGinnis McGuire/ Operations Training P. Guill McGuire Regulatory Compliance R. Newtnan McGuire/ Operations Training D. Bumgardner McGuire/ OPS Leo Kachnik GS/NE/PRA P. Abraham Duke /PRA Group G. Gilbert Duke /McGuire Safety Assurance K. Hashim Puke /McGuire HPES Coordinator D. Brewer Severe Accident Analysis E. Chow USNRC/RES/DSIR/SAIB end analysis for Catawba and the back end analysis for McGuire and has been extensively involved with Duke's PRA efforts, pmvided a summary overview of the Duke PRA process and history. His presentation addressed planning, team training, the continuous internal review process, the IPE model solution process, sensitivity studies, exterr..d review, and documentation / reporting. It also summarized the history of the overall effort starting with the original Oconee PRA presented as NSAC 60 in 1984, through the Catawba IPE in September, 1992. Considerable emphasis was placed on the fact that the McGuire IPE is not a new PRA, but a revision of the original McGuire PRA in 1984, and that the Duke process has evolved through the various versions of the Oconee, McGuire and Catawba PRAs or IPEs, which have consecutively built on the experience gained.

b L .

Leo Kachnik' the principal person performing the McGuire IPE HRA, summarized the McGuire IPE HRA process, including implementation of the SHARP structure, the taxonomy of human errors used, the review of operational history, quantification techniques used for latent and dynamic errors, and integration of HEPs into the plant model.

B.2 Interviews and Discussions With IPE Team and Support Staff A major portion of the site visit was devoted to intensive discussions with the key McGuire personnel responsible for the IPE and the HRA and conctment and iterative review of supporting documentation. These discussions evolved from general discussions of overall methodology, 'o .

interactive discussions of specific issues remaining unresolved from the step 1 review, to detailed examination of calculations and data sources for specific HEP estimates. Detailed tier 2

. documentation was provided by the Duke staff and was reviewed by the Concord team member.

Three key Duke personnel were involved in all of the discussions: Leo Kacimik, the primary HRA analyst; Duncan Brewer, who ; ad lead responsibility for the McGuire back end analysis and the Catawba front end analysis; and, Mr. P.M. Abraham, Head of the Duke Severe Accident Group. All three were intimately involved with and familiar with the Duke PRA process and both McGuire and Catawba PRAs.

Also present for almost all of the discussions was Mr. Jim Painter, cunently Nuclear Instructor, on loan to the emergency procedures / abnormal procedures group. Mr. Painter is a former shift supervisor who has experience with operations, training and procedures at McGuire. He provided input on plant operations and training throughout the IPE. Mr. Len Azzarello, currently in the Oconee Engineering Department, participated in the first aftemoon's discussions. Mr. Azzarello s has been involved wkh the Duke PRA/HRA efforts since the initial Oconee PRA. He was the primary individual responsible for many of the formative decisions such as selection of the HCR model, and provided support for McGuire and Catawba HRAs.

Other important contributors to the discussions were Mr. Ien Firebaugh, Nuclear Services / Operations and Mr. Richard Casler, Technical Manager, Nuclear Services. Mr.

s Firebaugh is the licensed SRO who was the primary source of " expert judgment" regarding critical HRA parameters, such as estimated times to accomplish operator actions. He participated I in the initial discussions and was available on demand throughout the visit to answer specific

!- questions. Mr. Casler heads the Human Performance Excellence Program, a Duke corporate wide effort to improve human performance. He presented an overview of the program, focusing on activities related to procedure enhancement.

B.3 Information Audited b The licensee had prepared two notebooks of information in advance which addressed some of the general and specific issues identified in the audit plan. In addition to copies of the presentation 7 overheads, the first notebook contained a copy of the HRA section (Chapter 5) of the McGuire PRA Rev 0 (the origirmi " luire PRA) and Rev 1 (the IPE update), the HRA section of the

, Catawba PRA Rev C r- ns and Duke responses from the Step 1 McGuire IPE HRA review, and NRC qut r ake responses on the Oconee IPE HRA. These were used to

~

compare and demon < 4tinuously evolving Duke PRA process and provide ready u reference for discuss L

0 l

The second notebook, used during the more detail discussions with the HRA team, corppiled tier 2 information on each specific dynamic (operational) human error (DHE) and recovery (REC) identified for review in the NRC audit plan. For each human action, the notebook presented:

• a narrative description of the action and the HEP quantification

• a summary of pertinent information used to assess the human action (essentially an edited and typed version of raw HRA data sheets that comprise the primary tier 2 documentation for the i

HRA)

+ excerpts from the appmpriate emergency / abnormal procedure related to the action

+ computerized output from the PRA code (RMQS) showing pertinent cutsets The licensee's HRA team members had also prepared direct responses to each of the 15 specific questions / comments in the NRC audit plan. These served as a basis for discussion of these specific unresolved issues from the step 1 review.

As indicated above, the central documentation of assumptions and technical basis for the HRA maintained by Duke, outside of the IPE submittal and the PRA repon, is the HRA data sheets.

These consisted of handwritten entries on a pre-printed form. These handwritten sheets have recently been entered in computer form, though some of the detailed handwritten notes are not always included in full in the computer file. The NRC team reviewed approximately two thirds of the computerized file and a substantial sampling (approximately 20%) of the handwTitten forms to verify the nature of the information considend and completeness of the documentation.

In addition to these prepared materials and basic tier 2 documentation, the Duke staff pmvided numerous documents on demand during the course of discussions to substariate or further clarify information presented in t'ie submittal and/or presented in the audit discussions. These included:

- additional emergency / abnormal procedures, including key procedures related to some of the human acdons addmssed during the plant walkdown

- PRA team memos pertinent to decisions / assumptions about HRA

. samples of system analyst presentations to the PRA team for two systems (safety injection

' and containment spray) which illustrate the format for the systems reviews and the type and level of information discussed. These presentations and the related memos documenting I feedback and comments constitute a significant portion of the Duke documentation for the HRA modeling and the overall PRA modeling

- documentation of plant-specific data sources, such as diesel generator failure data

- documentation of follow-up by appropriate Duke management / programs on enhancemen:s identified in the IPE, typically procedure change records or documentation from the Problem Investigation Process, which has been initiated to coordinate all changes in the McGuire piant

- a memo documenting chlculations which provided the ba:is for a reduction in risi. expected from reducing the test in;erval for the FW$T level transmitters ' rom 18 months to 6 months

. miscellaneous documentation of reference soumes. supporting eticulations, etc. pertinent to

{ quantification of specific HEPs.

, B.4 Walkthroughs and General Clpervation of Facilities

~

The detailed audit plan transmitted to Duke prior to the visit hr J identified a number of specific human actions which wcre of particular interest as a focas for the plant walkthroughs. Emphasis b

l

was placed on key recovery actions outside of the control room. It was anticipated that it would be difficult ~to examine control room actions in the actual control room and that most of those would be assessed in the simulator. However, since one unit was not operating at the time of the visit, and since the simulator was heavily scheduled for training, control room actions were examined in the control room. Further, the original planned tin 4e of two hours for the walkthroughs was extended to almost four hours, and additional actions were reviewed. All actions on the list identified by the NRC review team, plus additional actions that were identified in a Duke sensitivity study as among the 10 most important human error contributors to plant risk, were walked through. The walk through was guided by two Duke operations staff (including J. Painter). The two primary IPE/HRA leads (D. Brewer and L. Kachnik) accompanied the NRC team and provided continued background information relating the plant equipment etc., to the IPE sequences and modeling. Procedures were reviewed in the context of the plant setting, and copies were provided for later !a depth review as requested. Specific human actions examined are listed in Table B.2 A visit was conducted to the simulator facility for general observation on the moming of the third day. The final stages of a full training exercise were observed and a limited walkthrough of the simulator was conducted. No dynamic exercises were planned or executed for purposes of the audit. Information was obtained on the general design and use of the simulator and on potential future plans for training related to accident management. The observation of the training exercise provided input of use to assessing the reasonableness of operator actions in response to accidents (e.g., information about administrative practice such as the designated procedure reader)

B.5 Debriefing A brief closcout meeting was held with Duke management and staff to summarize the initial findings and major conclusions from the site visit. Individuals attending that meeting are listed in Table B.3.

m s

d W

3.-

t e.-

s  :

u Ih

Table B.2 Operator Actions Reviewed During Plant Walkthroughs IDENTIFER ACTION

1. TRECIRCDHE Failure to establish high pmssure recirculation

2. FCAHOWWDHE Failure to switch CA suction to the hotwell

3. WRNPBFSDHE Failure to start RN pump 1B

4. RNUNIT2 REC Failure to align Unit 2 RN to Unit I with offsite power available

5. NNVSSFBDHE Failure to initiate standby shutdown system operation in time (plant power not available)

6. FCATRHODHE Failure to manually throttle auxiliary feedwater flow

7. WRNRVBKREC Failure of RV to back up RN 8, NNVSSFADHE Failure to initiate standby shutdown system operation in time (plant power available)

9. SMANRECREC Failure to recover auto switchover failure

10. TFBLD01DHE Failure to establish frml and bleed cooling

11. FIREFLDREC Failure to provide makeup from SSF standby makeup pump to avert pump seal LOCA

12. ISLOCA1DHE Failure to isolate an ISLOCA through the ND pump discharge cold leg injection valves in time

13. ISLOCA2 DIE Falure to isolate relief valves after they open due to an ISLOCA

14. RNLOUT2 REC Failure to cross-connect Unit 2 RN to Unit I during LOCA

15. RVIBACKDHE Failure to manually align nitrogen to PORVs

16. RVIPORVDFE Failure to manually restablish instrument air to the pressurizer PORVs f 17.~ SAGRCOLDDHE Failure to aggressively cool down to RHR conditions after small or medium LOCA

. 18. YAGRCOLDDHE Failure to aggressively :ool down to RHR condition! ifter a SGTR L.

r I

l .

Table B.3 List of Attendees for Debriefing NAME ORGANIZATION L. Kunka Compliance /Eng/McGuire -

D. Brewer Severe Accident Analysis L. Kachnik Severe Accident Analysis Group J. Painter OPS / Procedures D. McGinnis Operations Training T. Cooner NRC Resident Inspector M. Geddie MNS/STAMGR P. Haas Concord Associates. Inc. (NRC team)

E. Chow USNRC/RES/DSIR/SAIB K. Canady Duke /I!uclear Engineering P. Abraham Duke - Engineering Supervisor M. Pacetti Duke /MNS/ Mech / Nuclear e

b 4

I [

-i

. L.

• f 3

r

-- -