ML17349A418

Step 2 Review,Turkey Point Units 3 & 4 IPE Submittal, Human Reliability Analysis

ML17349A418
Person / Time
Site:	Turkey Point
Issue date:	12/31/1991
From:	Bovell C, Haas P CONCORD ASSOCIATES, INC.
To:	NRC
Shared Package
ML17349A413	List: ML17349A411 ML17349A414 ML17349A415 ML17349A417 ML17349A418
References
CON-NRC-04-91-069, CON-NRC-4-91-69 CA-TR-91-19.01, CA-TR-91-19.01-1, NUDOCS 9210210160
Download: ML17349A418 (63)

Text

ENCLOSURE 4

'0 CA/TR-91-19.01-1

. STEP 2 REVIEW TURKEY. POINT UNITS 3 AND 4 IPE SUBMITTAL HUMANRELIABILITYANALYSIS P.M. Haas and C.R. Bovell Prepared for U.S. Nuclear Regulatory Commission Office of Nuclear Regulatory Research Division of Safety Issue Resolution

December, 1991 CONCORD ASSOCIATES INC.

Systems Performance Engineers 725 Pellissippi Parkway KnoxvilleTN37932 Contract No. NRC-04-91-069 Task Order No. 1 92102101b0 921015 tj PDR ADOCK 05000250 P

PDR,i

-I

0

TABLEOF CONTENTS

1.0 INTRODUCTION

1.1 Background

1,2 Scope, Objectives and Organization of this Report 1.3 Turkey Point HRA Review Objectives and Approach 1.3.1 Objectives of the HRA Review 1.3.2 HRA Review Approach

.1.4 Summary of Findings, Conclusions and Recommendations 2.0 STEP 2 HRA REVIEW ACTIONS 2.1 Collection and Assessment of Information, Preparation for Site Visit 2.1.1 KickoffMeeting at NRC 2.1.2 Review of Available Information 2.1.3 Site Visit Plan 2.2 Site Visit 2.2.1 Review of HRA Process 2,2.1.1 Definition 2.2.1.2 Screening 2.2.1.3 Breakdown 2.2.1.4 Representation 2.2.1.5 Impact Assessment 2.2.1.6 Quantification 2.2.1.7 Documentation 2.2.2 Plant Walkdowns 2.2,3 Visit to Training Facility and Interviews with Training and Operations Staff 8

8 11 11 12 13 14 14 15 16 18 2.3 Tier 2 Documentation Review and Final'eport 19 2.3.1 Review of Additional Tier 2 Documentation 2.3.2 Final'eport 19 19

0

TABLEOF CONTENTS (CONT'D)

3.0 CONCLUSION

S AND RECOMMENDATIONS 21 3.1 Findings and Conclusions 3.1.1 Licensee Met the General Intent of Generic Letter 88-20 3.1.2 'nalytic Processes Used for HRA Were Generally Acceptable 3,1.3 HRA Summary in the IPE Submittal is Inadequate 3.1.4 Documentation of Site-Specific Analysis Needs Improvement 3.1.5 Personnel Changes Impacted HRA Knowledge Acquisition 3.1.6 Clarification of Some Methodology is Required 3.1.7 Commendable Progress Has Been Made Toward Application of IPE Results and Involvement of Groups Outside of PRA/PE 21 21 21 21 22 22 23 24 3.2 Recommendations 24 APPENDIX A APPENDIX B LETTER REPORT WITH PROPOSED AUDITAPPROACH EXAMPLES OF HRA RECOVERY ANALYSIS WORK SHEETS

e

LIST OF TABLES Table Title.

Major Turkey Point IPE Documentation and Background Sources Reviewed Prior to the Site Visit 9

V Schedule for Site Visit to Turkey Point Units 3&4 Data Sheets Included in the HRA Recovery Analysis Work Package Items Identified for Examination During Plant Visit/Walkdown 1

Documentation Reviewed After the Site Visit 10 13 17 20

0

=

Background===

The NRC Generic Letter 88-20 (Ref. 1) requires licensees of all nuclear power reactor facilities to conduct an Individual Plant Examination (IPE) oftheir plant to identify any plant-specific vulnerabilities to severe accidents and to report the results to the Commission.

Per Ref. 1, the general purpose of the IPE is for each utility:

(1) to develop an appreciation of severe accident behavior, (2) to understand the most likely severe accident sequences that could occur at its plant, (3) to gain a more quantitative understanding of the overall probabilities of core'damage and fission product releases, and (4) if necessary, to reduce the overall probabilities of core damage and fission product releases by modifying, where appropriate, hardware and procedures that would help prevent or mitigate severe accidents.

Guidance to the utilities for submitting their reports on the IPE to the NRC was provided in NUREG-1335 (Ref. 2).

AllIPE submittals are to be reviewed by the NRC staff to determine ifthe licensee met the intent of Generic Letter 88-20. This staff review is being accomplished in two steps.

In the first, or "Step 1" review, NRC staff evaluates the licensee's submittal, gathers important IPE information, and, as necessaxy, formulates review questions for which a response is requested of the licensee.

Allsubmittals willundergo a Step 1 review.

For some plants, a more detailed "Step 2" review may be required to provide additional information upon which the NRC staff can base a final evaluation of the licensee's IPE process.

The NRC staff is supported in these Step 2 reviews by contractors specializing in three separate areas - "Front-end" analysis, Human Reliability Analysis (HRA),'and "Back-end" analysis.

Each contractor willconduct an assessment of the IPE, including a site audit, and submit a report to NRC which willbe used by NRC staff to make its final evaluation.

Draft NRC guidance (Ref. 3) states that the contractor assessment is to include:

(1)

An assessment of any limitations or weakness in the licensee's IPE methodology identified by the NRC staff during the Step 1 review.

(2)

An evaluation of any inconsistencies or shortcomings associated with the accident frequency estimates based on previous PRA experience.

(3)

An audit of 'the licensee's fault tree

models, human event

trees, reliability block diagrams, or other analytic process, and an evaluation of any inconsistencies with known and accepted methods.

(4)

A determination whether the analytic methods used by the licensee are capable of identifying decay heat removal vulnerabilities at the level expected for resolution of USI A-45.

(5)

An evaluation of the licensee's process used to identify, eliminate, or reduce the effects of vulnerabilities.

(6)

An evaluation of the licensee's response to" identified vulnerabilities, and identification of any vulnerabilities that may appear to inquire further analysis of evaluation.

(7)

An assessment of the dominant contributors to. severe accidents,,

including the strengths and weaknesses of unique design features.

1.2 Scope, Objectives and Organization of this Report This report is the contractor report on the HRA portion of the IPE submittal by Florida Power and Light Company (FPL) for Turkey Point Units 3 & 4 (Ref. 4). The Turkey Point IPE is the first to undergo a Step 2 review.

The scope of the review, and of this report, is limited to HRA, though HRA is. closely interrelated with the Front-end and the Back-end analyses.

In panicular, specific human actions of importance for closer examination were selected in close cooperation with the Front-end analysis contractor.

The objectives of this report are to document:

(1) the general approach used to conduct the Step 2 HRA review, (2) the specific audit activities conducted, (3) results of those audit activities, and (4) findings, conclusions and recommendations to NRC staff for their final evaluation.

C Objective 1 is addressed in the subsequent paragraphs of Section 1.

Objectives 2 and 3, related to the activities and results of the review are addressed in Section 2.

Findings, conclusions and recommendations are presented in Section 3.

For convenience, the major findings, conclusions and recommendations are briefly summarized in the final paragraphs of this introduction.

0

1.3 Turkey Point HRA Review Objectives and Approach Five precise HRA review objectives were evolved based on the seven areas of assessment explained in Section 1.1.

Three primary activities, described. in Section 1.3.2., were also prepared to attain the preceding objectives.

12.1 Objectives of the HRA Review Based on the seven areas of assessment ide'ntified above and other guidance from Reference 3 and &om NRC staff, the following specific objectives were identified for the Turkey Point HRA review:

(1)

Assist NRC staff in obtaining resolution of issues raised by the Step 1 review.

(2)

Assess the analytic processes used by FPL to perform the HRA; this includes identification of potentially important human

actions, screening, qualitative and quantitative

analysis, evaluation of impact, and any system improvements to reduce or eliminate vulnerability..

(3)

Identify any potential weaknesses in'the analysis and evaluate specific items as necessary to determine iffurther assessment by the licensee is required.

(4)

Assess to determine the degree to which the licensee was involved in the HRA.

(5)

Assess the documentation of the HRA process, assumptions, data sources, decisions, calculations, and results,to assur'e that the knowledge and information gained was accurately and appropriately recorded to provide a basis for future assessments such as accident management planning, and to assure continuity as staff changes and system changes occur in the future.

h 1.3.2 HRA Review Approach To accomplish these specific objectives, the Turkey Point Step 2 HRA review was organized into the following three primary activities:

(1)

Collection and assessment of information, and preparation for a site visit to conduct an audit of the HRA.

(2)

Conduct of the site visit, including document review, interviews, and plant walkdowns.

(3)

Evaluation of detailed "Tier 2" information, and preparation of a final report summarizing review activities, results, findings, conclusions and recommendations.

Early in the site visit, it was determined that a substantial difference existed between the HRA methodology summarized in Ref. 4 and the analysis which was actually performed. A rather detailed analysis of operator recovery actions was'erformed and was not explained.

On the other hand, an interesting approach to analyzing and quantifying pre-initiator maintenance, test and calibration (MTC) human actions was presented in a fair amount of detail in Ref. 4, but was never used.

The licensee provided substantial documentation of the HRA recovery analysis (Ref., 5),

It was reviewed in detail after the site visit.

The licensee has been requested to revise the sections of Ref. 4 dealing with HRA (primarily Section 3.4 and 3.5).

The post-visit review of the Tier 2 information provided by the licensee resolved the majority of the issues raised previously by the team and raised some others.

Remaining unresolved issues, or simply additional points ofclarification, have been addressed to the licensee through transmittal of a final set of questions.

The points addressed in that transmittal are included in this report as conclusions, findings and/or recommendations.

1.4 Summary of Findings, Conclusions and Recommendations The significant findings, conclusions and recommendations from the Step 2 HRA review are discussed in Section 3.0 of this report.

For convenience they are summarized below:

Findin s and Conclusions It is our assessment that the licensee has satisfied the general objectives pertinent to HRA identified in Generic Letter, 88-20, as interpreted in item 4, Section 1.3:1 above, and thereby has met the intent of the Generic Letter with re ard to the HRA ortions of the IPE.

2.

Overall the anal ic rocesses used b the a licant were ade uate to meet the ob'ectives of the Generic Letter and were consistent with rocesses used in other PRAs.

A number of specific issues remain open, and have been addressed in a transmittal from NRC to the licensee requesting additional information and clarification.

These issues are embodied in the recommendations summarized below and discussed in more detail in Section 3.2 of this report.

3.

A revision to the kev HRA ortions of the IPE submittal is re uired for the submittal to accuratel reflect the anal sis that was conducted.

The two major discrepancies in discussion ofmethods used

for MTC and for recovery actions were noted above.

Additional specific modifications may be required as part of the licensee's response to NRC questions transmitted after the site visit.

There are a number of general statements in the IPE Submittal and in supporting documentation which suggest that consideration was given to plant-specific information, design features, procedures, administrative controls, etc.

There is also anecdotal information Rom the interviews conducted at the site that indicates reasonable attention was given to site-specific issues.

However, there is a lack of documented anal sis showin how non-lant s ecific data was inter reted or ad usted for s

cific conditions at Turke Point., Examples of items referred to in general but not well documented include man-machine interface, symptom-based procedures, and focus of the training process.

A number of additional specific items have been identified for which documentation is inadequate to demonstrate site-specific adaptation of "generic" information or data, and requests forclarification or additional documentation have been made of the licensee by NRC.

These are discussed in Section 3.0 of this report.

Chan es of kev ersonnel in particular the departure of the lead FPL staff member for HRA after completion of the analysis and prior to preparation of the IPE submittal, are believed to have been a

si nificant detriment to FPL "internalizin " the knowled e and ex erience ained from the HRA.

His absence certainly made it difficultto assess the degree of understanding and direct involvement by the licensee (vs. the subcontractor HRA specialists).

Responsible FPL management (within the PRA group) recognize the need for strengthening in-house capabilities for HRA, though specific actions to do so were not noted.

Further difficulties with personnel changes, or communication, between the two subcontractor HRA specialists contributed to some confusion in reporting what was accomplished, and perhaps to a "loss" of knowledge gained from the analysis.

Additionalin utfromthelicenseere ardin someoftheunderl in assum tions methods and data sources used in the HRA is

~re uired.

While the overall structure for documentation of the HRA that is defined in the HRA procedure appears to be quite satisfactory, there are specific lapses in documentation, and additional information is needed to justify application of certain data sources.

This additional information is required before final acceptance of the IPE can be made by NRC.

Significant specific points have been identified as recommendations in Section 3.0 of this report.

7.

The licensee's PRA rou has made some commendable advances toward involvin other FPL or anizational units in a lications of results of the PRA/IPE rocess.

Two notable examples identified during the NRC site visit were (1) a study on Maintenance Risk Management, and (2) the'overall interaction with the training staff, who identified use of IPE results to help select malfunctions/scenarios for training as one practical operations use of the results.

Recommendations It is recommended that the following actions be considered by the licensee before final acceptance/approval of the IPE Submittal:

1.

The HRA portions of the summary discussions. in the IPE Submittal (Ref. 4), primarily Sections 3,4 and 3.5, should be rewritten to accurately reflect the analysis that was actually performed.

The licensee should be required to precisely document the technical basis for assumed screening values of 3.0E-3 and 3.0E-4 for pre-initiator actions.

These values are low compared to screening values typically used in PRAs, and are more representative of nominal values.

The licensee did, in effect, use these as nominal values, since no analysis was done beyond the screening.

The level of analysis conducted, or at least discussed in the IPE documentation, does not appear to justify use of these nominal values.

Additional specific information required for clarification or resolution of questions raised during the audit are listed in the recommendations in Section 3.0 of this report.

3.

The licensee should be required to provide further demonstration to NRC of site-specific

. consideration of human actions.

This demonstration should be requested at two levels:

(1) summary discussion in the IPE submittal of site-specific issues addressed and general findings about the important factors impacting human performance, and (2) specific responses, not necessarily included in the IPE submittal, to NRC issues raised during the audit.

These general and specific areas of inquests for further demonstration/documentation are identified in Section 3.0 of this repo'it.

4.

The., licensee should be required to provide additional clarification/explanation for two specific methods/sources used in the HRA, as identified in the HRA Recovery Analysis Work Package (Ref.

5), namely, SAICfI'HERP and the reference listed as WCAP-11992.

2.0 STEP 2 HRA REVIEW ACTIONS As noted above, the Step 2 Review involved completion of three major activities:

(1)

Collection and Assessment of Information and Preparation for the Site Visit (2)

Site visit, including review of documentation, interviews, and plant walkdowns (3)

Evaluation of results and finding, development and documentation of conclusions and recommendations in a final report.

2.1 Collection and Assessment of Information, Preparation for the Site Visit KickoffMeeting at NRC Prior to the site visit, the team reviewed all available IPE materials and examined all relevant HRA information.'fter thorough analyzation,'ssessments.

of materials were made and a detailed itinerary for the site visit was prepared (See Table II).

2.1.1 The initial activity for the Turkey Point IPE review was a "kickoff'eeting, held October 3, 1991 at NRC/RES offices in Rockville,'hat included NRC/RES staff, NRC/NRR staff and contractors for all three IPE areas.

A copy of the licensee's IPE submittal was transmitted to the NRC contractors prior to the meeting.

The submittal was briefly reviewed at the meeting, along with guidance provided by Generic Letter 88-20 and NUREG-1335. Updated guidance documents for Step 1 and Step 2 reviews were provided and discussed.

NRC Step 1 questions submitted to date were reviewed, along with the licensee responses, and additional questions were generated for transmittal to the licensee prior to the site visit. General plans for the Step 2 review and specific plans and schedule for the site visit were generated.

2.1.2 Review of Available Information Subsequent to the. kickoffmeeting, all IPE materials received at the meeting were reviewed.

Items pertinent to HRA were identified and examined in detail.

Additional background information was acquired and reviewed, and pertinent basic reference sources were scanned to refresh and/or update familiarity with important background information. A listing of the most significant documentation and background sources reviewed in preparation for the site visit is provided in Table I.

2.19 Site Visit Plan Based on results of the Step 1 Review, discussions at the kickoff meeting, review of all available information collected during and after the kickoff meeting, and additional independent assessment of the IPE Submittal, a detailed plan was prepared for conducting the site visit and audit of licensee documentation.

Additional information required from the licensee was identified and'forwarded to NRC for transmittal to the licensee.

Since the IPE Submittal indicated. that the licensee had followed, at least in principle, EPRI-'s Systematic Human Action Reliability Procedure (SHARP) (Ref. 6), a series of questions paralleling the basic elements of SHARP was developed as an aid for systematic evaluation of the licensee's HRA methodology.

Specific human actions for which more specific investigation and/or walkdown was required were identified in concert with the contractors for Front-end and for Back-end analysis.

Individuals with whom interviews were desired were identified by position. A letter report summarizing the proposed site visit plan was documented for NRC, and was in turn transmitted to the applicant.

A copy of that letter report is presented as Appendix A to this report.

2.2 Site Visit The site visit by the team was conducted over a two and one-half day period, November 19-21, 1991.

The overall 'schedule for the visit is shown in Table II. HRA team members included one NRC staff member and one Concord staff member.

After badging and an introductory meeting with FPL staff and management (including the Plant Manager and Operations Manager), the primaty activities by the two HRA team members consisted of (1) a review of HRA methodology and data sources with HRA staff, (2) a plant walkdown, including assessment of plant equipment associated with specific human actions identified in the HRA, and (3) a visit to the simulator facility, which included an observation of a requalification training exercise and interviews with key training and operations support staff.

Debriefing discussions with FPL HRA staff and a full team debriefing with FPL staff and management concluded the visit. A summary of activities and results in each of these three areas is provided in the paragraphs below.

2.2.1 Review of HRA Process Fairly extensive discussions with the FPL staff regarding the HRA methodology and data sources were conducted early in the visit, and later after the walkdowns and simulator visit.

The set of general questions for licensees using SHARP (see Appendix A) was used as a guide to structure. the discussions.

The summaries of discussions and results below parallels the seven major paragraphs in that question set, which are the seven basic process steps of

Tab1e I Major Turkey Point EPE Documentation and Background Sources Reviewed Prior to the Site Visit 1.

Turkey Point Units 3 &4 PRA IPE Submittal (Ref. 4) 2; IPE Generic Letter 88-20 (Ref. 1) 3.

IPE Generic Letter 88-20 Supplement No. 1, August 29, 1989 4.

NUREG-1335, Individual Plant Examination: Submittal Guidance, (Ref. 2) 5.

NRC staff questions to FPL from the Step 1 review, FPL responses to those questions, additional questions generated during the kickoff meeting, copies of NRC staff notes on HRA portions of Step.

1 review 6.

Turkey Point Units 3 &4 Interim Human Reliability Analysis (Ref. 11) 7..USNRC Draft Step 2 Review Guidance Document (Ref. 3) 8.

USNRC Draft Step 1 Review Guidance Document (Unpublished) 9.

EPRI NP-3583, "Systematic Human Action'Reliability Procedure (SHARP), (Ref.

6) 10.

NUREG/CR-2254, "AProcedure for Conducting a Human Reliability Analysis for Nuclear Power Plants," May, 1983 11.

NUREG/CR-4772, Accident Sequence Evaluation Program Human Reliability Analysis Procedure," (Ref. 7) 12.

NUREG/CR-2815, V1 and V2, R1, "Probabilistic Safety Analysis Guide," August, 1985 13.

NUREG/CR-4550-V1, "Analysis ofCore Damage Frequency fromInternal Events:

'ethodology Guidelines, Volume 1," September, 1987 14.

NUREG/CR-4835, "Comparison and Application of Quantitative Human Reliability Analysis Methods for the Risk Methods Integration and Evaluation Program (I~P)," January, 1989 15.

NUREG/CRA834/1 of 2, "Recovery Actions in PRA for the Risk Methods Integration and Evaluation Program (IMIEP), Volume 1," June, 1987 16.

NUREG/CRP762, "Shutdown Decay Heat Removal Analysis of a Westinghouse 3-Loo Pressurized Water Reactor

Table ZX Schedule for Site Visit to Turkey Point Units 3&4 Tuesday, November 19 Arrive at Site Check-in; Site Access Training 10:00 AM 10:15 AM Kickoffmeeting with management Brief presentation by Team Leader Agenda planning session 10:30 AM 11:00 AM 12:00 N 1:00 PM 4:00 PM 5:00 PM Licensee presentation, overview of site and overview of IPE effort Break into separate Group discussions Lunch Separate group discussions continue Groups combine to discuss next day's agenda and information needs Adjourn Wednesday, November 20 8:00 AM

. Groups meet to review Day's Agenda 8:15 AM Plant walkthroughs 12:00 NLunch 1:00 PM'eet with FPL HRA team, obtain and review Tier 2 documentation 3:00 PM Visit simulator facility, interview training staff, operations support staff, obtain training documents and procedures 5:00 PM Adjourn Thursday, November 21 8:00 AM Meeting with NRC team to discuss findings and need for follow-up information 8:30 AM Meeting with FPL HRA team and other IPE members to discuss Gndings and information needs, obtain additional input 10:30 AM Closing meeting and exit 10

SHARP. These summaries represent our best understanding of the licensee's process, based on the review of materials and the discussions during the site visit.

2.2.1.1 Definition The process used to ensure that key human actions were included in the logic structures used for the IPE was primarily interactive review by the HRA specialists of the logic constructed by the systems analysts.

Basic sources of information for the HRA team, in addition to system analyst results and input, were said to include the training syllabus for control room and ex-control room personnel, a record of training given at the simulator, emergency procedures, test procedures for key items, tagging procedure and example work plans for maintenance on high pressure injection system and auxiliary feedwater, and operational staff comments and feedback obtained through interviews.

Available documentation was not sufficient for the team to audit this portion of the HRA process.

It does not appear that there is a documented systematic classification scheme or prescribed approach." The thoroughness of the analysis, therefore depended largely on the knowledge, skills and experience of the FPL HRA team and their operational support. It is the team's judgement that the qualifications of the FPL HRA specialists and the operational support were excellent. In particular, the team members were impressed with the knowledge, experience and quality ofinformation provided by the lead operations person, John Crockford.

The review team did not identify any obviously risk-significant human actions that were

- missed by FPL. The NRC review, however, is intended not to perform or verify analyses, but to assess adequacy of process.

In this case, did the process used provide reasonable assurance that all important human actions have been identified?

One indicator is the existence of a written, systematic process; another is examples of human actions identified by the HRA team that had not already'identified by the systems analysts and/or findings of the HRA team that led to modifications of the logic structures.

In response to review team questions during the visit, the FPL staff indicated that there had been some specific modifications identified by the HRA team.

Documentation of an example of such findings was requested as one form of demonstration of the effectiveness of the process.

2.2.1.2 Screening There were two fundamental types of human actions considered by the HRA team - MTC and recovery. A screening value of 1.0E-3 or 1.0E-4 per MTC was selected by the HRA team as an estimate of the median for independent MTC errors, depending on whether or not the activity is followed by a full flow operational test to verify that the MTC was properly completed. An uncertainty range of 10 was assigned "arbitrarily'-'. The mean values for MTC

. errors were thus 3.0E-03 or 3.0E-4.

Apparently, no further analysis was conducted.

These values were used for essentially all MTC actions.

Essentially, the selected screening values were used as nominal values.

Response to review team questions regarding the source and/or rationale for selection of these values suggests that the HRA specialist's judgement is the source.

Additional clarification has been requested of the licensee.

11

The licensee HRA staff stated that these values are consistent with values used in other PRAs.

In fact, the values are considerably lower than typically used as screening values.

For example, Ref. 7 recommends a screening value of 0.03.

However, nominal values with appropriate analysis may be shown to be comparable to the values selected for the Turkey, Point analysis.

Since the screening values were, in effect, used as nominal values as well, it is arguable that the Turkey Point values are compatible with values typically used in PRAs.

However, this argument misses the fundamental point that the analysis is much more important than the number.

The analysis that Ref. 7 recommends, for example, could be rather detailed and extensive.

To merely assign values without assessment and justification for using the lower values is inappropriate.

If an assessment was made, it should be

- documented and scrutable by others. Ifthe assessment involved simply the expert judgement of the HRA specialist, the assumptions and rationale used by that expert should be documented.

Screening for the recovery actions was apparently coarse screening with most actions screened at a value of 1.0. Thus, most recovery actions were retained for detailed analysis. -Unlike the MTC actions,

however, a fairly detailed assessment of each action was performed and documented in the Recovery Analysis Work Package (Ref. 5). In fact, the lack of virtually any information on the detailed recovery analysis in the summary provided in the IPE submittal was one of the major problems noted with the submittal. The screening process was documented and is judged to be reasonable.

For complete documentation, it would be desireable to identify specifically any actions that were screened out and were not analyzed in detail.

I 2.2.1.3 Breakdown As indicated above, the Recovery Analysis Work Package documents a fairlydetailed analysis of all recovery actions analyzed, over 30 individual actions.

For each action there are five sheets of information, summarized in Table III. An example of each sheet is provided in Appendix B. While there is no explicit breakdown of actions to lower level actions, and no "task analysis" in typical format, the combined contents of these information sheets, in particular the HFE Record, demonstrate that a reasonably systematic process was used to identify and evaluate performance shaping factors of importance.

The HFE record and the other forms are reasonable and appropriate means of summary documentation for the analysis.

Examination of the HRA Recovery Analysis Work Package after the visit led to several additional questions and request forclarification orjustification, Overall, however, the process used for the HRA recovery analysis is judged to be systematic, 'reasonably comprehensive, and consistent with the state-of-the art and methods used in other PRAs.

For the MTC actions, there appears to have been minimal analysis.

While there is a large package ofdocumentation on the analysis, the package consists primarily ofvirtuallyidentical sheets with the 3.0E-3 or 3.0E-4 values assumed for mean error probabilities.

Most of the other information that would be derived from breakdown'nd analysis of tasks wa's left blank.

12

Table III Data Sheets Included in The HRA Recovery Analysis Work Package Rev.

0 (Ref.

5) 2.

3.

4.

5.

Recovery Analysis Worksheet (RAW)

Summary description and classification of type of human actions; primary input for determination of appropriate HRA method/technique.

Recovery Option Description Form (RODF)

Provides a

narrative description of the

action, identifies the cue for the

action, and documents estimated times required and available.

Recovery Analysis Follow Up Action Sheet (RAFAS)

Addition details of the recovery

action, systems

data, timing data, etc., to aid in quantification.

Recovery Analysis Summary Sheet (RASS)

Documentation of quantitative estimates of error probability,

median, mean, and uncertainty bounds; also provides verification signoff by the Recovery Analysis Task Leader that the appropriate documentation (RASS, RAFAS and RAW) were completed.

HFE Review Sheet Complete summary documentation of event description, event

category, HRA method

used, input parameters, calculated parameters, probability estimates with uncertainty

bounds, and results of intermediate calculations.

2.2.1.4 Representation The level of detail and documentation ofrepresentation of human actions as described in Ref.

6 was minimal for the Turkey Point HRA. The Beta Factor approach using a decision tree, which was summarized very well in the IPE Submittal, is effectively a representation of potential common cause human errors for the MTC actions.

It was judged to be straight forward and effective, and consistent with industry practice.

Independent MTC actions were not analyzed other than to distinguish between those that were and those that were not 13

followed by a full operational test.

Recovery actions were treated primarily using SAIC THERP or the SAIC Time Reliability Correlation (TRC) System.

No "trees" were developed for the THERP analysis; actions were not explicitly broken down to lower level actions.

As suggested above, the HFE Record Sheets and other HRA Recovery Analysis Work Package forms effectively served as documentation of the breakdown and representation of the human recovery actions, to the level of detail that they were analyzed.

The TRC System embodies a representation of the actions quantified.

The mechanics of the application of the TRC system are straight forward and reasonably well documented in either the submittal or references cited in Tier 2

information.

While there are questions for additional clarification/justification that have been submitted to the licensee regarding the technical basis for application of the TRC and of specific data used, the representation is systematic and generally accepted in the industry.

2.2.1.5 Impact Assessment There was no distinct step in the Turkey Point HRA process directly corresponding to this element in SHARP.

The Turkey Point HRA Procedure (Ref. 8), in fact, calls for a series of activities that are analogous to but organized somewhat differently from the SHARP document (Ref. 6). Breakdown and Representation are grouped into "Qualitative Analysis", and concepts parallel to Impact Assessment are incorporated into an activity designated "Integration" that follows detailed quantification.

These differences in organization of activities are not important.

The analysis processes are conducted interactively between'the HRA team and somewhat iteratively.

The important concepts of SHARP, in this case involvement by the HRA team in the incorporation of the human reliability analysis into the systems

analysis, have been retained in the Turkey Point approach.

2.2.1.6 Quantification Methods for quantification of the MTC actions involve essentially arbitrary designation of median values with uncertainty bounds for independent actions plus a decision tree to arrive at Beta Factors for common cause estimates.

The common cause treatment is judged to be effective and consistent.

Additional justification for selection of the 3.0E-3 and 3.0E-4 screening values and effectively using them as nominal values is required.

Quantification of many of the recovery actions was performed using "SAIC THERP". Few details of this methodology, particularly how it differs from THERP (Ref. 9), were provided.

The SAIC TRC System was used to quantify most of the other human actions in the recovery analysis.

TRCs, in general, have been accepted by many in the nuclear human reliability community as a reasonable approach to quantifying error probabilities for some nuclear plant human actions.

There are questions remaining regarding the specific application of the SAIC TRC System to Turkey Point. One issue is the applicability to Turkey Point of the simulator data (including BWR data) forming the basis of the correlation(s).

What did the FPL HRA Team do to assure itself that data used in the correlation(s) are appropriate for Turkey Point?

14

A second issue is the applicability of the correlation(s) to ex-control room actions; The general logic or "theory" behind the use of time reliability correlations seems more applicable to control room situations in which a specific annunciator or cue is presented and an action follows in a relatively constrained environment.

Practical factors affecting timing for ex-control room actions make the direct application ofthis simplified model and ofsimulator data questionable.

The licensee needs to provide justification for application of the correlation(s) for control toom actions.

A third general issue (one that is not expected to be resolved by the licensee, but should nevertheless'e shown to have been'considered) is the difference between simulator data and "real-world" response, whether in control room or ex-control room. While there is no firm technical basis for a quantitative extrapolation of simulator data to actual performance in an accident situation, some assessment should be made, and perhaps some conservatism should be added, by the analyst to account for potential differences between simulator data and expected response at a particular plant.

Several recovery actions were quantified apparently using data directly taken from a Westinghouse report (WCAP-11992).

No explanation of analysis, assumptions, or any site-specific assessment is provided in the HRA documentation provided. This is an unacceptable practice.

An explanation of the technical basis provided by the referenced Westinghouse document is required.

Preferably, a copy of the document should be provided to NRC.

~

Finally, an explanation and justification of the application of the referenced source to Turkey Point HRA Recovery Analysis should be'provided for review by NRC.

2.2.1.7 Documentation The central issue of documentation of the IPE is whether the important assumptions,

inputs,

~

activities, rationale, analytical methods, and results have been recorded adequately and explained for a user of the IPE and for future analysts to understand the technical basis underlying the results.

If,it is true that HRA is based on "expert judgement" more than the rest of the IPE analysis, then it is even more important that the underlying assumptions and judgments be clearly and explicitly documented.

Another important aspect of documentation is the degree to which interfaces have been established to maintain a two-way flow of information between the PRA Group and other FPL organizations to (1) enhance use of IPE/PRA results, and (2) assure continued updating and maintenance of a-"living" PRA.

A number of problems have already been noted above regarding documentation.

While these exceptions need to be addressed, the overall structure for documentation of the HRA is quite good.

The HRA Procedure (Ref. 8) and the HRA Recovery Procedure (Ref. 10) are useful documents appropriately summarizing the planned approaches.

In the BRARecovery Analysis Work Package, and in the IPE Submittal, it would be useful to provide clearer exposition of the model parameters for the models used and the assumptions made in selecting those parameters.

However, the HRA Recovery Analysis Work Package, especially the HFE Record 15

Sheet, is an excellent documentation of results.

Specific items that require attention by the licensee are identified in the recommendations listed in Section 3.0 of this report.

2.2.2 Plant Walkdowns A rather extensive tour of the plant was provided by FPL staff. The operations support staff escorting the NRC HRAreviewers provided exceOent information and background discussions on equipment operation, required operator actions, and various factors potentially impacting human performance.

C Prior to the visit, general areas of the plant and specific operator actions to be addressed during the walkdown had been defined in cooperation with the NRC Front-end analysis contractor. During early discussions with FPL staff at the site, a detailed list of specific items to be examined was prepared by the Front-end analysis contractor, and specific points of interface with the NRC HRA reviewers were identified.

Included in this listing were those items identified prior to the site visit, plus a number of other items in which human action was potentially a significant factor in an accident scenario.

The complete list ofitems is presented as Table 4.

Those preceded by the symbol "H" are the items identified forjoint assessment with the HRA team.

Unfortunately, the front-end reviewer and the HRA reviewer were in separate groups during the plant walkdowns.

The synergism between the two.in examining the same equipment and conditions at the same time would have been beneficial.

However, most of the items in Table IV were reviewed independently by the individual analysts.

Areas included in the walkdown were: the turbine area (not enclosed in a building) and a nearby area containing the two motor driven, non-safety related startup feedwater pumps; the component cooling water (CCW) pump area; the high head safety injection (HHSI) pump room; the Unit 4 residual heat removal (RHR) room; the charging pump room; reactor trip breaker room; 4.16 kV switchgear rooms; the intake structure, which contains circulating water pumps and intake cooling water (ICW) pumps; black start diesels; and, the twin control room and electrical equipment rooms in the control building.

The walkdowns provided a much better sense of the realism of the assumptions used to estimate human error probabilities in the HRA. For example, the NRC reviewers were able to see the hoses available for operators to connect service water to the charging pumps in the event CCW is lost and noted the apparent simplicity of the task.

The reviewers obtained a

-sense of the time required to get through the plant to reach key equipment following an

accident, as well as the routine environmental conditions (noise, heat, lighting, etc.).

Some unusual

features, such as the location of equipment outside of buildings, open to the atmosphere, and the need to walk outside of the plant fence, through security to the black start diesel area, were made much more evident than by simply reading about them.

Physically observing equipment, such as the various types ofbreakers, and looking at in-place procedures located in the actual environment of use helped provide a realistic sense of the operational requirements and human factors afTecting operator performance.

16

Table IV Items Identified for Examination During Plant Visit/Walkdown t a vents 0

0 0

0 0

Screening AFW steam supply line break Multiple SG tube breaks Instrument tube LOCA Core (Assembly) Blockage Transient induced small LOCA due to loss of grid (operator action)

Loss of HVAC Low frequency loss of MFW Low probability PORV faiis to open Low frequency loss of instrument air Success C 'te a

MSLB recriticality AFW to SG (375 gpm?)

Systems/De dencies Q

0 0

0 0

0 0

0 HVAC HHSI EPS, black start diesels MFW isolation RCP seal cooling, hot stages Feed and'Bleed CCW/ICW vent/Faul t rees 0

0 0

0 MSLB SGTR Small small LOCA AFW, HHSI, EPS, CCW, ICW ATWS, rod insertion Containment Filters Isolation.'A'B'urge Loss of coatainment cooling, EQ 0

0 0

0 0

Loss of MFW initiating event PORV fails to open Loss of instrumeat air ICW valve Diesel Generator data

]kecovcrr

~Sepia Discussion of dominaat actions ASCII files for fault trees Alarm procedures IS LOCA Loss of ICW Loss of instrumeat air Calculations 17

2.2.3 Visit to Training Facility and Interviews with Training and Operations StaA' The visit to the simulator facility and interviews with training and operations staff conducted during that visit were extremely useful.

They provided a wealth of information on control room human factors engineering, training capability and practices, operations support for the IPE, procedures, staffing and other design and personnel/organizational factors important to human performance.

Also, they confirmed that there had been fairly substantial involvement of the training and operations staff in supporting the IPE and, to some extent, applying results of the IPE.

After a brief introduction and overview, the team received a walkdown of the simulated control room, which gave a good general sense of the control room layout, alarms, control room staffing and responsibilities, accident management information displays, and other human factors considerations.

The simulator is a replica of Unit 3. Some visual sense of simulation of the twin control room for Unit 4 is obtained by use of a mirrored one-way viewing wall behind which is the simulator control station.

The simulator is a recent vintage full-scope, high fidelity simulator made by CAE. Simulator training and data collection capabilities, and apparent use of those capabilities by the FPL training staff, were impressive.

During the simulator walkdown it was noted that, while the overall control room layout for the two units is mirror image, some of the. individual panels are not.

Since there is some shared use of operators between units, the potential for operator error related to this human interface design issue probably should have been examined in the HRA. It is not clear that it was considered.

Another more general issue related to the dual unit control room, is the fact that the simulator simulates only Unit 3. While there certainly is no a priori reason to assume that there is a significant impact on training effectiveness, it seems the issue should at least be addressed in the HRA, and that some rationale should be provided as to why it is not significant. These two issues are examples ofsite-specific consideration of general human factors issues that could have been more explicitly addressed in the HRA.

The review team observed a requalification training session involving a fullcrew responding to an abnormal/emergency event sequence.

A number. of positive features of training practice were observed.

For example, the floor instructor is totally dedicated to running the training session, observing trainees, briefing and debriefing the crew, etc. Allsimulator operations ate conducted from the simulator control station by a dedicated simulator operator. Full audio and video recording of exercises and use of those recordings in trainee debriefings appears to be routine practice.

Documentation on simulator design and acceptance testing was obtained and reviewed after the site visit. A particular training sequence was randomly selected, and a detailed lesson plan for that sequence was produced from computer files immediately.

That material also was reviewed after the site visit.

=Copies of Emergency'Operating Procedures and sample maintenance/calibration procedures applicable to the HRA were obtained and reviewed after

e

the site visit. The team received, in addition to the observation and information on design and use of the full-scope simulator, a tour of the remainder of the training facility, including a scale model of the plant used for familiarization, a trainer that employs a "see through" model of the plant with dynamic simulation of transients, and "laboratories" with hands on operations and maintenance training on components.

r Associated with the visit, discussions were held with the lead operations support staff member supporting the IPE as well as with training center management and the simulator operator.

These interviews provided additional background on plant and control room design, training practice, and procedures, and on the active involvement of these individuals with the IPE.

Much of the operations support for the IPE was provided by training staff who had previous operating experience.

Training management was not only actively aware of and supporting the IPE effort but demonstrated a clear intent to develop practical applications for the results.

One area identified was the use of IPE results to help "design" and/or select malfunctions and training sequences that are important to risk as well as realistic and practical training vehicles.

2.3 Tier 2 Documentation Review and Final Report After the site visit, the review team requested and received materials affiliated with the HRA.

The final report was prepared after reviewing all available materials.

2.3.1 Review of Additional Tier 2 Documentation A sizeable package of additional detailed information was provided by the licensee and subsequently reviewed.

This included information directly associated with the planning, organization and execution of the HRA, with documentation of results, and with important supporting information, such as the EOPs. Allof this material was reviewed after the site visit, and was a factor directly or indirectly in evaluating the overall licensee response to Generic Letter 88-20. A listing of additional Tier 2 material obtained during the site visit and subsequently reviewed is provided in Table V.

2.3.2 Final Report As suggested above, this final report describes the first two Step 2 review activities and summarizes the findings, conclusions and results from the combination of all 'activities.

Preparation of the final report involved organization and, inevitably, additional assessment and reassessment of information and earlier "findings". Several "open" items remain for which additional interaction with the licensee is expected.

However, the overall conclusion is that with regard to the HRA portions, the licensee has met the general intent of Generic Letter 88-20.

19

Table V Tier 2 Documentation Reviewed After the Site Visit 1.

Human Reliability Analysis Procedure (SAIC-139 040),

Rev.

0, December 15, 1989 2.

Recovery Analysis Procedure (SAIC-139-90-060),

Rev.

0, December 3,

1990 3.

Recovery HRA Procedure (SAIC-139-90-061),

Rev.

0, 4.

HRA Interview/Raw Data

Document, Draft A,'lorida Power and Light Company, April 3, 1990 5.

Recovery Analysis Work Package, Rev. 0, Florida Power and Light Company 6.

Turkey Point Plant Simulator Exercise Guide, RCP Seal Failure / Steam Generator Tube Rupture, Rev.

0 7.

8.

Turkey Point Nuclear Plant Emergency Operating Procedures (over 40 procedures; reviewed several in

detail, comparing to statements in HRA; reviewed overall logic path through procedures for some events; examined for general human factors practice Sample Maintenance,

Test, Calibration Procedures Pressurizer Level Wide Range Instrumentation, Steam Generator Level (Narrow Range)

Protection Instrumentation Set II Channel Calibration, RCS Flow Instrumentation Calibration Protection Set I 9.

Turkey Point Simulator Initial Certification, Volume I and Volume II, Florida Power and Light Company, December 31, 1990 10.

Surveillance/Preventive Maintenance risk'Management for Turkey Point Units 3

& 4, Florida Power and Light

Company, August 16, 1991 11.

Responses of FPL staff to NRC questions submitted prior to the site visit (received and discussed-during site visit) 20

3.0 CONCLUSION

S AND RECOMMENDATIONS Subsequent to the site visit, the review team found five areas where improvement needs to be made and two areas which were generally acceptable with reference to Generic Letter 88-20.

Following Section 3.1, recommendations for correction of these inadequacies are described.

3.1 Findings and Conclusions r

The above mentioned areas are elaborated upon in Sections 3.1.1 through 3.1.7. Within these assessment paragraphs, specific conclusions made by the review team are documented.

3.1.1 Licensee Met the General Intent of Generic Letter 88-20 It is our assessment that the licensee has satisfied the general objectives pertinent to HRA identified in Generic Letter 88-20, and thereby has met the intent of the Generic Letter with re ard to the HRA ortions of the IPE. There are a number of improvements to the HRA portions of the IPE submittal an'd to supporting documentation that are necessary.

And there are some points of clarification for which the licensee needs to provide additional information to NRC.

However, the documentation reviewed and the interviews conducted during the site visit, taken as a whole, demonstrate that the licensee has:

gained an appreciation'for human performance contributions to and impact on Turkey Point severe accident behavior; confirmed and increased its understanding of the importance of human performance in the most likely severe accident sequences that could occur at Turkey Point; and gained a more quantitative understanding of the impact of human performance on the overall probabilities of core damage and fission product release.

3.1.2 Analytic Processes Used for HRA Were Generally Acceptable Overall the analytic rocesses used h the a licant were ade uate to meet the ob'ectives of the Generic Letter and were consistent with rocesses used in other PRAs Several recommendations are made in Section 3.2 below for additional clarifying information or documentation of analytic methods and specific analyses conducted.

However, the overall HRA process appears to be satisfactory, and no methods were identified which are extraordinary or unusual with respect to typical industry practice.

3.1.3 HRA Summary in the IPE Submittal is Inadequate A revision to the ke HRA ortions of the IPE sub'mittal is re uired for the submittal to accuratel reflect the anal sis that was conducted.

Sections 3.4 and 3.5 of the IPE submittal (Ref. 4) do not reflect the actual analysis performed for MTC human actions nor for recovery human actions.

The analysis of human actions implied by Figures 3.4-2 and 3.4-3 and discussed in paragraphs of Section 3 4 of Ref. 4, were, according to the HRA specialist interviewed, not actually performed.

Instead the mean values described as screening values were used for the MTC actions.

Screening for the HRA recovery was discussed in the IPE 21

submittal, but the fairlyextensive qualitative and quantitative analysis represented by the HRA Recovery Work Package was barely addressed.

A number of less significant discrepancies in numerical values reported in the summary vs. values actually used in the analysis were noted during the discussions with the HRA team members.

Apparently, these were simply problems associated with the use of earlier versions or incorrect versions of the summary values.

Other significant changes to the IPE submittal and/or Tier 2 documentation may be required as the licensee responds to recommendations made in Section 3.2.

3.).4 Documentation of Site-Specific Analysis Needs Improvement r

There is a lack of documented anal sis showin how non-lant-s ecific data was inter reted or ad'usted for s cific conditions at Turkey Point.

There are a number of general statements in the IPE Submittal and in supporting documentation that consideration

= was given to plant-specific information, design features, procedures, administrative controls, etc.

There is also anecdotal information from the interviews conducted at the site that indicates reasonable attention was given to site-specific issues.

It is clear that there was strong operational support available to the IPE/HRA team.

Further, there was substantial evidence from the interviews with HRA specialists, operational staff, and IPE systems analysts that theie was good communication and interaction among the various disciplines and areas of expertise on the team. It is our belief that a considerable effort was made on the part of all of th'e analysts to consider Turkey Point design and operational factors and to not blindly use generic input.

However, this interaction and the assumptions and evaluations made are not well documented.

Examples of items referred to in general but not addressed very specifically in documentation include man-machine interface, symptom-based procedures, and focus of the training process.

A number of additional specific items have been identified for which documentation is inadequate to demonstrate site-specific adaptation of "generic" information or data.

These are cited specifically as recommendations in Section 3.2.

3.1.5 Personnel Changes Impacted HRA Knowledge Acquisition, Illustrate Need for Thorough Documentation Chan es of kev ersonnel, in particular the departure of the lead FPL staff member forHRA after completion of the analysis and prior to preparation of the IPE submittal, are believed tore resenta tentiall si ificantdetrimenttoFPf."internalizin "theknowled eand ex erience ained from the HRA.

The absence of the FPL lead HRA team member certainly made. it difficultto assess the degree of understanding and direct involvement by the licensee (vs. the subcontractor HRA specialists).

Responsible FPL management (within the PRA group) recognize the need for strengthening in-house capabilities for HRA, though specific actions to do so were not noted.

An individual who is very knowledgeable of plant operations (though not human factors or HRA) was designated as lead near the end of the IPE 22

effort. He was apparently responsible for much of the IPE submittal summary of the HRA effort, and his lack of involvement throughout the process may be part of the reason for the considerable mismatch between the submittal summary and the actual work.

The situation was exacerbated by the fact that there was also some discontinuity between the two SAIC HRA specialists, who performed the bulk of the analysis.

The individual responsible for the analysis of the MTC human actions was involved in the project earlier in its history. He developed the approach for analysis referred to in Fig. 3.4-2 and Fig. 3.4-3 of Ref. 4. Apparently, after his departure from the project, it was determined to not employ that method, though references to it still existed in the draft version of the submittal (Ref. 11).

That analyst was not available for the site visit. Although both HRA specialists are members of the same corporation, there did not appear to be extensive communication between them.

Presumably, accurate documentation ofthe actual analysis still can be developed by interaction among the two HRA specialists, the current FPL HRA lead, and supporting members of the Turkey Point IPE team who played a role in the HRA.

The purpose for citing this issue is not simply to point out personnel problems, and certainly not to criticize individuals involved. These facts illustrate clearly the need for comprehensive, thorough and detailed documentation of not only the results of the IPE but also (perhaps more importantly) the underlying assumptions end rarienuie Ther.e will ~alwa a be personnel changes.

There will always be less than perfect verbal communications among participants and less than perfect memories about what was done and why. It is critically important to capture the substance of the assumptions, rationale, data sources, interactions and discussions, intermediate conclusions, etc., so that (1) the analysis is scrutable and traceable by users of the IPE results, and (2) when additional analysis, such as Accident Management

Planning, needs to make use of the knowledge gained about. human performance in the plant, it willbe available and explicit. This basic knowledge and understanding of human performance will likely be in the long run more valuable than the quantitative results of the HRA.

3.1.6 Clarification of Some Methodology is Required Additional in ut from the licensee re ardin some of the underl in assum tions methods and data sources used in the HRA is re uired.

While the overall structure for documentation of the HRA that is laid out in the HRA procedure appears to be quite satisfactory, there are specific lapses in documentation and additional information needed to justify application of certain data sources which need to be resolved prior to final acceptance of the IPE by NRC. Significant items that require additional input, perhaps improved documentation, and possibly some further analysis are identified in the recommendations in Section 3.2 below.

23

3.1.7 FPL or anizational units in a lications of results of the PRA/IPE rocess.

Two notable examples identified during the NRC site visit were a study on Maintenance Risk Management and the overall interaction with the training staff, who identified,-use of IPE results to help select malfunctions/ scenarios for training as one practical operations use of the results.

Commendable Progress Has Been Made Toward Application of IPE Bah and Involvement of Groups Outside of PRA/IPE Team.

The licensee's PRA rou has made some commendable advances toward involvin other 3.2 Recommendations The followingrecommendations are based on review of all applicable materials and findings.

3.2.1 Recommendations Regarding Closure ofOpen Issues and Final Acceptance of the IPE Submittal.

It is recommended that the following actions be considered by the licensee before final acceptance/approval of the IPE Submittal:

1.

The HRA ortions of the summ discussions in the IPE Submittal Ref.

4

'rimaril Section's 3.4 and 3.5 should be rewritten to accuratel reflect the anal sis tha't was actuall erformed.

2.

The licensee should be uired to recisel document the technical basis for assumed screenin values of 3.0E-3 and 3.0E-4 for re-initiator actions.

These values are low compared to screening values typically used in PRAs, and are more representative of nominal values.

The licensee did, in effect, use these as nominal values, since no analysis was done beyond the screening.

The level of analysis conducted, or at least discussed in the IPE documentation, does not appear to justify use of these nominal values.

Ifthe basis is a source document or published reference, that reference should be provided by the licensee and reviewed by the NRC. If the basis is expert judgement of the HRA specialist, a concise summary of" those judgments and any related assumptions, analysis or extrapolation of generic data should be provided in the revised HRA summary sections in the IPE submittal.

The licensee should identify actions taken by its staff to evaluate and understand the specialist's judgement.

3.

The licensee should be r uired to rovide further demonstrafion to NRC of site-ecific consideration of human actions.

This demonstration should be requested at two levels: (I) summary discussion in the IPE submittal of site-specific issues addressed and 24

0

general findings about the important factors impacting human performance, and (2) specific responses, not necessarily included in the IPE submittal, to NRC issues raised during the audit.

Requests for further demonstration/documentation should include:

1)

Exam les of an s ecific anal sis done of trainin human-machine interface s m tom based rocedures or anizational factors and other basic "human factors issues" that were used in the HRA or in some wa im acted HRA results.

The HRA Interview/Raw Data Document (Ref.

12) provides reasonably good documentation of interviews conducted with operations staff, and there is evidence that certain human factors issues were considered.

There are a

number of general comments in the IPE Submittal and supporting document recognizing of the importance of considering these factors in order "to make the HRA as realistic as possible," and there is an implication that many have been considered.

Further, the experience and knowledge of the HRA specialist(s) and the obvious plant knowledge of the HRA support team suggest that such factors would have been considered.

However, there is little documentation of how the HRA specialist used the information obtained in the qualitative and quantitative analysis to come up with specific judgments.

This documentation is highly desireable for future use of the IPE/HRA results, and continued updating of results in a "living PRA".

2)

Discussion and 'ustification for the decision to collect essentiall no site s ecific data as note in Section 2.0 of the Turke Point HRA Procedure ef. 8

. In lieu of plant specific data, the HRA used expert judgment to modify "generic" or "non-plant-specific" data forTurkey Point conditions, personnel, etc.

The licensee staff should illustrate to NRC what it did to assure that judgment of HRA specialists was appropriate for Turkey Point.

While there is anecdotal evidence ftom interviews that there was good interaction between FPL operations specialists and the SAIC specialists, there is little auditable evidence that the FPL staff rigorously assessed the technical judgments of the HRA specialist.

3)

Additional information and 'ustification for a lication of the SAIC Time Reliabili Correlation S stem to Turke Point Recove Anal sis. The technical basis for application of the data base for the TRC system to Turkey Point should be 25

provided.

Points that should be addressed include:

(a) application of the TRC to ex-control room

actions, (b) application of data from other simulators (including BWR simulators) on different events, using "event-based" rather than symptom based procedures, different operators, different training, etc., (c) extrapolation of simulator data to "real-world" post-accident

response, and (d) application of the TRC to any action when the time available greatly exceeds the time required for the action.

While it is true that these points are not completely resolved by the'human factors community and are not expected to be fullyresolved by the licensee's response, the important point is that the licensee demonstrate its recognition that differences may exist, whether or not such differences may be significant, and that the licensee has some systematic approach for considering site-specific differences.

,4)

Ex lanation of site-s ecific assessments in uanti in error robabilities for certain recove actions usin the reference source cited as "WCAP-11992". No information about this referenced source of data is provided in the submittal or in available Tier 2 information.

A copy of the reference and a discussion of technical basis for applicability to Turkey Point analyses should be provided to NRC.

5)

Demonstration that ractical considerations ertinent to Turkey Point ex-control room o erations e..

re uirements for availabili of buildin ersonnel were ad uatelv considered in the anal sis of ex-control room recove actions.

For those ex-control room recovery actions quantified using the TRC System, some data was based on estimates, "guesses", and some on actual walkdowns. Even in the case of the plant walkdowns, it is not self-evident that practical factors of an actual accident situation were fully considered.

One example is that it appears that administrative requirements permit a ten-minute delay before the auxiliary nuclear operators are physicaOy at the point of receiving instructions and initiating action.

Another is time delays that may occur due to post-accident radiological control and HP requirements may not have been'adequately considered."

Each of these points individually may be non-significant; or it may,be that indeed they were considered but not completely documented.

The overall point is that the licensee demonstrate thoughtful and systematic consideration of realistic practical

, issues affecting human performance in post-accident ex-control room actions.

General statements, or simply stating there was 26

a timed waikdown does not provide sufficient assurance that this was the case.

4.

The licensee should rovide additional clarificatio ex lanation of methodolo and anal sis in two areas: (I) interaction of HRA and systems analysts duiing the",definition" phase to describe the systematic process used to assure that all potentially risk-significant human actions were included in the IPE logic structures, and (2) clarification of the differences between "SAIC 'PERP" and THERP (Ref.9).

27

References 1.

USNRC letter to AllLicensees Holding Operating Licenses and Construction Permits for Nuclear Power Reactor Facilities, "Individual Plant Examination for Severe Accident Vulnerabilities - 10 CFR 50.54(f)," Generic Letter 88-20, November 23, 1988.

2.

USNRC "Individual Plant Examination: Submittal Guidance,"

NUREG-1335, Draft for Comment, January, 1989.

3.

USNRC, "Draft Step 2 Review Guidance Document," (Unpublished).

4.

"Turkey Point Units 3

and 4 Probabilistic Risk Assessment Individual Plant Examination Final Report," Florida Power and Light Company, June 25, 1991.

5.

"Recovery Analysis Work Package, Turkey Point Plant Probabilistic Risk Assessment,"

Revision 0, Florida Power and Light Company.

6.

Electric Power Research Institute, "Systematic Human Action Reliability Procedure (SHARP)," EPRI NP-3583, June, 1984.

7.

Sandia National Laboratories, "Accident Sequence Evaluation Program Human Reliability Analysis Procedure," NUREG/CR-4772, February, 1987.

8.

"Human Reliability Analysis Procedure for Florida Power &Light Company Turkey Point Units 3 &4 Probabilistic Risk Assessment Project," Procedure SAIC-139-90-040, Science Applications International Corporation, December, 1989.

9.

Sandia National Laboratories, "Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications (Final Report)," NUREG/CR-1278-F, August, 1983.

10.

"HRARecovery Analysis Procedure for Florida Power &LightCompany Turkey Point Units 3 & 4 Probabilistic Risk Assessment Project," Procedure SAIC-139-90-061, Science Applications International Corporation, "Human Reliability Analysis Interim Report Turkey Point Plant Probabilistic Risk Assessment Revision 0," Florida Power and Light Company, October, 1990.

12.

"HRA Interview/Raw Data Document'urkey Point Plant Probabilistic Risk Assessment," Draft A, Florida Power and Light Company, April 3, 1990.

28

0,

APPENDIX A PROCESS REVIEW FOR LICENSEES USING SEEP

IPE SUBMITTALREVIEW HUMAN RELIABILITYANALYSIS PROCESS REVIEW FOR APPLICANTS FOLLOWING, SHARP

1. 0 DEFINITION Explain the inputs, activities, rationale/rules, and outputs of the process

~ used to ensure that key human actions have been included in the'logic structures (fault trees, event trees, etc.) used for the IPE.

1.1 What information sources were used to identify key human actions?

1.2 What were the qualifications of the analyst(s) responsible for identification ofkey human actions?

1.3 Describe the process used to examine event/fault trees for human interactions, and provide an example of its use. Was a systematic classification scheme used? Ifso, what was it, and what evidence is there that it was effective?

1.4 1.5 Was there a list prepared ofredundant components susceptible to common influences of humans? Ifso, provide that list and some examples of findings.'ere there human actions identified from this process that had not previously been identified by systems analysts, or were there any modifications to event/fault trees as a result of applying this process? Ifso, provide some ex'amples.

1.6 Were there human actions with the potential to improve system performance, reduce the likelihood of system failure, or mitigate the impact of system failures identified from this assessment?

Ifso, provide examples.

2.0 SCREENING Explain the inputs, activities, rationale/rules, and outputs of the process used to reduce the number of human interactions to those that. potentially might affect the safety of the plant.

2.1 What technique was used to rank and select key human interactions for detailed

~ analysis?

2.2 What was the "cut-off"criterion, and the basis for its selection?

2.3 2.4 Provide some examples of the application of the screening technique to select human actions for more detailed assessment.

Who was involved in selection and application of the screening process?

3. 0 BREAKDOWN Explain the inputs, activities, rationale/rules, and outputs of the process used to breakdown the descriptions of human actions into tasks/subtasks or otherwise amplify the qualitative description of the key human interactions (task analysis and identification of performance shaping factors or influence factors).

3.1 Identify the analytical process used to produce more detailed task information and.

identify influences on human performance (performance shaping factors).

3.2 How was itdetermined which influence (performance shaping) factors were important and which were not?

3.3 Identify sources ofinformation on generic and plant-specific operating experience, and pr'ovide examples of use of these sources to identify possible mishaps or corrective actions.

.~

3.4 Provide examples ofresults of this analysis.

3.5 Who performed this analysis?

3.6 Provide an example of any structured process or aids used for walk-throughs or talk-throughs with plant operations/ maintenance personnel.

Include examples of results of application of that process.

3.7 For time-critical actions, how were time available and expected (average and bounds) time ofperformance determined?

Provide examples.

4. 0 REPRESENTATION Explain the inputs, activities, rationale/rules, and outputs of the process used to represent the key human interactions for subsequent quantification and examination of impact to event and fault trees..

4.l Identify the various forms ofrepresentation used in this IPE. Provide an example ofeach representation forkey human interactions examined in this IPE.

4.2 Explain the rationale by which the appropriate form was selected for each key human interaction.

4.3 Show how availability ofdata sources was factored into the decision on selection of representation.

4.4 Who selected and constructed the representations?

4.5 Explain the rationale used to develop the representations ofhuman actions. Provide

~

examples of any questions or other aids used.

4.6 Were any new systems or equipment failure modes identified? Ifso, provide examples.

4.7 Explain how dependencies between redundant human actions were treated.

5.0 IMPACT ASSESSMENT Explain the inputs, activities, rationale/rules, and outputs of the process used to assess and incorporate impacts of insights derived from the representations of human interactions.

5.1 Explain how human interactions were grouped, screened and incorporated into the system analysis at this point.

5.2 Show how the followingkinds of impacts would have been identified ifthey exist:

~

Changes to equipment reliabilityestimates

~

New initiating events identified-

'dditional common~use links identified

~

Modifications to the logic ofevent/fault trees

~

Changes to quantitative values in sequence frequency or fault tree data:

5.3 Were any new key human interactions identified as a result of this assessment? If so, provide examples.

5.4 Were there any adjustments or modifications to event/fault trees as a result of this

, assessment?

Ifso, provide examples.

5.5 Who performed this assessment?

6. 0 QUANTIFICATION Explain the inputs, activities, rationale/rules, and'utputs of the process used to estimate probabilities of success/failure of human interactions.

6.1 Identify all data sources used to obtain basic HEP data, and provide examples of human interactions for which probabilities were estimated using each source.

6.2 For each "generic" data source, explain how the basic data were interpreted and adapted for site-specific analysis.

6.3 Identify site-specific data sources, including any site-specific experience base, simulator data, expert judgement, walk-throughs, etc. Bom which HEPs were determined'.4 Using several of the most important human interactions as examples, state the key dependencies, uncertainties, and sensitivities, and show how they were addressed.

6.5 Who was involved in quantification, and what were their respective roles?

3

7. 0 DOCUMENTATION Explain the inputs, activities, rationale/rules, and outputs of the process'sed to assure

accurate, complete, and useful documentation of the complete HRA process.

7.1 List all documentation ofthe analysis maintained by the plant.

7.2 Identify responsibilities for maintaining, updating, or periodic review ofHRA information.

7.3 Identify any interfaces, i.e., other assessments, information sources, programs or organizations that have used or willuse (or contribute to) the information obtained fxom the HRA.

7.4 Explain how "lessons learned" during the analysis, as well as final results, are communicated to plant'personnel.

7,5 Identify any impacts to training, procedures, or other "human performance" related aspects ofoperations that resulted directly or indirectly &om the HRA.

7.6 Explain how results of the HRAwillbe used to develop accident management plans, strategies, procedures, etc.

APPENDIX B EXAMPLES OF HRA RECOVERY ANALYSISWORK SHEETS

Appendix A-17 Page 1 oE 9 U3OPS ICOOL FAILURETO ESTABLISH LONG TERM CORE COOLING S I LOCA's

Appendh A-17 Page 2 of 9 INDEXAPPENDIX A-17 Cover Page Index General Information RU3S1CD3-1 RU3S1CD3-1 RU3S1CD3-1 RU3S1CD3-1 RU3S1CD3-1 RAW FORM RODF FORM RAFAS FORM RASS FORM HFE RECORD SHEET 4

5 6

7 8

e

0 Appendix A-17 Page 3 of 9 The events discussed in this Appendix cover the operator actioas required to shift from the injection phase of emergency core cooling to long term core cooling for S1 LOCA's. The tong term cooling phase for S 1 LOCA's is either cold leg recirculation or shutdown cooling.

The action taken willdepend oa the rate of RWST depletion.

Should the RWST be depleted prior to anainiag RHR entry conditions the operator would switchover to recirculation. It is

. generally assumed that LOCA's in this size range (3/8" - 2") will not result in long term containment spray actuation, and therefore the operator will have sufficient time prior to RWST depletion to achieve shutdown cooling entry conditions. Hardware faults of equipment required for recirculation and shutdown cooling were explicitly modelled via the system event trees.

This Appendix only concerns itself with human failures.

A detailed look at the procedures governing the switchover to recirculatioa was conducted in Document No. 163-0142, Rev. 0 "Functional Eveat Sequence Diagrams for Florida Power h Light Company, Turkey Point Units 3 8c 4, Probabilistic-Risk Assessmeat Project".

As a result of this review, it was concluded that the actions most critical to completion of the switchover to recirculation was the manual alignment of the breakers, which provide power to the MOV's required to accomplish switchover.

The breakers of coacera are normally locked open and are located in the Auxiliary Building corridor immediately adjacent to the Auxiliary Nuclear Operator (ANO) eaclosure.

It is the ANO, who upon direction from the control room, willunlock and re-align these breakers.

'Based on walkdowas aad talk throughs with the operators it is estimated that this evolution will take approximately 5 minutes to perform.

The time available to perform this manipulation is estimated to be several hours.

The shutdown cooling mode requires the operator to align the RHR and CCW Systems for shutdown cooling. This is'a routinely performed evolution and is assessed as highly reliable.

The overall'FE associated with establishing long term core cooling is 73x10~.

Appendh A-17 Page4of 9 RECOVERY ANALYSISWORKSHEEI'RAW)

Nonrecovery Event Name U3OP 1CO L I.

Sequences Applied I L CA's Rev. ~

Date ~1~81 2.

List of Recovery Options Coasidered Establishin ion term core coolin via switchover to recirculation or attainin shutdown coolin en conditions 3.

0 'p'R pOp'(S I

I('

'I d~y i &

h dROGF Note:

A Recovery Option Description Form (Attachment 2) is completed for each selected option. The forms are a'ttached to the RAW.

4.

Descriptioa Of Nonrecovery Event:

Sl LOCA failure to establish ion term core coolin 5.

Applicable Cutset Descr'iptioa:

S1 and the o rator fail to establish ion term core coolin 6.

Applicable Modeling Method(s):

Human X Data System Timing Study 7.

Further Information:

None sis Task Leader Aaxhnac 1 4 SAIC.139-90460

0

~5

Appendix A-17 Page50f 9 RECOVERY OPTION DESCRjPTION FORM (RODF)

Recovery Option No.'

Describe action(s):

'Ihe o rator fail to rform the in-control room and ex-control room actions to establi h ion term core coolin Procedure(s) directing action:

EOP-E-1 E -1

'ue for action (i.e., alarms, control room indications, etc.):

R%'ST low level Recovery Option Timeline 0

Estimated time for cue to occur after initiating event has occurred (T1):

~ 18 minutes Estimated time available to complete action(s) after cue bas occurred (T3-Tl):

100 minutes er Ta le 2 Estimated response time (T2-Tl):

minutes able 1 - R nse Time Table (Optional)

Comments:

1 Auxili Buildin a sumed to be habitable 2

Hardware failures modelled directl 'his o tion onl includes human failures.

~

~

.r Aasdtrsmsk 2 to SAIC I39-tOOta

<<0

~

Appendix A-I7 Page 6 of 9 RECOVERY ANALYSISFOLLOW-UP ACTION SHEET (RAFAS)

Rev. ~

Sheet No.

1 Date ~1~1 Nonrecovery Event Name 3OPS1C OL Description ofFollow-up Action Develo d Human Failure Event model HFE Record Sheet attached Modeling Method(s) Applied:

Human X Data System Timing Study N/A Responsible Analysis Task Leader Rec very ysis Task Leader hasdmea 3 lo SAIC.I39-90460

~

W I

Appendix A-17 Pago 7 of 9 Rev.

0 Date ~1~

1 Nonrecovery Event Name:

U3OPS1 L

Non Recovery Event is supported by RAS and RAFASs Yes Outstanding Issues identified RAW are resolved es Nonrecovery Event Probability Mean:

7.5E-6 Median:

2.8E-5th Percentile:

2.8E-7 95th Percentile:

2, E-5 Rec very ysis Task Leader J>Z t Aaachma i to SA1C-139-90460

Appendix A-17 Page 8 of 9 HFE Record Sheet EVENT EVENT IDENTIFICATION U3OPS1 COOL 2.1 2.2 Descriptor Description operators foal to establish long-term cooing S1 LOCAs recirculation is not likelyto be needed; coupled event probably have many hours available 3.

3.1 3.2 3.3 4,,

5.

5.1 5.2 5.3 5.4 5.5 5.6 5.7 6.

6.1 6.2 6.3 6.4 6.4 EVENT CATEGORIZATION Event type Location ot action Failure mode METHOD USED INPUT PARAMETERS Type of behavior Presence ot burden SLI (0.0 to 1.0)

Median time (min), m1 Model enor factor, f1 Model uncertainty error factor, fU Available time (min), t CALCULATEDPARAMETERS Behavior factor SLI tactor Burden factor Adjusted median response time, m Adjusted model error factor, fR post-initiator InCR untimely response SAIC TRC system nominal is 0.5 generic is 4 generic is 3.2 generic is 1.68 rule detault default default default 60 0.50 1.0'1 2.0 3.2

?.

7.1 7.2 7.3 EVENT OCCURRENCE PROBABIUSTIC ESTIMATES Mean occurrence probability 95th percentile 5th percentile 7.5E4 Il.aEW 2.1E-7

Appendix A-t7 Page 9 of 9 HFE Record Sheet, cont.

8.

INTERMEDIATECALCULATIONS Nine parameters are calculated:

8.1 8.2 8.3 e.4 8.5 8.6 8.7 8.8 8,9 BF SLIF BuF fR sigmaR sigmaU sigma mean 0.25 ifverification; 0.5 ifrule; 1 otherwise 2" (1 2 SLI) 2 if burdened; 1 otherwise 4x BFx SLIF 3.2x BuF In (fR)/1.645 ln (fU)/1.645 4 (sigmaR"2+sigma V"2) m x exp (sigmaU"2/2) 0.50 1.0 1

2.0 3.2 0.707 0.315 0.774 2.1 Three normal standard distribution variates, z, are calculated:

e.10 8.11 8.12 z1 Z2 Z3 In (t/mean)/sigma tin(t/m) -1.645 sigma V]/sigma tin (t/m) -1.645 sigmaUj /sigma 4.3288

~3s7229

-5.0631 9.

STANDARDVARIATEAPPROXIMATION Then an approximation Is used to generate the probahlitfes:

a b

c d

e Y

0.319381530 4.356563782 m

1.781477937 a

-1.821255978 1.330274429, 0231641900 9.1 0.499 density(zi7 D

3.4E-5 polynomial(x) pnrduct P

0~0 VS'<

DxP 9.2 9.3 0.537 OA60 3.9EP 1.1E-6 0.252 0.191 9.8EH 2.1E-7 DxP DxP